In the last lesson, we made use of the bufio scanner in order to effectively count words inside of a file, without us needing to worry about the complexity of decoding runes. Despite this however, our new optimised code is restricted to only working with the os.File type. This restriction of needing to use an os.File explicitly is known as "coupling" in software development, and it can sometimes cause issues.

For example, currently we don't have any tests for our CountWordsInFile function. Which is a shame, because we spent a lot of time building them for our CountWords function in order to validate and test that our code was actually working correctly. In fact, it would also be nice to be able to add in the contents of the utf8.txt file we were using in the previous lessons. We can of course make our CountWords test cases work with our CountWordsInFile, but it's a little bit tricky to do so.

First of all we would need to create a new file on the file system which would contain our input values. Then we could pass in this file to our CountWordsInFile function. Although we would actually have to reset the file back to the beginning after we've written to it. Which means we would most likely need to just open it up yet again. However, opening up a file as you know using the os.Open method can itself result in an error. Which means we would then need to check this error in our test and report that the function was failing. Not because of the test case, but instead because there was a problem opening the file. All in all, it just adds some complexity to our test cases.

Instead, a better approach for both our tests and our software quality would be to decouple our code from the concrete type of an os.File. But how can we do that? Well, we sort of had a decoupled solution with our previous CountWords function, which accepted a slice of bytes. Although this isn't exactly a decoupled solution, it was a little more versatile. And definitely a step in the right direction.

So what can we do to decouple our CountWordsInFile function? Well, if you remember back to the last lesson where we created both the bufio.Scanner and bufio.Reader types, the actual parameter that these two functions expect isn't an os.File. Instead, it's an io.Reader. Which, if we go ahead and look at the Go documentation for, is an interface that wraps the basic read method, of which the os.File has. By accepting this interface, it allows both the NewReader and NewScanner methods to accept more than one single type. For instance, they can accept a file as we're doing here, but they can also accept a slice of bytes. Not directly, however, but instead using a type that conforms to the io.Reader but wraps the bytes.

One such type is the Reader type of the bytes package, which implements io.Reader, io.WriterAt, io.WriterTo, io.Seeker, io.ByteScanner and io.RuneScanner. Basically, it implements a number of different interfaces, including one, the io.RuneScanner, which would have allowed us to easily pull out runes from our underlying []byte buffer, without having to write code to do so. We can create a new reader by using the bytes.NewReader function, which accepts a []byte, which we can pass in our data too. This would allow us to create a bufio.Scanner that works with a slice of bytes.

As you can see, by accepting an interface, it decouples the bufio.Scanner and Reader from any concrete types, allowing them to have a much greater level of support. So, let's go ahead and decouple our CountWordsInFile function from the os.File into an io.Reader, which is actually incredibly easy to do so. All we need to do is replace the reference to os.File with a reference to io.Reader, and make sure to import the io package, and that's it. Our code should now run exactly the same as it did before, which if I go ahead and call the go run command, we can see works with the lots_of_words.txt file and should also work with our words.txt as follows.

As you can see, decoupling your code into a base interface makes it incredibly easy. Especially because we weren't actually using the os.File methods, especially because we weren't using any of the actual methods from the os.File, other than the fact that it was an io.Reader. However, now our function name is incorrect. We're no longer counting words inside of a file. Instead, we're counting words inside of a reader. But the actual inReader part of the name is redundant. We can guess that we're counting words from an io.Reader as we're passing it in as a parameter. So really, the function name should be CountWords. But we actually have a conflict. However, this old CountWords function is actually a little outdated. As we've seen, it's not exactly an efficient function, and can run into issues based on byte boundaries.

So let's just go ahead and delete it, and replace the CountWords function with our new one that uses an io.Reader. Then we can go ahead and remove the reference to CountWordsInFile with the new CountWords function, and our code should be good to go. Sort of. Whilst it will work for the go run command, it no longer works for our go test command. This is because we're trying to pass in a slice of bytes as an io.Reader. However, as we saw before, it's actually possible to do this, by making use of the bytes.Reader type, as follows.

However, the bytes.Reader isn't the only type that we can actually use. And in fact, there's actually one better suited to our current setup. Currently, we've defined our test inputs as a string. Mainly because it's a lot easier for us to type these out than it would be if it was a []byte. However, in order to get it to work with our previous CountWords function, we were having to cast these input strings into a slice of bytes, which was a non-zero cost operation. Now, however, it doesn't actually make sense to cast it into a slice of bytes in order to then cast it into a bytes.Reader, as instead we can actually use another type that provides the read method to a string.

This is the Reader type of the strings package, which if we check out the documentation of, implements very similar interfaces as the bytes.Reader that we saw before, with the one that we're interested in being the io.Reader. This means we can go ahead and use the strings.Reader instead of casting our string into a byte slice and then a bytes buffer. To do so, we can go ahead and use the strings package, followed by the NewReader function, passing in our test case input.

Then we can go ahead and pass this reader into our CountWords function. With that, we can now go ahead and test our code using the go test command and everything should work as expected. With that, we've managed to improve the functionality of our count words function by changing the input parameter from a concrete type, the os.File, into an interface, the io.Reader. By doing so, we've not only managed to be able to support the same functionality we had before, which is being able to pass all of the words inside of a file, but we've also enabled this function to support many other types as well, such as strings and byte slices.

Not only this, but because the io.Reader is supported by many other different types, then we're able to extend this functionality to other things we haven't even looked at, such as TCP sockets, UDP sockets, or even other data streams, such as standard input.

Hopefully this shows how powerful decoupling through the use of interfaces can be when it comes to working with Go.

However, as the saying says, "With great power comes great responsibility". And whilst it's a good thing to decouple in order to increase functionality, doing so without any benefit shouldn't be considered lightly, in my opinion. In our case, changing from the os.File into the io.Reader made a lot of sense, as it didn't break any existing functionality and only had a net positive, adding functionality with many other different types.

In any case, that wraps up the end of this lesson, as well as the core implementation of our word counting algorithm, at least for the meantime. Over the next few lessons we're going to be digging into some more advanced CLI-based features and adding it into our code, such as supporting command line arguments in order to choose the file that we pass in, as well as being able to support multiple file inputs. In addition to this, we're also going to add support into standard input, CLI flags and the ability to count even more attributes beyond just the number of words, just to name a few of the features that are coming.

Before we head on, now would be a good time to go ahead and commit the changes to your code, of which the main changes are the main.go file and the main test files. In my case, I also have a bunch of txt files as well, which I'm not going to go ahead and add. In any case, once you've committed your code, we're now ready to move on to the next lesson.