A common operation when it comes to building command line applications is to execute other commands or scripts that are available on your system. We actually saw a couple of examples of this in the last module, when we were adding in our end-to-end tests.

The first example is within the testMain function, where we're building our Go application using the exec.Command function, passing in the command of go with the arguments of build, -o and then our binary name. In addition to this, we were also calling the command for each test, passing in our inputs and capturing the output in order to understand whether or not our application was working.

Whilst end-to-end testing of a CLI application is one use for executing commands, there's actually many other reasons to do so in the context of a command line application. One major use case is to make use of existing tooling in order to provide additional functionality without having to implement that functionality yourself. Making use of tools such as FFmpeg for video encoding, automating scripts for deployment, or even just monitoring system tools and resources. The possibilities are endless!

In fact, for this course, I created my own CLI application, based on the knowledge of everything that I learnt through it. This application allows me to transcode each of the videos that I created into this course, into a variable bitrate for streaming, and would then create an ID for each video, upload them to a Cloudflare bucket, and would store the upload in the database. This was achieved in part by using the exec command.

For example, here I'm calling the ffprobe command in order to understand the framerate of my actual videos. And if we scroll down, we should be able to see another command. Here I'm calling the ffmpeg command with a lot of arguments in order to transcode the video into various bitrates to support HLS streaming, which is now what you're watching the video on.

All of this was achieved by executing other commands within my CLI application, which made it a lot easier to leverage existing functionality that was available on my system, specifically ffmpeg.

When it comes to executing commands in Go, there are a few different ways to do so. However, the simplest way by far is to use the os/exec package, which is the one we used inside of our end-to-end tests. If we take a look at the documentation, you can see that package exec runs external commands. It wraps the os.StartProcess function to make it easier to remap standard input and standard output, connect IO with pipes and do other adjustments.

When it comes to executing commands, this is the package you're going to want to use, as it provides a number of niceties on top of the StartProcess command already.

To take a deeper look at how this package works, here I have a simple piece of example code, which is calling the ls command, passing in the -la flags, and capturing the output using a bytes.Buffer, similar to what we were doing inside of our end-to-end tests. If I go ahead and open up a new terminal window and run the following command using the go run command, you can see it produces a very similar output to say if I used ls -la. In fact, the output is almost identical, apart from the fact that I have colours when it comes to my output, and no colours in the actual command itself. And I have a prefix of the command string, which is what I'm adding in my code.

Whilst this has been the approach that we've used in order to pull out data from the standard output stream of our executed commands, Go actually provides a much simpler approach, which is to make use of the Output method, which returns both a slice of bytes and an error. The slice of bytes is the standard output stream of the actual command. So in our case, rather than creating a bytes.Buffer of standard_out, and rather than assigning it to the command.StandardOut variable, we could instead replace our code as follows.

And we can do our if err != nil check here. If err is not equal to nil, then we can go ahead and just wrap this in a string, standard_out as follows. And we'll get rid of the bytes package. Now if I open up a new terminal window, we should see that this works exactly the same. As you can see however, this is a much more simple implementation. And we could even simplify it further, by instead of capturing the command, we could just capture std_out and error, and call Output on the actual command itself.

In fact, this is so simple that we're going to make a couple of changes to our end-to-end tests in order to make use of it. If we head back on over to our end-to-end test code, and we'll start with the multi-file_bytes.go, you can see here we're doing the same dance of assigning a bytes.Buffer to the standard output stream, and using the Run method of the command. Let's instead go ahead and just replace this with output, err = command.Output. And rather than having this error check here, we can do a simple if err != nil. And for the result, we can just cast it to a string as output.

As you can see, this is a lot more simple. And it does the exact same thing we were doing before, which if we now go ahead and test using the go test command, you can see works as it did before.

In addition to being able to capture standard output more simply using the Output method, we can also capture standard error in a similar way. To show how this works, if I go ahead and use the wc command, and we'll just count the, and we'll pass in a couple of files, such as, let's go ahead and create a new file. So echo "one two three four five" and we'll just pass this to a words.txt, and we should have words.txt. Perfect.

So if we pass in words.txt, and we'll also pass in no_exist.txt, which does not exist. If I go ahead and run this command using the go run command, you can see we get failed to run command: exit status 1, and then we get the output of our actual command, which was a total and a number of words.

And we'll go ahead and actually put a new line here, I think as well, just to make it a little easier to see what's going on. Do a printf, or just a standard print, maybe. Now if I run this again, this will look a little bit clearer. So this is the output of the command, the wc command, and we still get an error due to the fact that we had an exit status code.

However, the wc command is actually producing more output than we're actually seeing. For example, if we call the wc command that we're executing within our code ourselves, such as wc words.txt and wc no_exist.txt, you can see we get this additional output of wc: no_exist.txt: No such file or directory.

However, in our go run command, we're not seeing this. Fortunately, we can easily capture this output by using the following method of CombinedOutput, which runs the command and returns its combined standard output and standard error. Now, if we go ahead and run this code again, we should see it work very similarly to what we had before, which it does.

In fact, I'm going to go ahead and get rid of the error logging, just so that we can see it a little clearer. And if we go ahead and run this code again, you can see that this is the output. And if we run the wc command again, you can see the output is exactly the same. Although we have a slightly different file name, but that's only because of what I passed in.

As you can see, by using the CombinedOutput, it allows you to get both standard output and standard error back in a bytes slice. In some cases, that's going to be acceptable. However, there is sometimes an issue when it comes to using the CombinedOutput method, in that because it merges both standard output and standard error, you lose the information about which is actually which.

Therefore, in most cases, you'll probably want to use the Output method just to capture standard out. And if we go back to using, say, the commands and we can do command.StandardErr = new(bytes.Buffer). Then in this case, we could then use command.Output to get std_out. And if an error did occur, which we can check here, we could then print out the standard error stream if we wanted to. That way we're still keeping both standard output and standard error separate and we can use the information however we want.

With that, we've given a basic overview of how to execute commands when it comes to Go. In the next lesson, we're going to take a look at how we can actually pass input to our commands and some of the various different ways to do so. Then throughout the rest of this module, we'll take a look at how we can manage long-running processes, how we can capture output asynchronously or concurrently, and how we can look at some more advanced process management when it comes to our command line applications.