As we saw in the previous lesson, the Go standard library makes it incredibly easy to work with low-level networking protocols through the use of the net package. However, the majority of the time when working with networking in Go, you'll likely want to interact with a higher-level protocol such as HTTP, which is what's typically used when it comes to working with websites and APIs.

Fortunately, the Go standard library provides another package called net/http, which makes it rather simple to work with the HTTP protocol for things such as spinning up an HTTP web server or creating an HTTP client in order to send web requests. When it comes to command-line applications, most of the interactions with HTTP involve sending requests to websites or APIs instead of setting up a web server, although that does occasionally happen. For this lesson, however, we're going to focus on sending HTTP requests to a third-party service.

That service is HTTP Bin, which provides a number of different endpoints you can use for testing HTTP requests. To do so, let's go ahead and create a new project called HTTP. Again, setting up the main.go using the go mod init command as we would normally do.

Then inside of the main function, let's go ahead and import the net/http module as follows, followed by setting up a main function. The first thing we want to do is send an HTTP request to the /get endpoint. In this case, we're going to be sending a GET request, which is what is accepted by this HTTP endpoint, and it should return some data about the request we send to it.

In order to create a new HTTP request, you can use the NewRequest function of the HTTP package. Although personally, I prefer to use the NewRequestWithContext, which takes a context.Context as its first parameter. As for the context we're going to use, let's go ahead and create one that will respond to the os.Signal interrupt signal. And we can defer the cancel as follows.

Additionally, you can feel free to use a background context here if you want, but having a context that responds to the interrupt is always going to be a better option when it comes to command line applications. Additionally, let's go ahead and capture the return values of this function, which returns a request and an error.

The next parameter we want to define is a method, which is a string, followed by a URL, which again is a string and a body. First, let's go ahead and set the method. You may be tempted to just hard code the string value here of GET, which is the request we're going to send to the GET endpoint. However, it's a much better practice to instead use the method constants that are provided by the HTTP package. In this case, MethodGet.

Next, we can then go ahead and define the address that we want to send a request to. I'm going to define this above just to keep things on the same line, which in this case is going to be https://httpbin.org/get. Then we can pass in the URL as follows. The last parameter we need to pass in is an io.Reader, which represents the body of the request. Because we're setting up a GET request, we don't need to set any body. So we can use the NoBody variable of the HTTP package, which is basically, which acts very similar to nil.

Next, let's go ahead and check to see if an error occurred. And if one did, we'll just go ahead and use log.Fatal line failed to create request. And we now have a request that we can send up to the HTTP Bin service. In order to do so, we can use the http.DefaultClient to send up a request, which does allow us to perform individual requests like GET and POST. But instead, we want to just use the Do method in order to do our request, which in this case is going to be rec.

When it comes to your own code, whilst you can use the default client of the HTTP package, most of the times you'll want to create your own HTTP client, which you can do as follows. This will then allow you to use the Do method as if it was the default client as follows. There are some technical reasons why you may want to use your own HTTP client, which you can find on the following blog post from 2006. I actually have this bookmarked already.

The main reason you don't want to use the default client is because it doesn't have a timeout set, which is very similar to the issue we saw when working with the net package performing socket scanning. Therefore, if you create your own client, go ahead and set the timeout field to be something like a single second, or 10 seconds is probably a little bit more graceful. Additionally, you can also set a context timeout using the context package if you want to. In any case, I'm going to use the default client as it doesn't matter too much if my HTTP request hangs.

In any case, let's go ahead and capture the return values of the Do method as follows. Then we can do a quick if error check and we'll use log.Fatal again. Simple. With that, our request should now be sent. However, it's going to be useful to check the response for any data that we might be sent.

When it comes to an HTTP response, this provides a body parameter, which contains the response body from the actual request. What's nice about Go is this response body is streamed on demand as the body field is read, meaning that you're not loading all of the data before we have access to it. Therefore, we can use one of our familiar io commands in order to copy the data over to stdout.

Let's go ahead and copy it using the io.Copy command. In production, you may want to use something like CopyN instead of io.Copy, as there are some caveats with using the copy function of the io package. This is because if the reader that we're reading from never terminates with an end of file, we'll read from our io.Writer infinitely, which can sometimes happen when dealing with network requests.

In those cases, using the CopyN function allows you to specify a number of bytes that you should read. In our case, using the copy function would be fine, but let's take a look at how we can use the CopyN function just to gain an understanding of how it works. To do so, let's first pass in our os.Stdout, so that we write to the standard output stream. Then we can use the res.Body, which we'll need to make sure that we close using a defer statement just above. And finally, we can pass in the amount of bytes we want to read.

In our case, let's go ahead and set this to be a single megabyte, which we can do using 1024, which is a kilobyte, times 1024. This means we'll only copy a single megabyte from our response body into our operating system standard out. With that, our code should now work.

If I go ahead and run this using the go run command, you should see that we get the response back as follows. The form that this data has come back in is known as JSON, which is a type of encoding when it comes to data. We'll talk a little bit more about encoding in the next lesson. For the meantime, however, you can think of these as kind of key-value pairs. For example, the headers is a key, which has an object inside, containing more key-value pairs. For example, the accept-encoding header, which has gzip, the host header, which has httpbin.org, etc.

Additionally, you can see some other key pairs underneath, such as origin, which is the IP address where my request came from, which is on a VPN, by the way, and the URL that I sent to, which in this case was https://httpbin.org/get. If we take a look at the args parameter, you can see there's no arguments being sent with this HTTP request. When it comes to GET methods, you can typically perform arguments by passing in query parameters. In your browser, this would typically look as follows: passing in the starting the query string with a question mark and then setting a key equal to value.

To send up query parameters with your GET request in Go, whilst you could set them within the actual URL string, this is generally a bad idea, as the values may not be URL encoded. Instead, a better approach is to set them through the URL property of the actual request, which as you can see provides the actual raw query. Therefore, let's go ahead and send up the property of name=John. To do so, let's first obtain the current URL values using the rec.URL property of Query. This will obtain any queries that already exist within the request. Currently, we don't have any, but it's a good idea to use this just in case you do.

Then we can go ahead and call the Add method of this URL values, passing in our key value string. In this case, I want to set the property of name to be John. Now with our URL value set, we can go ahead and replace the RawQuery property of our request URL with the URL.Values.Encode with the result of the Encode method of the URL.Values type. Now, if I go ahead and run this code again, you can see this time the args field has been populated with the URL query that we passed in name=John. You can use this approach to add any URL values that you like, including duplicate values for the same key.

Now if I go ahead and run this code again, you should see these new arguments come through. Here we have age=20, and we have an array now for our name field, with John and Sally. As well as query parameters, another common form of configuration when it comes to sending up HTTP requests is to send up custom headers. If we take a look at the request object, you can see the headers that are already being sent up by our individual client. In this case, the host, which is derived from the URI schema of the actual host, in this case httpbin.org, the user-agent, which in this case is the GoHttpClient version 2, and the accept-encoding, which is gzip.

The x-amazon-trace-id is most likely sent by whatever load balancer the actual service is being loaded behind. This is used for tracing, and it's not something that we're actually sending up. In any case, let's go ahead and send up our own custom header. We can do so by just accessing the http.Header property of the header value, which is a type definition of a map of key-value strings. In this case, we can just simply add, call the Add function, passing in the header we want. In my case, I'm going to set this to be x-user-id. Then we can just pass in some random string. In this case, we can set it to be foobar.

Now, if we go ahead and run this code, you should see the header appear inside of the request's headers, and it does here. As you can tell, setting headers is more simple than setting query parameters. This is because the header value itself is able to be mutated in the request object, whereas the request's URL property requires you to override the raw query value in order to override the query parameters. This is just a bit of a nuance when it comes to working with the Go standard library, but is to do with how the actual implementations of each of these properties are under the hood.

When it comes to HTTP, in addition to sending GET requests, other methods are also used as well to convey different meaning. These include the PUT method, HEAD method, PATCH method, DELETE method, and, perhaps most commonly used, the POST method, which is generally used for creating different resources. We can use either of these methods by just replacing the second property in the NewRequest function, or in my case, NewRequestWithContext.

Let's go ahead and change this to be a POST method, and send up our request yet again. However, when I do so, you can see that the body no longer returns a JSON schema, and instead returns some HTML, letting us know that the method isn't allowed. This is actually an HTTP status code, but it's being sent in the actual body. We can actually pull out these status codes from our request using the StatusCode field, which returns the integer number representing the individual status code. Let's go ahead and print out this status code using the following line. Now when I go ahead and run this code, you can see this time we should see the 405 appear.

You can see we're now printing out the status of 405. By being able to access the status code in this way, it allows us to handle the error in a number of different ways. Typically, when it comes to making HTTP requests, any status codes that fall within the 400 range, so greater than or equal to 499, any status codes that fall in this range represent a client error, which means that the client itself has sent up a request that isn't supported by the server. Additionally, any requests that fall within the 500 range, these codes are used to represent an error with the server, meaning something went wrong outside of our own control.

Typically, when a 500 error code is encountered, it's a good practice to retry the request at some point later on in the future, which can be done by using a retrying client or something similar. We'll take a look at how to do that in the final lesson on this course. For the meantime, however, let's go ahead and fix our code by changing the URL that we're sending our POST request to, to one that's supported. In this case, this is the POST method provided by the HTTP Bin service, so we can set it as follows.

Now, when I go ahead and run this code, again, you can see that our status this time is going to be 200, which means the request was okay. For reference, any status codes that are in the 200 range mean that the request was accepted or successful. The last range that I haven't talked about is the 300 status code range, which is used for things such as redirects or caching. In any case, now that we've sent the POST request, you can see this time we actually have some additional fields. As well as the args field, we also have a data field, a files field, a form field and a json field. We'll take a look at how to encode JSON later on in the course.

For the meantime, however, let's take a look at how we can populate this data field by sending up a POST request. If you'll remember, when we created our request at the beginning using a GET request, we defined the HTTP NoBody field. This is because GET requests typically do not have an HTTP body associated with them. However, now that we're sending a POST request, we can actually send more data up by setting this property. So let's go ahead and send some data up.

To do so, we need to pass in an io.Reader into this parameter field. So in this case, let's go ahead and just set some data field to be strings.NewReader, passing in the value of foobar. Then let's go ahead and replace this reference to NoBody with our data property. Now, if I go ahead and run this code, we should see the data field be populated with our actual data value.

However, you'll notice that the other fields are still set to null. This is because our data field is not being set in any of the encodings that match these other properties. As I said before, we'll take a look at JSON in the next lesson. Files themselves are done slightly differently when it comes to HTTP. Instead, let's go ahead and take a look at how we can send up some form data. In this case, we want to set up some form data of username as equal to string, what we were doing with our query parameters.

And similar to what we did with the query parameters, we can actually do this using url.Values. So let's go ahead and create one as follows. Then we can go ahead and define our values using the values.Add and we'll do username=Thanos, let's say. Now all that remains is to set this form value using the values.Encode method that we saw before when we were setting the query body.

With that, we can go ahead and run our code as follows. And this time we should see our form data also be populated, which it's not. This is because we actually need to set the Content-Type property of our request. To do so we need to add in a new header, which we can do using the rec.Header.Add method. Setting the Content-Type to application/x-www-form-urlencoded. Now, if I go ahead and run this, you can see that our form data is now passed and we're getting more information back from the HTTP Bin service.

With that, we've managed to look at how we can interface with an HTTP API using the net/http package of the standard library. This is the way that I would typically interface with HTTP when it comes to working with Go. However, it's worthwhile covering that the HTTP client and HTTP package do provide a number of convenience functions in order for you to interface without having to create these more complex structures.

For example, while setting the form data this way is, in my opinion, the correct way to do so, we could of course remove all of this code by using the http.DefaultClient.PostForm method, which just takes a URL and the data values as follows. This is a really nice convenience wrapper, but as you'll notice, it doesn't have any of the context handling, which means you'll need to make sure that you set a timeout in the actual request itself. Personally, I much prefer to just use the approach that I showed before, but it's still worthwhile understanding that you can use these helper functions in case you just want to whip something up really quickly.

In any case, feel free to choose whichever approach makes sense to you and the project that you're building. In the next lesson, we're going to build on top of this function by taking a look at how we can both encode and decode data from various different encodings. For example, taking a look at how we can post JSON data up to our endpoint, and how we can also decode the JSON data that we receive back, converting this into a concrete Go type that we could then work with.

I'll see you in the next lesson.