Developing an HTTP Client in Go: From Installation to First Requests

Using APIs to communicate with external services is becoming more and more crucial when creating applications. With APIs, applications can transmit and receive data across a network and communicate with each other. One of the most popular standards for creating and using APIs is REST (Representational State Transfer), which is based on the HTTP protocol.

Go has established itself as a powerful programming language for web development due to its performance, simplicity, and built-in support for network protocols. One of the key tasks that Go developers often need to solve is creating HTTP clients to interact with third-party REST APIs.

In this article, we will help developers who are new to Go and REST APIs build their first HTTP client. We will start with the basics and progress to more advanced topics, such as sending different types of HTTP requests, handling responses, and automating requests. Additionally, we will explore practical examples and best practices to help you create secure and reliable HTTP clients.

Setting Up the Environment

First, let’s set up our working environment. We need to install Go tools, configure a development environment, and initialize a new project.

Installing the Go Compiler

Go supports all major operating systems: Windows, Linux, and macOS. We’ll briefly show the installation process for all of them. Let’s start with  Windows.

Follow these steps:

1. Go to the official Go website.

2. Download the installation package for your operating system (either 32-bit or 64-bit version).

3. Run the downloaded file and follow the installation wizard's instructions.

Verify the installation was successful by checking the Go version.

go version

For macOS, you can either download and run the installer or use a package manager like Brew or MacPorts:

brew install go

Or:

sudo port install go

For Linux distributions, use a package manager:

Ubuntu:

sudo snap install go --classic

Debian:

sudo apt-get install golang-go

CentOS/AlmaLinux: 

sudo dnf install golang

Arch Linux:

sudo pacman -S go

Configuring an IDE or Text Editor

You don’t exactly have to use an IDE (integrated development environment) as Go provides a flexible set of tools for building applications using the command line. 

However, an IDE or a text editor with Go support can still enhance your development experience, making it more convenient and efficient.

Below are some popular options:

• Visual Studio Code (VSCode): A lightweight yet powerful editor with excellent Go support through extensions. This is the editor we use in this article.

• Vim/Neovim: Highly customizable editors with plugin support for Go, such as vim-go.

• Emacs: A powerful and customizable text editor widely used for text editing, with Go support available through various packages and extensions.

If you decide to use VSCode, install the official "Go" extension from the Go development team to enable autocomplete, debugging, and other useful features. To do this:

1. Open VSCode.

2. Go to the Extensions tab or press Ctrl+Shift+X.

3. Search for the Go extension and install it.

Initializing a New Project

Now that your development environment is ready, let's create a new Go project to develop our HTTP client.

Create and navigate to your project directory:

mkdir httpclient && cd httpclient

Initialize a new Go module:

go mod init httpclient

After running this command, a go.mod file should appear, which will store information about the module and its dependencies.

Create and open the main project file using VSCode:

code main.go

If everything is working correctly, intermediate command outputs should look normal.

Open the main.go file in your editor and add the following code:

package main

import (
    "fmt"
)

func main() {
    fmt.Println("Hello, HTTP Client in Go!")
}

Run the program to verify everything is working correctly:

go run main.go

If you have followed the steps correctly, you should see the message:

Hello, HTTP Client in Go!

Now, you have a fully set up Go development environment and an initialized project. In the next chapters, we will start building a full-fledged HTTP client, sending requests to an API, and handling responses.

Sending HTTP Requests in Go

In this section, you will learn how to send different HTTP requests (GET, POST, PUT, DELETE) using Go’s standard net/http library. We will start with basic methods and gradually move on to more complex scenarios.

Creating and Configuring an HTTP Client: GET and POST Requests

Before sending requests, you need to create an instance of an HTTP client. In Go, this is done using the http.Client{} struct.

For this example, we will use JSONPlaceholder, a free test API that provides basic resources accessible via HTTP methods. Such APIs are an excellent solution for testing and understanding how different requests work. No special tokens, registration, or authentication are required — you can run all the code on your local machine to see how it works in practice.

The GET method is used to retrieve data. Here’s how it is implemented in Go using the http.Get() function.

In your main.go file, add the following code:

package main

import (
	"context"
	"fmt"
	"net/http"
	"time"

	"httpclient/client"
)

func main() {
	// Initialize a custom HTTP client
	httpClient := client.NewHTTPClient(&http.Client{
		Timeout: 10 * time.Second,
	})

	ctx := context.Background()

	// Fetch an existing blog post using the custom HTTP client
	blogPost, _, err := httpClient.GetBlogPost(ctx, 1)
	if err != nil {
		fmt.Println("Error:", err)
		return
	}

	fmt.Println("Blog Post:")
	fmt.Printf("  ID: %d\n", blogPost.ID)
	fmt.Printf("  Title: %s\n", blogPost.Title)
	fmt.Printf("  Body: %s\n", blogPost.Body)
	fmt.Printf("  User ID: %d\n", blogPost.UserID)

	// Attempt to fetch a non-existing post
	blogPost, _, err = httpClient.GetBlogPost(ctx, -1)
	if err != nil {
		fmt.Println("Error:", err)
		return
	}

	fmt.Println("Blog Post:", blogPost)
}

Now, create a client.go file inside the client subdirectory and add the following code:

package client

import (
	"bytes"
	"context"
	"encoding/json"
	"errors"
	"fmt"
	"io"
	"net/http"
	"net/url"
	"strings"
)

const (
	defaultBaseURL = "https://jsonplaceholder.typicode.com/"
)

type HTTPClient struct {
	client  *http.Client
	BaseURL *url.URL
}

// Initialize a new HTTP client
func NewHTTPClient(baseClient *http.Client) *HTTPClient {
	if baseClient == nil {
		baseClient = &http.Client{}
	}

	baseURL, _ := url.Parse(defaultBaseURL)

	return &HTTPClient{
		client:  baseClient,
		BaseURL: baseURL,
	}
}

// Create a new HTTP request
func (c *HTTPClient) NewRequest(method, urlStr string, body any) (*http.Request, error) {
	if !strings.HasSuffix(c.BaseURL.Path, "/") {
		return nil, fmt.Errorf("BaseURL must have a trailing slash, but %q does not", c.BaseURL)
	}

	u, err := c.BaseURL.Parse(urlStr)
	if err != nil {
		return nil, err
	}

	var buf io.ReadWriter
	if body != nil {
		buf = &bytes.Buffer{}
		err := json.NewEncoder(buf).Encode(body)
		if err != nil {
			return nil, err
		}
	}

	req, err := http.NewRequest(method, u.String(), buf)
	if err != nil {
		return nil, err
	}

	if body != nil {
		req.Header.Set("Content-Type", "application/json")
	}

	return req, nil
}

// Execute the HTTP request
func (c *HTTPClient) Do(ctx context.Context, req *http.Request, v any) (*http.Response, error) {
	if ctx == nil {
		return nil, errors.New("context must be non-nil")
	}

	req = req.WithContext(ctx)

	resp, err := c.client.Do(req)
	if err != nil {
		select {
		case <-ctx.Done():
			return nil, ctx.Err()
		default:
		}

		return nil, err
	}
	defer resp.Body.Close()

	err = CheckResponse(resp)
	if err != nil {
		return resp, err
	}

	switch v := v.(type) {
	case nil:
	case io.Writer:
		_, err = io.Copy(v, resp.Body)
	default:
		decErr := json.NewDecoder(resp.Body).Decode(v)
		if decErr == io.EOF {
			decErr = nil // Ignore EOF errors caused by empty response body
		}
		if decErr != nil {
			err = decErr
		}
	}

	return resp, err
}

// Check if the HTTP response indicates an error
func CheckResponse(resp *http.Response) error {
	if c := resp.StatusCode; 200 <= c && c <= 299 {
		return nil
	}

	return fmt.Errorf("%s %s: %s", resp.Request.Method, resp.Request.URL, resp.Status)
}

// BlogPost represents a blog post entity
type BlogPost struct {
	ID     int64  `json:"id"`
	Title  string `json:"title"`
	Body   string `json:"body"`
	UserID int64  `json:"userId"`
}

// Fetch a blog post by ID
func (c *HTTPClient) GetBlogPost(ctx context.Context, id int64) (*BlogPost, *http.Response, error) {
	u := fmt.Sprintf("posts/%d", id)

	req, err := c.NewRequest(http.MethodGet, u, nil)
	if err != nil {
		return nil, nil, err
	}

	b := new(BlogPost)
	resp, err := c.Do(ctx, req, b)
	if err != nil {
		return nil, nil, err
	}
	defer resp.Body.Close()

	return b, resp, nil
}

• main.go: Contains the application's entry point, initializes the HTTP client, and performs basic operations.

• client.go: Handles the HTTP client logic, defining its structure, initialization functions, and request methods. This modular approach allows for easy reuse in other projects and makes testing the client independent of the main application.

The problem with http.DefaultClient is that it is a global variable, meaning any changes to it affect the entire program which creates security and stability risks. Besides, http.DefaultClient lacks flexible configuration options, such as setting timeouts, TLS settings, proxies, or cookie management.

By initializing our own HTTP client with http.Client{} and custom settings, we avoid these issues and ensure greater flexibility and security in our application.

The POST method is used to send data to a server. In Go, there are two ways to send a POST request:

1. Post() — Used for sending data in various formats (JSON, XML, binary). Features:

• Requires explicitly setting the Content-Type header (e.g., application/json).

• Data is sent as a byte array ([]byte).

• Allows custom request headers.

2. PostForm() — Optimized for submitting HTML form data (application/x-www-form-urlencoded). Features:

• Automatically sets the Content-Type header.

• Accepts data as a url.Values structure (similar to map[string][]string).

• Simplifies working with form parameters (login, registration, search).

To send POST requests, we need to add functions that allow us to send data to a server. Below, we will implement two types of POST requests:

• CreateBlogPost: Sends JSON data.

• PostForm: Sends form-encoded data.

Copy the following function into your client.go file:

func (c *HTTPClient) CreateBlogPost(ctx context.Context, input *BlogPost) (*BlogPost, *http.Response, error) {
	req, err := c.NewRequest(http.MethodPost, "posts/", input)
	if err != nil {
		return nil, nil, err
	}

	b := new(BlogPost)
	resp, err := c.Do(ctx, req, b)
	if err != nil {
		return nil, nil, err
	}
	defer resp.Body.Close()

	return b, resp, nil
}

Copy the following function into your client.go file:

func (c *HTTPClient) PostForm(myUrl string, formData map[string]string) (string, error) {
	form := url.Values{}
	for key, value := range formData {
		form.Set(key, value)
	}

	resp, err := c.client.PostForm(myUrl, form)
	if err != nil {
		return "", fmt.Errorf("error making POST form request: %w", err)
	}
	defer resp.Body.Close()

	body, err := io.ReadAll(resp.Body)
	if err != nil {
		return "", fmt.Errorf("error reading response body: %w", err)
	}

	return string(body), nil
}

Don’t forget to import the net/url package in client.go.

Now, modify your main.go file to call the CreateBlogPost function:

package main

import (
	"context"
	"fmt"
	"net/http"
	"time"

	"httpclient/client"
)

func main() {
	// Initialize a custom HTTP client
	httpClient := client.NewHTTPClient(&http.Client{
		Timeout: 10 * time.Second,
	})

	ctx := context.Background()

	input := &client.BlogPost{
		Title:  "foo",
		Body:   "bar",
		UserID: 1,
	}

	// Create a new blog post using the custom HTTP client
	blogPost, _, err := httpClient.CreateBlogPost(ctx, input)
	if err != nil {
		fmt.Println("Error:", err)
		return
	}

	fmt.Println("Created Blog Post:")
	fmt.Printf("  ID: %d\n", blogPost.ID)
	fmt.Printf("  Title: %s\n", blogPost.Title)
	fmt.Printf("  Body: %s\n", blogPost.Body)
	fmt.Printf("  User ID: %d\n", blogPost.UserID)
}

After running the program (go run .), you should see an output similar to this:

Working with Other Request Types (PUT, DELETE, etc.)

Similarly to GET and POST, you can send other HTTP requests. 

• PUT is used to completely replace a resource or create it if it does not exist.

• DELETE is used to remove a resource at the specified URL.

To work with PUT and DELETE, use a universal approach with http.NewRequest.

Add the following functions to client.go:

func (c *HTTPClient) PutJSON(myUrl string, jsonData []byte) (string, error) {
	req, err := http.NewRequest(http.MethodPut, myUrl, bytes.NewBuffer(jsonData))
	if err != nil {
		return "", fmt.Errorf("error creating PUT request: %w", err)
	}
	req.Header.Set("Content-Type", "application/json")

	resp, err := c.client.Do(req)
	if err != nil {
		return "", fmt.Errorf("error making PUT request: %w", err)
	}
	defer resp.Body.Close()

	body, err := io.ReadAll(resp.Body)
	if err != nil {
		return "", fmt.Errorf("error reading response body: %w", err)
	}

	return string(body), nil
}

func (c *HTTPClient) Delete(myUrl string) (string, error) {
	req, err := http.NewRequest(http.MethodDelete, myUrl, nil)
	if err != nil {
		return "", fmt.Errorf("error creating DELETE request: %w", err)
	}

	resp, err := c.client.Do(req)
	if err != nil {
		return "", fmt.Errorf("error making DELETE request: %w", err)
	}
	defer resp.Body.Close()

	body, err := io.ReadAll(resp.Body)
	if err != nil {
		return "", fmt.Errorf("error reading response body: %w", err)
	}

	return string(body), nil
}

Modify your main.go file to call these new functions:

package main

import (
	"fmt"
	"net/http"
	"time"

	"httpclient/client"
)

func main() {
	httpClient := client.NewHTTPClient(&http.Client{
		Timeout: 10 * time.Second,
	})

	// Example PUT request
	jsonToPut := []byte(`{"id": 1, "title": "foo", "body": "bar", "userId": 1}`)
	putResp, err := httpClient.PutJSON("https://jsonplaceholder.typicode.com/posts/1", jsonToPut)
	if err != nil {
		fmt.Println("Error:", err)
	} else {
		fmt.Println("PUT Response:", putResp)
	}

	// Example DELETE request
	deleteResp, err := httpClient.Delete("https://jsonplaceholder.typicode.com/posts/1")
	if err != nil {
		fmt.Println("Error:", err)
	} else {
		fmt.Println("DELETE Response:", deleteResp)
	}
}

After running the program (go run .), you should see the following output:

For more complex scenarios, you can configure:

• Client timeouts
• Retry logic
• Custom authorization headers

This section has covered how to create and configure an HTTP client and send different types of HTTP requests. Now, you can move on to more advanced REST API interactions.

Interacting with REST API in Go

Now that we understand how to send HTTP requests in Go, let's explore how to interact with a REST API. We will:

• Create data models to handle API responses

• Convert received data into structured objects

• Demonstrate an example of usage

We will start by sending a request to retrieve a list of posts and processing the received response.

Creating Data Models for API Responses

In Go, API responses are typically processed using structs. Defining structs to store data allows us to handle API responses more conveniently and safely.

Here is an example of a Post struct:

package main

type Post struct {
    UserID int    `json:"userId"`
    ID     int    `json:"id"`
    Title  string `json:"title"`
    Body   string `json:"body"`
}

This struct matches the JSON format returned by the API.

The attributes are marked with JSON tags to ensure correct data conversion.

Converting API Responses into Structured Data

Now, let's send a GET request to the API and convert the response into a Go struct.

Here is the full main.go implementation:

package main

import (
	"fmt"
	"net/http"
	"time"

	"httpclient/client"
)

type Post struct {
	UserID int    `json:"userId"`
	ID     int    `json:"id"`
	Title  string `json:"title"`
	Body   string `json:"body"`
}

func main() {
	// Initialize HTTP client
	httpClient := client.NewHTTPClient(&http.Client{
		Timeout: 10 * time.Second,
	})

	// Fetch post data
	post, err := httpClient.GetBlogPost(1)
	if err != nil {
		fmt.Println("Error:", err)
		return
	}

	// Print post details
	fmt.Printf("Post ID: %d\n", post.ID)
	fmt.Printf("User ID: %d\n", post.UserID)
	fmt.Printf("Title: %s\n", post.Title)
	fmt.Printf("Body: %s\n", post.Body)
}

Modify the GetBlogPost function in client.go:

func (c *HTTPClient) GetBlogPost(postID int) (*Post, error) {
	resp, err := c.Client.Get(fmt.Sprintf("https://jsonplaceholder.typicode.com/posts/%d", postID))
	if err != nil {
		return nil, fmt.Errorf("error making GET request: %w", err)
	}
	defer resp.Body.Close()

	var post Post
	err = json.NewDecoder(resp.Body).Decode(&post)
	if err != nil {
		return nil, fmt.Errorf("error decoding response body: %w", err)
	}

	return &post, nil
}

In this example, we:

1. Initialize the HTTP client

2. Send a GET request

3. Retrieve post data

4. Convert the JSON response into a Post struct

5. Print the post details

After running the program (go run .), you should see output similar to this:

Handling API Responses in Go

In this chapter, we will explore how to process responses from a REST API in Go.

We will cover topics such as checking HTTP status codes, handling response bodies, and managing and logging HTTP errors.

Checking HTTP Status Codes

An HTTP status code indicates the result of an HTTP request. It helps determine whether an operation was successful or if an error occurred.

Two of the most common HTTP status codes are:

• 200 (OK) indicates that the request was successful.

• 404 (Not Found) means the requested resource does not exist.

The main.go file:

package main

import (
	"fmt"
	"net/http"
)

type Post struct {
	UserID int    `json:"userId"`
	ID     int    `json:"id"`
	Title  string `json:"title"`
	Body   string `json:"body"`
}

func main() {
	httpClient := NewHTTPClient()

	// GET request
	response, err := httpClient.Get("https://jsonplaceholder.typicode.com/posts/1")
	if err != nil {
		fmt.Println("Error:", err)
		return
	}
	defer response.Body.Close()

	if response.StatusCode != http.StatusOK {
		fmt.Printf("Error: Received non-200 response code: %d\n", response.StatusCode)
		return
	}

	fmt.Printf("Received a successful response. Status code: %d\n", response.StatusCode)
}

In the client.go file, we will define a  simple Get() method:

func (c *HTTPClient) Get(url string) (*http.Response, error) {
	resp, err := c.Client.Get(url)
	if err != nil {
		return nil, fmt.Errorf("error making GET request: %w", err)
	}
	return resp, nil
}

In this example, we send a GET request and check the response status code. Depending on whether the request is successful or not, you will see different output messages.

Processing the Response Body (XML)

Once we have checked the HTTP status code, we can move on to processing the response body. Most APIs return data in JSON format, but some may use XML or other formats. Previously, we demonstrated handling JSON responses. Here, we will cover XML processing instead.

Since JSONPlaceholder does not support XML, we will use a different public API in main.go that can work with XML:

package main

import (
	"fmt"
)

type Post struct {
	UserID int    `json:"userId"`
	ID     int    `json:"id"`
	Title  string `json:"title"`
	Body   string `json:"body"`
}

type Response struct {
    XMLName xml.Name `xml:"objects"`
    Objects []Object `xml:"object"`
}

type Object struct {
    ID        int    `xml:"id"`
    Name      string `xml:"name"`
    Email     string `xml:"email"`
    Avatar    string `xml:"avatar"`
    CreatedAt string `xml:"created-at"`
    UpdatedAt string `xml:"updated-at"`
}

func main() {
    httpClient := NewHTTPClient()

    var response Response

    err := httpClient.GetXML("https://thetestrequest.com/authors.xml", &response)
    if err != nil {
        fmt.Println("Error:", err)
        return
    }

    for _, obj := range response.Objects {
        fmt.Printf("ID: %d, Name: %s, Email: %s, Avatar: %s, CreatedAt: %s, UpdatedAt: %s\n",
            obj.ID, obj.Name, obj.Email, obj.Avatar, obj.CreatedAt, obj.UpdatedAt)
    }
}

In client.go, we’ll define a new function for a GET request, in XML:

func (c *HTTPClient) GetXML(url string, v any) error {
	req, err := http.NewRequest("GET", url, nil)
	if err != nil {
		return fmt.Errorf("error creating GET request: %w", err)
	}

	resp, err := c.Client.Do(req)
	if err != nil {
		return fmt.Errorf("error making GET request: %w", err)
	}
	defer resp.Body.Close()

	if resp.StatusCode != http.StatusOK {
		return fmt.Errorf("received non-200 response code: %d", resp.StatusCode)
	}

	body, err := io.ReadAll(resp.Body)
	if err != nil {
		return fmt.Errorf("error reading response body: %w", err)
	}

	err = xml.Unmarshal(body, v)
	if err != nil {
		return fmt.Errorf("error unmarshalling XML response: %w", err)
	}

	return nil
}

In this example, we:

• Read the response body.

• Convert the XML response into our predefined structure.

• Print the formatted data to the console for better readability.

After running the code, you will see the following output:

To learn more about JSON and XML, their key differences, and best use cases, check out our article: "JSON vs. XML: Comparing Popular Data Exchange Formats."

Handling HTTP Errors and Logging

Proper error handling is a critical part of integrating with an API. Let's break it down into three key failure points:

• Request Sending Errors — Occur due to network issues, incorrect URLs, or an unreachable server.

• Response Reading Errors — Even a successful 200 OK status does not always guarantee valid data.

• Data Conversion Errors — A common issue when working with JSON/XML responses.

Proper error handling is important as it prevents application crashes and simplifies debugging when something goes wrong with API communication.

We will implement error logging using the following code:

package main

import (
	"fmt"
	"log"
	"os"
)

type Post struct {
	UserID int    `json:"userId"`
	ID     int    `json:"id"`
	Title  string `json:"title"`
	Body   string `json:"body"`
}

func main() {
	if err := run(); err != nil {
		log.Printf("Error: %v", err)
		os.Exit(1)
	}
}

func run() error {
	client := NewHTTPClient()

	post, err := client.GetBlogPost(1)
	if err != nil {
		return fmt.Errorf("error occurred while getting post: %w", err)
	}

	fmt.Printf("ID: %d\nUser ID: %d\nTitle: %s\nBody: %s\n", post.ID, post.UserID, post.Title, post.Body)

	return nil
}

In this example, we use the log package to log errors. The log.Errorf function outputs an error message. The result of the code execution will remain the same as before since there will be no errors in the requests, but you can try changing variables to see error messages.

Automating HTTP Requests

In this chapter, we will explore the possibility of automating the sending of multiple HTTP requests. We will look at different approaches, including using loops, utilizing goroutines for parallel requests, and asynchronous handling of requests and responses.

Using Loops to Send Multiple Requests

To send multiple HTTP requests, we can use loops:

package main

import (
	"fmt"
	"log"
)

type Post struct {
	UserID int    `json:"userId"`
	ID     int    `json:"id"`
	Title  string `json:"title"`
	Body   string `json:"body"`
}

func main() {
	client := NewHTTPClient()

	for i := 1; i <= 5; i++ {
		post, err := client.GetBlogPost(i)
		if err != nil {
			log.Printf("Error getting post %d: %v", i, err)
			continue
		}

		fmt.Printf("Request to post %d returned:\nID: %d \n%s \n\n",
			i, post.ID, post.Title)
	}
}

We use the for loop to send requests to different URLs. Then, we print the requests with the number, PostID, and title to the console. After execution, you will receive the following message:

Using Goroutines for Parallel HTTP Requests

Go provides built-in capabilities for parallel task execution through goroutines. This allows sending multiple requests simultaneously, significantly speeding up the program's execution.

package main

import (
	"fmt"
	"log"
	"sync"
)

type Post struct {
	UserID int    `json:"userId"`
	ID     int    `json:"id"`
	Title  string `json:"title"`
	Body   string `json:"body"`
}

// fetchPost handles fetching a post using the GetBlogPost method and outputs the result.
func fetchPost(client *HTTPClient, postID int, wg *sync.WaitGroup) {
	defer wg.Done()

	post, err := client.GetBlogPost(postID)
	if err != nil {
		log.Printf("Error getting post %d: %v", postID, err)
		return
	}

	fmt.Printf("Request to post %d returned:\nID: %d\nUser ID: %d\nTitle: %s\nBody: %s\n\n",
		postID, post.ID, post.UserID, post.Title, post.Body)
}

func main() {
	client := NewHTTPClient()
	var wg sync.WaitGroup

	postIDs := []int{1, 2, 3, 4, 5}

	for _, postID := range postIDs {
		wg.Add(1)
		go fetchPost(client, postID, &wg)
	}

	wg.Wait()
}

In this example, we create the fetchPost function, which sends a request and prints the status.  sync.WaitGroup is used to wait for the completion of all goroutines. Run this code and compare the execution speed with the previous solution. The script output may vary due to its asynchronous nature.

Example of Asynchronous Request and Response Handling

Asynchronous processing allows sending requests and processing responses as they arrive. Let's look at an example using a channel to transmit results:

package main

import (
	"fmt"
	"log"
	"sync"
)

type Post struct {
	UserID int    `json:"userId"`
	ID     int    `json:"id"`
	Title  string `json:"title"`
	Body   string `json:"body"`
}

type Result struct {
	PostID int
	Post   *Post
	Err    error
}

// fetchPost handles fetching a post through the GetBlogPost method and sends the result to the channel.
func fetchPost(client *HTTPClient, postID int, results chan<- Result, wg *sync.WaitGroup) {
	defer wg.Done()

	post, err := client.GetBlogPost(postID)
	results <- Result{PostID: postID, Post: post, Err: err}
}

func main() {
	client := NewHTTPClient()
	var wg sync.WaitGroup

	postIDs := []int{1, 2, 3, 4, 5}
	results := make(chan Result, len(postIDs))

	// Launch goroutines for parallel request execution
	for _, postID := range postIDs {
		wg.Add(1)
		go fetchPost(client, postID, results, &wg)
	}

	// Function to close the channel after all goroutines finish
	go func() {
		wg.Wait()
		close(results)
	}()

	// Process results as they arrive
	for result := range results {
		if result.Err != nil {
			log.Printf("Error fetching post %d: %v\n", result.PostID, result.Err)
			continue
		}
		fmt.Printf("Request to post %d returned:\nID: %d\nUser ID: %d\nTitle: %s\nBody: %s\n\n",
			result.PostID, result.Post.ID, result.Post.UserID, result.Post.Title, result.Post.Body)
	}
}

In this example, we introduce a new Result structure to store requests' results and use the results channel to pass results from goroutines to the main function. At first glance, the last two approaches might seem very similar, and they are to some extent, but there are still differences:

• Result Handling: In the asynchronous approach with channels, results are processed in the main thread as they arrive, while in the approach without channels, results are processed within the goroutines.
• Synchronization: Channels provide built-in mechanisms for safely passing data between goroutines, whereas in the approach without channels, sync.WaitGroup is needed.
• Resource Usage: Asynchronous processing with channels may handle resources more effectively. In the first approach, all tasks are executed in parallel, but the results may be processed less efficiently.

Due to the asynchronous nature, results are processed as they arrive from the channel, meaning the order of posts may not always be the same when rerunning the code. One possible output is shown below:

Advanced Features and Tips

The guide above is enough to write your first HTTP client. However, if you plan to advance in this area, you will be interested in exploring advanced features and best practices for development. This chapter includes the use of third-party libraries, debugging and optimization techniques, as well as security considerations.

Using Third-Party Libraries for Working with APIs

The Go standard library provides basic functionality for working with HTTP requests, but sometimes it's more convenient to use third-party libraries that offer advanced features and simplify the process. One such library is go-resty.

To install the library, use the following command:

go get -u github.com/go-resty/resty/v2

Some of the advantages of go-resty include:

• Automatic serialization (the process of converting data structures) and deserialization.
• Session management (cookie support) and retries for failed requests.
• Asynchronous requests.
• Flexible configuration of timeouts, headers, parameters, and other options.
• Built-in debugging features, including logging.
• Testing tools such as mocking.

Here is an example for sending GET and POST requests using the go-resty library:

package main

import (
	"fmt"
	"log"

	"github.com/go-resty/resty/v2"
)

func main() {
	client := resty.New()

	// GET request
	resp, err := client.R().
		SetQueryParam("userId", "1").
		Get("https://jsonplaceholder.typicode.com/posts")
	if err != nil {
		log.Fatalf("Error on GET request: %v", err)
	}
	fmt.Println("GET Response Info:")
	fmt.Println("Status Code:", resp.StatusCode())
	fmt.Println("Body:", resp.String())

	// POST request
	post := map[string]any{
		"userId": 1,
		"title":  "foo",
		"body":   "bar",
	}
	resp, err = client.R().
		SetHeader("Content-Type", "application/json").
		SetBody(post).
		Post("https://jsonplaceholder.typicode.com/posts")
	if err != nil {
		log.Fatalf("Error on POST request: %v", err)
	}
	fmt.Println("POST Response Info:")
	fmt.Println("Status Code:", resp.StatusCode())
	fmt.Println("Body:", resp.String())
}

The library significantly simplifies working with HTTP requests and provides many useful features. Debugging and optimization are crucial aspects of development, so let's look at some examples.

Logging Requests and Responses

For debugging purposes, it's helpful to log requests and responses. We can do this using the library we installed earlier:

client := resty.New().
    SetDebug(true)

Also, use http.Transport to manage the number of open connections:

client := resty.New()
transport := &http.Transport{
    MaxIdleConns:       10,
    IdleConnTimeout:    30 * time.Second,
    DisableKeepAlives:  false,
}

client.SetTransport(transport)
client.SetTimeout(10 * time.Second)

Best Practices for Developing Secure and Reliable HTTP Clients

An example of a secure and reliable HTTP client using go-resty:

• Error Handling: Resty automatically handles errors, simplifying response checks.
• Using TLS: Resty supports custom transport settings to enable TLS.
• Secure methods for storing and transmitting authentication tokens:

package main

import (
  "crypto/tls"
  "fmt"
  "log"
  "net/http"

  "github.com/go-resty/resty/v2"
)

func main() {
  // Create client with configured TLS
  client := resty.New()

  // Configure security transport layer
  client.SetTransport(&http.Transport{
    // Using standard TLS configuration
    TLSClientConfig: &tls.Config{
      // Additional configuration parameters can be set here
      MinVersion: tls.VersionTLS12, // Example: minimum TLS version 1.2
    },
  })

  token := "your_auth_token_here"

  // Sending GET request with error handling and TLS verification
  resp, err := client.R().
    SetHeader("Authorization", "Bearer "+token).
    Get("https://jsonplaceholder.typicode.com/posts/1")
  if err != nil {
    log.Fatalf("Error: %v", err)
  }

  if resp.StatusCode() != http.StatusOK {
    log.Fatalf("Non-200 response: %d", resp.StatusCode())
  }

  // Handle response body
  fmt.Printf("Response: %s\n", resp.String())
}

Using the SetHeader method to set the "Authorization" header with a bearer token is a standard and secure practice, provided other security aspects are followed:

• Proper and secure storage of tokens. On the client side, this could be a secure container protected from unauthorized access.
• Transmitting tokens through secure channels, such as HTTPS.
• Minimizing token lifespan and regularly updating tokens. Using time-limited tokens and periodic rotation increases overall security.

Additional recommendations for reliable HTTP clients:

• Timeouts:

client.SetTimeout(15 * time.Second)

• Retries:

client.R().SetRetryCount(3).Get("https://jsonplaceholder.typicode.com/posts/1")

• Logging Requests and Responses:

client.SetDebug(true)

Using go-resty significantly simplifies the process of creating an HTTP client in Go. The library provides extensive capabilities and features for flexible configuration according to your needs. Additionally, go-resty allows you to handle more complex requests, such as file uploads, multipart forms, or custom requests, and it automatically manages headers with minimal code and effort.

Conclusion

Developing HTTP clients in Go is an essential skill for any developer working with web services and APIs. In this article, we covered all key aspects of creating an HTTP client, from the basics to the advanced features of the language. 

For further study and a deeper understanding of the topic, we recommend the following resources:

• Official Go documentation

• net/http package documentation

• GitHub repository for go-resty