HTTP introduction to Go
I am a developer from the paradise island of Mauritius. I particularly enjoy working with Go and started Gophers Mauritius.
This is a very basic tutorial on how to use http and sql in Go. It presumes that you have basic programming knowledge and that you have installed Go.
Hello World
To start with we need to create a new directory and initialize a new go module.
We create a new directory called http-intro and change into it.
Then we need to initialize a new go module by running go mod init github.com/fluxynet/http-intro.
mkdir http-intro
cd http-intro
go mod init http-intro
This will create a file called go.mod which contains the name of the module and the go version.
It will also contain dependencies as we add them.
The file go.sum will contain the checksums of the dependencies to ensure that everyone gets the correct version of the dependencies.
Create a file called main.go and add the following code:
package main
import (
"fmt"
"net/http"
)
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hello World!")
}
func main() {
http.HandleFunc("/", handler)
http.ListenAndServe(":8080", nil)
}
The handler function takes two arguments, a http.ResponseWriter and a *http.Request.
The http.ResponseWriter is used to write the response to the client.
HTTP Handlers
The request object
The *http.Request contains information about the request.
We can use the *http.Request to get information about the request.
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hello World!\n")
fmt.Fprintf(w, "Method: %s\n", r.Method)
fmt.Fprintf(w, "URL: %s\n", r.URL)
fmt.Fprintf(w, "Protocol: %s\n", r.Proto)
fmt.Fprintf(w, "Host: %s\n", r.Host)
fmt.Fprintf(w, "RemoteAddr: %s\n", r.RemoteAddr)
fmt.Fprintf(w, "RequestURI: %s\n", r.RequestURI)
fmt.Fprintf(w, "Path: %s\n", r.URL.Path)
for name := range r.Header {
fmt.Fprintf(w, "Header: %s: %s\n", name, r.Header.Get(name))
}
for name, values := range r.URL.Query() {
for _, value := range values {
fmt.Fprintf(w, "Query: %s: %s\n", name, value)
}
}
for _, cookie := range r.Cookies() {
fmt.Fprintf(w, "Cookie: %s: %s\n", cookie.Name, cookie.Value)
}
fmt.Fprintf(w, "Body: %s\n", r.Body)
}
Let us test the above code by running go run main.go and then open a browser and go to http://localhost:8080.
We can also test the code by sending direct http requests. For this purpose we will use the vscode extension REST Client.
Create a file called requests.http and add the following code:
GET http://localhost:8080/
Then click on the Send Request link above the request.
We can add query parameters to the request.
GET http://localhost:8080/?name=apple&color=red
And cookies
GET http://localhost:8080/?name=apple&color=red
Cookie: user=john
We can also use different methods like POST, PUT, PATCH and DELETE.
POST http://localhost:8080/
Hello World!
The request body
The *http.Request also contains a Body which is a io.ReadCloser.
We can read the body by using the ioutil.ReadAll function.
func handler(w http.ResponseWriter, r *http.Request) {
body, err := ioutil.ReadAll(r.Body)
if err != nil {
fmt.Fprintf(w, "Error reading body: %v", err)
return
}
fmt.Fprintf(w, "Hello World!\n")
fmt.Fprintf(w, "Method: %s\n", r.Method)
fmt.Fprintf(w, "URL: %s\n", r.URL)
fmt.Fprintf(w, "Protocol: %s\n", r.Proto)
fmt.Fprintf(w, "Host: %s\n", r.Host)
fmt.Fprintf(w, "RemoteAddr: %s\n", r.RemoteAddr)
fmt.Fprintf(w, "RequestURI: %s\n", r.RequestURI)
fmt.Fprintf(w, "Header: %s\n", r.Header)
fmt.Fprintf(w, "Body: %s\n", body)
}
We can also use the json package to decode the body into a struct.
type Fruit struct {
Name string `json:"name"`
Color string `json:"color"`
}
func handler(w http.ResponseWriter, r *http.Request) {
body, err := ioutil.ReadAll(r.Body)
if err != nil {
fmt.Fprintf(w, "Error reading body: %v", err)
return
}
var fruit Fruit
err = json.Unmarshal(body, &fruit)
if err != nil {
fmt.Fprintf(w, "Error unmarshaling body: %v", err)
return
}
fmt.Fprintf(w, "Hello World!\n")
fmt.Fprintf(w, "Method: %s\n", r.Method)
fmt.Fprintf(w, "URL: %s\n", r.URL)
fmt.Fprintf(w, "Protocol: %s\n", r.Proto)
fmt.Fprintf(w, "Host: %s\n", r.Host)
fmt.Fprintf(w, "RemoteAddr: %s\n", r.RemoteAddr)
fmt.Fprintf(w, "RequestURI: %s\n", r.RequestURI)
fmt.Fprintf(w, "Header: %s\n", r.Header)
fmt.Fprintf(w, "Fruit: %s\n", fruit)
}
We can test this by sending a POST request with the following body:
POST http://localhost:8080/
{
"name": "apple",
"color": "red"
}
We can use different encodings like url encoded and form data.
func handler(w http.ResponseWriter, r *http.Request) {
err := r.ParseForm()
if err != nil {
fmt.Fprintf(w, "Error parsing form: %v", err)
return
}
fmt.Fprintf(w, "Hello World!\n")
fmt.Fprintf(w, "Method: %s\n", r.Method)
fmt.Fprintf(w, "URL: %s\n", r.URL)
fmt.Fprintf(w, "Protocol: %s\n", r.Proto)
fmt.Fprintf(w, "Host: %s\n", r.Host)
fmt.Fprintf(w, "RemoteAddr: %s\n", r.RemoteAddr)
fmt.Fprintf(w, "RequestURI: %s\n", r.RequestURI)
fmt.Fprintf(w, "Header: %s\n", r.Header)
fmt.Fprintf(w, "Form: %s\n", r.Form)
}
We can test this by sending a POST request with the following body:
POST http://localhost:8080/
Content-Type: application/x-www-form-urlencoded
name=apple&color=red
The response object
The http.ResponseWriter is used to write the response to the client.
We can use the http.ResponseWriter to specify the status code, set headers, send cookies and write the response body.
func handler(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
http.SetCookie(w, &http.Cookie{
Name: "user",
Value: "john",
MaxAge: 86400,
})
// status code must be set before writing the response body but after setting headers
w.WriteHeader(http.StatusCreated)
fmt.Fprintf(w, `{"message": "Hello World!"}`)
}
We can test this by sending a GET request with the following body:
GET http://localhost:8080/
We can also use the http.ResponseWriter to send json responses.
type Fruit struct {
Name string `json:"name"`
Color string `json:"color"`
}
func handler(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
// status code must be set before writing the response body but after setting headers
w.WriteHeader(http.StatusCreated)
json.NewEncoder(w).Encode(Fruit{
Name: "apple",
Color: "red",
})
}
Data Persistence
Before moving to REST APIs let us look at how we can persist data.
Go has a built in package called database/sql which is a generic interface around SQL databases.
We will use the database/sql package to connect to a sqlite database.
We will also use the github.com/mattn/go-sqlite3 package which is a sqlite driver for the database/sql package.
We will create a new file called db.go and add the following code:
package main
import (
"database/sql"
"fmt"
"log"
_ "github.com/mattn/go-sqlite3"
)
func main() {
initialise()
read()
}
func initialise() {
db, err := sql.Open("sqlite3", "fruits.db")
if err != nil {
log.Fatal(err)
}
defer db.Close()
sqlStmt := `
CREATE TABLE IF NOT EXISTS fruits (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT,
color TEXT
);
`
_, err = db.Exec(sqlStmt)
if err != nil {
log.Fatal(err)
}
sqlStmt = `
INSERT INTO fruits (name, color) VALUES
("apple", "red"),
("banana", "yellow"),
("grape", "purple"),
("orange", "orange"),
("strawberry", "red"),
("watermelon", "green");
`
_, err = db.Exec(sqlStmt)
if err != nil {
log.Fatal(err)
}
}
func read() {
rows, err := db.Query("SELECT id, name, color FROM fruits")
if err != nil {
log.Fatal(err)
}
defer rows.Close()
for rows.Next() {
var id int
var name string
var color string
err = rows.Scan(&id, &name, &color)
if err != nil {
log.Fatal(err)
}
fmt.Println(id, name, color)
}
err = rows.Err()
if err != nil {
log.Fatal(err)
}
}
Repository Pattern
We will use the repository pattern to abstract away the database.
This abstraction has the following benefits:
We can easily switch databases
We can easily mock the database for testing
We can easily add caching
The code is easier to read and understand
We will also add more structure to our code to make it easier to add functionality as we go along.
Let us start over with a file called types.go. This will contain all the types we will use.
Types
package httpintro
type Fruit struct {
ID int `json:"id"`
Name string `json:"name"`
Color string `json:"color"`
}
type FruitRepository interface {
Get(id int) (*Fruit, error)
GetAll() ([]Fruit, error)
Create(fruit *Fruit) error
Update(fruit *Fruit) error
Delete(id int) error
}
FruitRepository is an interface which defines the methods we will use to persist Fruit objects.
Errors
Additionally we will create a file errors.go which will contain any errors we will use.
package httpintro
import "errors"
var (
ErrNotFound = errors.New("not found")
)
Implementation
Next let create a package called repos which will contain different repositories for our application.
Create a file as repos/fruits-repo/fruits_repo.go and add the following code:
package fruitsrepo
import (
"database/sql"
"github.com/fluxynet/httpintro"
_ "github.com/mattn/go-sqlite3"
)
type Repository struct {
db *sql.DB
}
func New(db *sql.DB) *Repository {
return &Repository{
db: db,
}
}
func (r *Repository) Get(id int) (*httpintro.Fruit, error) {
var fruit httpintro.Fruit
row := r.db.QueryRow("SELECT id, name, color FROM fruits WHERE id = ?", id)
if err := row.Scan(&fruit.ID, &fruit.Name, &fruit.Color); errors.Is(err, sql.ErrNoRows) {
return nil, httpintro.ErrNotFound
} else if err != nil {
return nil, err
}
return &fruit, nil
}
func (r *Repository) GetAll() ([]httpintro.Fruit, error) {
rows, err := r.db.Query("SELECT id, name, color FROM fruits")
if err != nil {
return nil, err
}
defer rows.Close()
var fruits []httpintro.Fruit
for rows.Next() {
var fruit httpintro.Fruit
if err = rows.Scan(&fruit.ID, &fruit.Name, &fruit.Color); err != nil {
return nil, err
}
fruits = append(fruits, fruit)
}
return fruits, nil
}
func (r *Repository) Create(fruit *httpintro.Fruit) error {
result, err := r.db.Exec("INSERT INTO fruits (name, color) VALUES (?, ?)", fruit.Name, fruit.Color)
if err != nil {
return err
}
id, err := result.LastInsertId()
if err != nil {
return err
}
fruit.ID = int(id)
return nil
}
func (r *Repository) Update(fruit *httpintro.Fruit) error {
result, err := r.db.Exec("UPDATE fruits SET name = ?, color = ? WHERE id = ?", fruit.Name, fruit.Color, fruit.ID)
if err != nil {
return err
}
if rowsAffected, err := result.RowsAffected(); err != nil {
return err
} else if rowsAffected == 0 {
return httpintro.ErrNotFound
}
return nil
}
func (r *Repository) Delete(id int) error {
if _, err := r.db.Exec("DELETE FROM fruits WHERE id = ?", id); err != nil {
return err
}
return nil
}
Importing data
Let us add an import functionality that will allow us to import data from a json file. It will also be an opportunity to see if our repository works.
It is common to have a cmd folder which in turn contains a folder for each command.
We will create a folder called cmd and a folder called import inside it.
Create a file called cmd/import/main.go and add the following code:
package main
import (
"database/sql"
"encoding/json"
"fmt"
"log"
"os"
"github.com/fluxynet/httpintro"
fruitsrepo "github.com/fluxynet/httpintro/repos/fruits-repo"
)
func main() {
file, err := os.ReadFile("fruits.json")
if err != nil {
log.Fatal(err)
}
var fruits []httpintro.Fruit
err = json.Unmarshal(file, &fruits)
if err != nil {
log.Fatal(err)
}
db, err := sql.Open("sqlite3", "fruits.db")
if err != nil {
log.Fatal(err)
}
defer db.Close()
createTableSQL := `
CREATE TABLE IF NOT EXISTS fruits (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT,
color TEXT
);
`
_, err = db.Exec(createTableSQL)
if err != nil {
log.Fatal(err)
}
repo := fruitsrepo.New(db)
for _, fruit := range fruits {
err = repo.Create(&fruit)
if err != nil {
log.Fatal(err)
}
}
fmt.Println("Import successful")
}
Let us add another command to display all the fruits in the database. Create a file as cmd/list/main.go and add the following code:
package main
import (
"database/sql"
"fmt"
"log"
"github.com/fluxynet/httpintro/repos/fruits-repo"
_ "github.com/mattn/go-sqlite3"
)
func main() {
db, err := sql.Open("sqlite3", "fruits.db")
if err != nil {
log.Fatal(err)
}
defer db.Close()
repo := fruitsrepo.New(db)
fruits, err := repo.GetAll()
if err != nil {
log.Fatal(err)
}
for _, fruit := range fruits {
fmt.Println(fruit)
}
}
Let us test the import command by running go run cmd/import/main.go and then run go run cmd/list/main.go. First ensure that the fruits.json file exists.
You may use the following:
[
{
"name": "apple",
"color": "red"
},
{
"name": "banana",
"color": "yellow"
},
{
"name": "grape",
"color": "purple"
},
{
"name": "orange",
"color": "orange"
},
{
"name": "strawberry",
"color": "red"
},
{
"name": "watermelon",
"color": "green"
}
]
REST API
Now that we have a way to persist data let us create a REST API to interact with the data.
We will create a new package called web/fruits-api which will contain the REST API. The web package itself may be used to contain commonly used code for http applications.
Create a file called web/fruits-api/fruits_api.go and add the following code:
For the purpose of this tutorial we will use the github.com/go-chi/chi package to handle routing.
This package is not part of the standard library and can be installed by running go get github.com/go-chi/chi/v5. It provides a powerful router which supports url parameters, middlewares and subrouters.
Helpers
We will create a file called web/web.go which will contain commonly used functions.
package web
import (
"encoding/json"
"net/http"
)
type ErrorResponse struct {
Error string `json:"error"`
}
func Error(w http.ResponseWriter, err error, status int) {
w.WriteHeader(status)
json.NewEncoder(w).Encode(ErrorResponse{
Error: err.Error(),
})
}
API
package fruitsapi
import (
"encoding/json"
"net/http"
"strconv"
"github.com/fluxynet/httpintro"
"github.com/fluxynet/httpintro/web"
"github.com/go-chi/chi/v5"
_ "github.com/mattn/go-sqlite3"
)
type API struct {
repo httpintro.FruitRepository
}
func New(repo httpintro.FruitRepository) *API {
return &API{
repo: repo,
}
}
func (a *API) Route(r chi.Router) {
r.Get("/", a.GetAll)
r.Post("/", a.Create)
r.Get("/{id}", a.Get)
r.Put("/{id}", a.Update)
r.Delete("/{id}", a.Delete)
}
func (a *API) Get(w http.ResponseWriter, r *http.Request) {
id, err := strconv.Atoi(chi.URLParam(r, "id"))
if err != nil {
web.Error(w, err, http.StatusBadRequest)
return
}
fruit, err := a.repo.Get(id)
if errors.Is(err, httpintro.ErrNotFound) {
web.Error(w, err, http.StatusNotFound)
return
} else if err != nil {
web.Error(w, err, http.StatusInternalServerError)
return
}
json.NewEncoder(w).Encode(fruit)
}
func (a *API) GetAll(w http.ResponseWriter, r *http.Request) {
fruits, err := a.repo.GetAll()
if err != nil {
web.Error(w, err, http.StatusInternalServerError)
return
}
json.NewEncoder(w).Encode(fruits)
}
func (a *API) Create(w http.ResponseWriter, r *http.Request) {
var fruit httpintro.Fruit
err := json.NewDecoder(r.Body).Decode(&fruit)
if err != nil {
web.Error(w, err, http.StatusBadRequest)
return
}
err = a.repo.Create(&fruit)
if err != nil {
web.Error(w, err, http.StatusInternalServerError)
return
}
w.WriteHeader(http.StatusCreated)
json.NewEncoder(w).Encode(fruit)
}
func (a *API) Update(w http.ResponseWriter, r *http.Request) {
id, err := strconv.Atoi(chi.URLParam(r, "id"))
if err != nil {
web.Error(w, err, http.StatusBadRequest)
return
}
var fruit httpintro.Fruit
err = json.NewDecoder(r.Body).Decode(&fruit)
if err != nil {
web.Error(w, err, http.StatusBadRequest)
return
}
fruit.ID = id
err = a.repo.Update(&fruit)
if errors.Is(err, httpintro.ErrNotFound) {
web.Error(w, err, http.StatusNotFound)
return
} else if err != nil {
web.Error(w, err, http.StatusInternalServerError)
return
}
json.NewEncoder(w).Encode(fruit)
}
func (a *API) Delete(w http.ResponseWriter, r *http.Request) {
id, err := strconv.Atoi(chi.URLParam(r, "id"))
if err != nil {
web.Error(w, err, http.StatusBadRequest)
return
}
err = a.repo.Delete(id)
if err != nil {
web.Error(w, err, http.StatusInternalServerError)
return
}
w.WriteHeader(http.StatusNoContent)
}
Next we will create a new command called cmd/server which will start a http server.
Create a file called cmd/server/main.go and add the following code:
package main
import (
"database/sql"
"log"
"net/http"
fruitsrepo "github.com/fluxynet/httpintro/repos/fruits-repo"
fruitsapi "github.com/fluxynet/httpintro/web/fruits-api"
"github.com/go-chi/chi/v5"
_ "github.com/mattn/go-sqlite3"
)
func main() {
db, err := sql.Open("sqlite3", "fruits.db")
if err != nil {
log.Fatal(err)
}
defer db.Close()
repo := fruitsrepo.New(db)
api := fruitsapi.New(repo)
r := chi.NewRouter()
r.Route("/fruits", api.Route)
http.ListenAndServe(":8080", r)
}
We can use the following requests.http file to test the API:
GET http://localhost:8080/fruits
###
POST http://localhost:8080/fruits
Content-Type: application/json
{
"name": "cherry",
"color": "red"
}
###
GET http://localhost:8080/fruits/1
###
PUT http://localhost:8080/fruits/1
{
"name": "apple",
"color": "green"
}
###
DELETE http://localhost:8080/fruits/1
Middlewares
Middlewares are functions that are executed before or after a request is handled. They are useful for adding functionality like logging, authentication, authorization, rate limiting, etc.
Chi has a built in middleware called chi/middleware.Logger which logs the request and response.
We can add it to our server by adding the following code:
r.Use(middleware.Logger)
We can also create our own middleware.
Let us create a middleware that will check if the request has a valid api key.
Create a file called web/middlewares/api_key.go and add the following code:
package middlewares
import (
"net/http"
)
func APIKey(apiKey string) func(next http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
key := r.Header.Get("X-API-Key")
if key != apiKey {
http.Error(w, "Invalid API Key", http.StatusUnauthorized)
return
}
next.ServeHTTP(w, r)
})
}
}
We can use the middleware by adding the following code:
r.Use(middlewares.APIKey("secret"))
Testing
Testing is an important part of software development. It allows us to ensure that our code works as expected and that we do not break anything when we make changes.
Go has a built in testing package called testing which we will use to write our tests.
We will be writing unit tests for the fruitsapi package.
Mocking
Our api has a dependency on the fruitsrepo package which we will mock.
We will use the mockery package to generate mocks for our interfaces.
In our root directory we will create a file called tools.go and add the following code:
package httpintro
//go:generate go install github.com/vektra/mockery/v2@latest
//go:generate mockery --name=FruitRepository --output=mocks/repos/fruits-repo --outpkg=fruitsrepo --filename=fruits_repo.go
The above contains two directives which will be used by the go generate command:
The first line will install the mockery package.
The second line will generate a mock for the
FruitRepositoryinterface and place it in themocks/repos/fruits-repofolder.
go generate is a command that will run any directives in a go file that is prefixed with //go:generate.
The generated file uses the package github.com/stretchr/testify/mock which is a mocking library. We need to install it by running go get github.com/stretchr/testify/mock.
Unit Tests
Test files are named *_test.go and are placed in the same directory as the code they are testing.
Basic Test
Create a file called web/fruits-api/fruits_api_test.go and add the following code:
package fruitsapi
import (
"net/http"
"net/http/httptest"
"testing"
"github.com/fluxynet/httpintro"
fruitsrepo "github.com/fluxynet/httpintro/mocks/repos/fruits-repo"
"github.com/go-chi/chi/v5"
"github.com/stretchr/testify/assert"
)
func TestGet(t *testing.T) {
var repo fruitsrepo.FruitRepository
repo.On("Get", 1).Return(&httpintro.Fruit{
ID: 1,
Name: "apple",
Color: "red",
}, nil)
api := New(&repo)
r := httptest.NewRequest("GET", "/fruits/1", nil)
w := httptest.NewRecorder()
router := chi.NewRouter()
router.Route("/fruits", api.Route)
router.ServeHTTP(w, r)
assert.Equal(t, http.StatusOK, w.Code)
assert.JSONEq(t, `{"id": 1, "name": "apple", "color": "red"}`, w.Body.String())
repo.AssertExpectations(t)
}
We can run the tests by running go test ./....
We can also run the tests with coverage by running go test ./... -coverprofile=coverage.out and then go tool cover -html=coverage.out.
Table Driven Tests
Table driven tests are useful when we want to test a function with different inputs. We can easily add more test cases without having to write more code.
Let us rewrite the above test using table driven tests.
package fruitsapi
import (
"errors"
"net/http"
"net/http/httptest"
"testing"
"github.com/fluxynet/httpintro"
fruitsrepo "github.com/fluxynet/httpintro/mocks/repos/fruits-repo"
"github.com/go-chi/chi/v5"
"github.com/stretchr/testify/assert"
)
func TestGet(t *testing.T) {
tests := []struct {
name string
id int
expectedStatus int
expectedBody string
mock func(repo *fruitsrepo.FruitRepository)
}{
{
name: "success",
id: 1,
expectedStatus: http.StatusOK,
expectedBody: `{"id": 1, "name": "apple", "color": "red"}`,
mock: func(repo *fruitsrepo.FruitRepository) {
repo.On("Get", 1).Return(&httpintro.Fruit{
ID: 1,
Name: "apple",
Color: "red",
}, nil)
},
},
{
name: "not found",
id: 1,
expectedStatus: http.StatusNotFound,
expectedBody: `{"error": "not found"}`,
mock: func(repo *fruitsrepo.FruitRepository) {
repo.On("Get", 1).Return(nil, httpintro.ErrNotFound)
},
},
{
name: "internal server error",
id: 1,
expectedStatus: http.StatusInternalServerError,
expectedBody: `{"error": "internal server error"}`,
mock: func(repo *fruitsrepo.FruitRepository) {
repo.On("Get", 1).Return(nil, errors.New("internal server error"))
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var repo fruitsrepo.FruitRepository
tt.mock(&repo)
api := New(&repo)
r := httptest.NewRequest("GET", "/fruits/1", nil)
w := httptest.NewRecorder()
router := chi.NewRouter()
router.Route("/fruits", api.Route)
router.ServeHTTP(w, r)
assert.Equal(t, tt.expectedStatus, w.Code)
if tt.expectedBody == "" {
assert.Equal(t, tt.expectedBody, w.Body.String())
} else {
assert.JSONEq(t, tt.expectedBody, w.Body.String())
}
repo.AssertExpectations(t)
})
}
}
The path forward
We have covered the basics of http, sql and testing in go.
For the sake of simplicity and clarity, we have stuck with the standard library and a few third party packages. There are many more packages that can be used to make our lives easier, for example:
(Cobra)[https://github.com/spf13/cobra] - A library for creating command line applications
(Gorm)[https://gorm.io/] - An ORM for SQL databases
There is still a lot to learn and we have only scratched the surface.
Here are some topics you can look into next: