RESTful Go: An API Server with 90%+ Test Coverage in 260 Lines of Code

I wanted to try building a small, RESTful API for a mobile app. And, like any respectable piece of software, I wanted close to 100% test coverage.

The Premise: Signature Collection for Petitions

I built an API to collect signatures for online petitions. Each signature is composed of a name, age, and short message. The server responds to the following requests:

HTTP Verb Path Use
GET /signatures List all signatures
POST /signatures Create a signature

The Dependencies

If you're following along at home, you'll need to go get these libraries:

go get github.com/go-martini/martini
go get github.com/martini-contrib/binding
go get github.com/martini-contrib/render
go get labix.org/v2/mgo
go get github.com/onsi/ginkgo
go get github.com/onsi/gomega
go get github.com/modocache/gory

You'll also need MongoDB. On OS X you can install it via Homebrew, with brew install mongodb.

Project Structure

signatures/
    main.go
    signatures/
        signature.go
        database.go
        server.go
        signatures_suite_test.go
        server_test.go
File LoC Purpose
signatures/signature.go 24 Defines the `Signature` struct
signatures/database.go 36 Connects to MongoDB
signatures/server.go 38 Defines our application routes
signatures/signatures_suite_test.go 34 Used by Ginkgo to run unit tests
signatures/server_test.go 124 Unit tests
main.go 7 Simply runs the server

That's a grand total of only 263 lines of code! The rest of this post explains those lines.

Representing a Signature with the Signature Struct

First, I define the Signature struct.

When the server receives "GET /signature", it will grab these from the database and display them as JSON.

When it receives "POST /signature" with valid JSON data, it will create one of these and insert it into the database.

// signatures/signature.go

package signatures

import "labix.org/v2/mgo"

/*
Each signature is composed of a first name, last name,
email, age, and short message. When represented in
JSON, ditch TitleCase for snake_case.
*/
type Signature struct {
	FirstName string `json:"first_name"`
	LastName  string `json:"last_name"`
	Email     string `json:"email"`
	Age       int    `json:"age"`
	Message   string `json:"message"`
}

/*
I want to make sure all these fields are present. The message
is optional, but if it's present it has to be less than
140 characters--it's a short blurb, not your life story.
*/
func (signature *Signature) valid() bool {
	return len(signature.FirstName) > 0 &&
		len(signature.LastName) > 0 &&
		len(signature.Email) > 0 &&
		signature.Age >= 18 && signature.Age <= 180 &&
		len(signature.Message) < 140
}

/*
I'll use this method when displaying all signatures for
"GET /signatures". Consult the mgo docs for more info:
http://godoc.org/labix.org/v2/mgo
*/
func fetchAllSignatures(db *mgo.Database) []Signature {
	signatures := []Signature{}
	err := db.C("signatures").Find(nil).All(&signatures)
	if err != nil {
		panic(err)
	}

	return signatures
}

Establishing a MongoDB Session

I need to connect to MongoDB to retrieve and insert signatures. The main database will eventually contain tons of signatures, but I want my unit tests to run with a fresh database every time.

Therefore, when establishing a session, I'll allow the caller to specify the name of the database. The main app database will be called "signatures", and the tests will use "signatures_test".

// signatures/database.go

package signatures

import (
	"github.com/go-martini/martini"
	"labix.org/v2/mgo"
)

/*
I want to use a different database for my tests,
so I'll embed *mgo.Session and store the database name.
*/
type DatabaseSession struct {
	*mgo.Session
	databaseName string
}

/*
Connect to the local MongoDB and set up the database.
*/
func NewSession(name string) *DatabaseSession {
	session, err := mgo.Dial("mongodb://localhost")
	if err != nil {
		panic(err)
	}

	addIndexToSignatureEmails(session.DB(name))
	return &DatabaseSession{session, name}
}

/*
Add a unique index on the "email" field.
This doesn't prevent users from signing twice,
since they can still enter
"dudebro+signature2@exmaple.com". But if they're
that clever, I say they deserve the extra signature.
*/
func addIndexToSignatureEmails(db *mgo.Database) {
	index := mgo.Index{
		Key:      []string{"email"},
		Unique:   true,
		DropDups: true,
	}
	indexErr := db.C("signatures").EnsureIndex(index)
	if indexErr != nil {
		panic(indexErr)
	}
}

/*
Martini lets you inject parameters for routing handlers
by using `context.Map()`. I'll pass each route handler
a instance of an *mgo.Database, so they can
get and insert signatures.

For more information, check out:
http://blog.gopheracademy.com/day-11-martini
*/
func (session *DatabaseSession) Database() martini.Handler {
	return func(context martini.Context) {
		s := session.Clone()
		context.Map(s.DB(session.databaseName))
		defer s.Close()
		context.Next()
	}
}

Defining the Routes

The server responds to requests by retrieving or creating signatures.

// signatures/server.go

package signatures

import (
	"github.com/go-martini/martini"
	"github.com/martini-contrib/binding"
	"github.com/martini-contrib/render"
	"labix.org/v2/mgo"
)

/*
Wrap the Martini server struct.
*/
type Server *martini.ClassicMartini

/*
Create a new *martini.ClassicMartini server.
We'll use a JSON renderer and our MongoDB
database handler. We define two routes:
"GET /signatures" and "POST /signatures".
*/
func NewServer(session *DatabaseSession) Server {
	// Create the server and set up middleware.
	m := Server(martini.Classic())
	m.Use(render.Renderer(render.Options{
		IndentJSON: true,
	}))
	m.Use(session.Database())

	// Define the "GET /signatures" route.
	m.Get("/signatures", func(r render.Render, db *mgo.Database) {
		r.JSON(200, fetchAllSignatures(db))
	})

	// Define the "POST /signatures" route.
	m.Post("/signatures", binding.Json(Signature{}),
		func(signature Signature,
			r render.Render,
			db *mgo.Database) {

			if signature.valid() {
				// signature is valid, insert into database
				err := db.C("signatures").Insert(signature)
				if err == nil {
					// insert successful, 201 Created
					r.JSON(201, signature)
				} else {
					// insert failed, 400 Bad Request
					r.JSON(400, map[string]string{
						"error": err.Error(),
					})
				}
			} else {
				// signature is invalid, 400 Bad Request
				r.JSON(400, map[string]string{
					"error": "Not a valid signature",
				})
			}
		})

	// Return the server. Call Run() on the server to
	// begin listening for HTTP requests.
	return m
}

Taking the Server Out for a Spin

Having built the server, it's time to see if it actually works. I define an executable in main.go that creates a new server object and runs it.

// main.go

package main

import "github.com/modocache/signatures/signatures"

/*
Create a new MongoDB session, using a database
named "signatures". Create a new server using
that session, then begin listening for HTTP requests.
*/
func main() {
	session := signatures.NewSession("signatures")
	server := signatures.NewServer(session)
	server.Run()
}

Now it's time to compile and run this bad boy. You'll need mongod running on a standard port, then execute:

src/signatures/ $ go install
src/signatures/ $ signatures
[martini] listening on :3000 (development)

Sweet! I can create a signature:

$ curl -i -X POST \
    -H "Content-Type: application/json"
    -d '{"first_name": "Cervantes", "last_name": "Foreman", "age": 21, "email": "cervantesforeman@zilch.com", "message": "Dolore irure proident."}' \
    localhost:3000/signatures

HTTP/1.1 201 Created
Content-Type: application/json; charset=UTF-8
Date: Mon, 12 May 2014 06:41:49 GMT

{
  "first_name": "Cervantes",
  "last_name": "Foreman",
  "email": "cervantesforeman@zilch.com",
  "age": 21,
  "message": "Dolore irure proident."
}

And I can grab a list of all signatures:

$ curl -i localhost:3000/signatures

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Date: Mon, 12 May 2014 06:43:24 GMT

[
  {
    "first_name": "Cervantes",
    "last_name": "Foreman",
    "email": "cervantesforeman@zilch.com",
    "age": 21,
    "message": "Dolore irure proident."
  }
]

Unit Testing the Server

Ginkgo provides a command-line executable to generate unit tests.

src/signatures/signatures/ $ ginkgo bootstrap
Generating ginkgo test suite bootstrap for test in:
	signatures_suite_test.go

src/signatures/signatures/ $ ginkgo generate server
Generating ginkgo test for Server in:
	server_test.go

Ginkgo uses signatures_suite_test.go to kick off the unit tests, which are contained in server_test.go.

Defining Factory Objects with gory

My unit tests will be creating a bunch of Signature objects, both valid and invalid. I don't want to write a bunch of object creation boilerplate in my unit tests, though, so I used an object factory.

// signatures/signatures_test_suite.go

package signatures_test

import (
	. "github.com/modocache/signatures/signatures"
	. "github.com/onsi/ginkgo"
	. "github.com/onsi/gomega"

	"fmt"
	"github.com/modocache/gory"
	"testing"
)

/*
Ginkgo generated this function
to kick off our unit tests. Hook into it
to define factories.
*/
func TestSignatures(t *testing.T) {
	defineFactories()
	RegisterFailHandler(Fail)
	RunSpecs(t, "Signatures Suite")
}

/*
Define two factories: one for a valid signature,
and one for an invalid one (too young).
*/
func defineFactories() {
	gory.Define("signature", Signature{},
		func(factory gory.Factory) {
			factory["FirstName"] = "Jane"
			factory["LastName"] = "Doe"
			factory["Age"] = 27
			factory["Message"] = "I agree!"
			factory["Email"] = gory.Sequence(
				func(n int) interface{} {
					return fmt.Sprintf("jane-doe-%d@example.com", n)
				})
		})

	gory.Define("signatureTooYoung", Signature{},
		func(factory gory.Factory) {
			factory["FirstName"] = "Joey"
			factory["LastName"] = "Invalid"
			factory["Age"] = 10
			factory["Email"] = "joey-invalid@example.com"
		})
}

Writing the Tests

Now it's time to actually write the unit tests.

Note that when I built this application, I tested throughout its development. I don't recommend you save your tests for the very end. I've done so in this post so as to present the material more clearly.

Testing HTTP in Go is a snap, thanks to the httptest package in the Go standard library. For more information, this blog post by Pivotal Labs is spectacular.

The gist is that we can use an instance of *httptest.ResponseRecorder to access responses from our server.

// signatures/server_test.go

package signatures_test

import (
	. "github.com/modocache/signatures/signatures"

	"bytes"
	"encoding/json"
	"github.com/modocache/gory"
	. "github.com/onsi/ginkgo"
	. "github.com/onsi/gomega"
	"net/http"
	"net/http/httptest"
)

/*
Convert JSON data into a slice.
*/
func sliceFromJSON(data []byte) []interface{} {
	var result interface{}
	json.Unmarshal(data, &result)
	return result.([]interface{})
}

/*
Convert JSON data into a map.
*/
func mapFromJSON(data []byte) map[string]interface{} {
	var result interface{}
	json.Unmarshal(data, &result)
	return result.(map[string]interface{})
}

/*
Server unit tests.
*/
var _ = Describe("Server", func() {
	var dbName string
	var session *DatabaseSession
	var server Server
	var request *http.Request
	var recorder *httptest.ResponseRecorder

	BeforeEach(func() {
		// Set up a new server, connected to a test database,
		// before each test.
		dbName = "signatures_test"
		session = NewSession(dbName)
		server = NewServer(session)

		// Record HTTP responses.
		recorder = httptest.NewRecorder()
	})

	AfterEach(func() {
		// Clear the database after each test.
		session.DB(dbName).DropDatabase()
	})

	Describe("GET /signatures", func() {

		// Set up a new GET request before every test
		// in this describe block.
		BeforeEach(func() {
			request, _ = http.NewRequest("GET", "/signatures", nil)
		})

		Context("when no signatures exist", func() {
			It("returns a status code of 200", func() {
				server.ServeHTTP(recorder, request)
				Expect(recorder.Code).To(Equal(200))
			})

			It("returns a empty body", func() {
				server.ServeHTTP(recorder, request)
				Expect(recorder.Body.String()).To(Equal("[]"))
			})
		})

		Context("when signatures exist", func() {

			// Insert two valid signatures into the database
			// before each test in this context.
			BeforeEach(func() {
				collection := session.DB(dbName).C("signatures")
				collection.Insert(gory.Build("signature"))
				collection.Insert(gory.Build("signature"))
			})

			It("returns a status code of 200", func() {
				server.ServeHTTP(recorder, request)
				Expect(recorder.Code).To(Equal(200))
			})

			It("returns those signatures in the body", func() {
				server.ServeHTTP(recorder, request)

				peopleJSON := sliceFromJSON(recorder.Body.Bytes())
				Expect(len(peopleJSON)).To(Equal(2))

				personJSON := peopleJSON[0].(map[string]interface{})
				Expect(personJSON["first_name"]).To(Equal("Jane"))
				Expect(personJSON["last_name"]).To(Equal("Doe"))
				Expect(personJSON["age"]).To(Equal(float64(27)))
				Expect(personJSON["message"]).To(Equal("I agree!"))
				Expect(personJSON["email"]).To(
					ContainSubstring("jane-doe"))
			})
		})
	})

	Describe("POST /signatures", func() {

		Context("with invalid JSON", func() {

			// Create a POST request using JSON from our invalid
			// factory object before each test in this context.
			BeforeEach(func() {
				body, _ := json.Marshal(
					gory.Build("signatureTooYoung"))
				request, _ = http.NewRequest(
					"POST", "/signatures", bytes.NewReader(body))
			})

			It("returns a status code of 400", func() {
				server.ServeHTTP(recorder, request)
				Expect(recorder.Code).To(Equal(400))
			})
		})

		Context("with valid JSON", func() {

			// Create a POST request with valid JSON from
			// our factory before each test in this context.
			BeforeEach(func() {
				body, _ := json.Marshal(
					gory.Build("signature"))
				request, _ = http.NewRequest(
					"POST", "/signatures", bytes.NewReader(body))
			})

			It("returns a status code of 201", func() {
				server.ServeHTTP(recorder, request)
				Expect(recorder.Code).To(Equal(201))
			})

			It("returns the inserted signature", func() {
				server.ServeHTTP(recorder, request)

				personJSON := mapFromJSON(recorder.Body.Bytes())
				Expect(personJSON["first_name"]).To(Equal("Jane"))
				Expect(personJSON["last_name"]).To(Equal("Doe"))
				Expect(personJSON["age"]).To(Equal(float64(27)))
				Expect(personJSON["message"]).To(Equal("I agree!"))
				Expect(personJSON["email"]).To(
					ContainSubstring("jane-doe"))
			})
		})

		Context("with JSON containing a duplicate email", func() {
			BeforeEach(func() {
				signature := gory.Build("signature")
				session.DB(dbName).C("signatures").Insert(signature)

				body, _ := json.Marshal(signature)
				request, _ = http.NewRequest(
					"POST", "/signatures", bytes.NewReader(body))
			})

			It("returns a status code of 400", func() {
				server.ServeHTTP(recorder, request)
				Expect(recorder.Code).To(Equal(400))
			})
		})
	})
})

Running the Tests

Our tests get us pretty great coverage:

src/signatures/ $ ginkgo -r --randomizeAllSpecs -cover
Running Suite: Signatures Suite
===============================
Random Seed: 1399880786 - Will randomize all specs
Will run 8 of 8 specs
...
Ran 8 of 8 Specs in 5.055 seconds
SUCCESS! -- 8 Passed | 0 Failed | 0 Pending | 0 Skipped PASS
coverage: 93.5% of statements

Ginkgo ran in 8.278165839s
Test Suite Passed

Note that you need MongoDB running in order to execute the tests.

Where to Go from Here

You can check out the source code for this project on GitHub. Try adding some features and sending a pull request:

For more information on unit testing and continuous integration in Go, check out another post of mine: Continuous Integration in Go: Ginkgo & Coveralls.