RESTful Go: An API Server with 90%+ Test Coverage in 260 Lines of Code
I wanted to try building a small, RESTful API for a mobile app. And, like any respectable piece of software, I wanted close to 100% test coverage.
The Premise: Signature Collection for Petitions
I built an API to collect signatures for online petitions. Each signature is composed of a name, age, and short message. The server responds to the following requests:
HTTP Verb | Path | Use |
---|---|---|
GET | /signatures | List all signatures |
POST | /signatures | Create a signature |
The Dependencies
- Martini: A Go web framework, like Sinatra. Used for routing.
- mgo: Pronounced "mango". A Go driver for MongoDB, used to persist the signatures.
- Ginkgo & Gomega: Essential for Go unit testing.
- gory: Used to easily create signatures for unit testing.
If you're following along at home, you'll need to go get
these libraries:
go get github.com/go-martini/martini
go get github.com/martini-contrib/binding
go get github.com/martini-contrib/render
go get labix.org/v2/mgo
go get github.com/onsi/ginkgo
go get github.com/onsi/gomega
go get github.com/modocache/gory
You'll also need MongoDB. On OS X you can install it via Homebrew, with brew install mongodb
.
Project Structure
signatures/
main.go
signatures/
signature.go
database.go
server.go
signatures_suite_test.go
server_test.go
File | LoC | Purpose |
---|---|---|
signatures/signature.go | 24 | Defines the `Signature` struct |
signatures/database.go | 36 | Connects to MongoDB |
signatures/server.go | 38 | Defines our application routes |
signatures/signatures_suite_test.go | 34 | Used by Ginkgo to run unit tests |
signatures/server_test.go | 124 | Unit tests |
main.go | 7 | Simply runs the server |
That's a grand total of only 263 lines of code! The rest of this post explains those lines.
Representing a Signature with the Signature
Struct
First, I define the Signature
struct.
When the server receives "GET /signature", it will grab these from the database and display them as JSON.
When it receives "POST /signature" with valid JSON data, it will create one of these and insert it into the database.
// signatures/signature.go
package signatures
import "labix.org/v2/mgo"
/*
Each signature is composed of a first name, last name,
email, age, and short message. When represented in
JSON, ditch TitleCase for snake_case.
*/
type Signature struct {
FirstName string `json:"first_name"`
LastName string `json:"last_name"`
Email string `json:"email"`
Age int `json:"age"`
Message string `json:"message"`
}
/*
I want to make sure all these fields are present. The message
is optional, but if it's present it has to be less than
140 characters--it's a short blurb, not your life story.
*/
func (signature *Signature) valid() bool {
return len(signature.FirstName) > 0 &&
len(signature.LastName) > 0 &&
len(signature.Email) > 0 &&
signature.Age >= 18 && signature.Age <= 180 &&
len(signature.Message) < 140
}
/*
I'll use this method when displaying all signatures for
"GET /signatures". Consult the mgo docs for more info:
http://godoc.org/labix.org/v2/mgo
*/
func fetchAllSignatures(db *mgo.Database) []Signature {
signatures := []Signature{}
err := db.C("signatures").Find(nil).All(&signatures)
if err != nil {
panic(err)
}
return signatures
}
Establishing a MongoDB Session
I need to connect to MongoDB to retrieve and insert signatures. The main database will eventually contain tons of signatures, but I want my unit tests to run with a fresh database every time.
Therefore, when establishing a session, I'll allow the caller to specify the name of the database. The main app database will be called "signatures", and the tests will use "signatures_test".
// signatures/database.go
package signatures
import (
"github.com/go-martini/martini"
"labix.org/v2/mgo"
)
/*
I want to use a different database for my tests,
so I'll embed *mgo.Session and store the database name.
*/
type DatabaseSession struct {
*mgo.Session
databaseName string
}
/*
Connect to the local MongoDB and set up the database.
*/
func NewSession(name string) *DatabaseSession {
session, err := mgo.Dial("mongodb://localhost")
if err != nil {
panic(err)
}
addIndexToSignatureEmails(session.DB(name))
return &DatabaseSession{session, name}
}
/*
Add a unique index on the "email" field.
This doesn't prevent users from signing twice,
since they can still enter
"dudebro+signature2@exmaple.com". But if they're
that clever, I say they deserve the extra signature.
*/
func addIndexToSignatureEmails(db *mgo.Database) {
index := mgo.Index{
Key: []string{"email"},
Unique: true,
DropDups: true,
}
indexErr := db.C("signatures").EnsureIndex(index)
if indexErr != nil {
panic(indexErr)
}
}
/*
Martini lets you inject parameters for routing handlers
by using `context.Map()`. I'll pass each route handler
a instance of an *mgo.Database, so they can
get and insert signatures.
For more information, check out:
http://blog.gopheracademy.com/day-11-martini
*/
func (session *DatabaseSession) Database() martini.Handler {
return func(context martini.Context) {
s := session.Clone()
context.Map(s.DB(session.databaseName))
defer s.Close()
context.Next()
}
}
Defining the Routes
The server responds to requests by retrieving or creating signatures.
// signatures/server.go
package signatures
import (
"github.com/go-martini/martini"
"github.com/martini-contrib/binding"
"github.com/martini-contrib/render"
"labix.org/v2/mgo"
)
/*
Wrap the Martini server struct.
*/
type Server *martini.ClassicMartini
/*
Create a new *martini.ClassicMartini server.
We'll use a JSON renderer and our MongoDB
database handler. We define two routes:
"GET /signatures" and "POST /signatures".
*/
func NewServer(session *DatabaseSession) Server {
// Create the server and set up middleware.
m := Server(martini.Classic())
m.Use(render.Renderer(render.Options{
IndentJSON: true,
}))
m.Use(session.Database())
// Define the "GET /signatures" route.
m.Get("/signatures", func(r render.Render, db *mgo.Database) {
r.JSON(200, fetchAllSignatures(db))
})
// Define the "POST /signatures" route.
m.Post("/signatures", binding.Json(Signature{}),
func(signature Signature,
r render.Render,
db *mgo.Database) {
if signature.valid() {
// signature is valid, insert into database
err := db.C("signatures").Insert(signature)
if err == nil {
// insert successful, 201 Created
r.JSON(201, signature)
} else {
// insert failed, 400 Bad Request
r.JSON(400, map[string]string{
"error": err.Error(),
})
}
} else {
// signature is invalid, 400 Bad Request
r.JSON(400, map[string]string{
"error": "Not a valid signature",
})
}
})
// Return the server. Call Run() on the server to
// begin listening for HTTP requests.
return m
}
Taking the Server Out for a Spin
Having built the server, it's time to see if it actually works. I define an executable in main.go
that creates a new server object and runs it.
// main.go
package main
import "github.com/modocache/signatures/signatures"
/*
Create a new MongoDB session, using a database
named "signatures". Create a new server using
that session, then begin listening for HTTP requests.
*/
func main() {
session := signatures.NewSession("signatures")
server := signatures.NewServer(session)
server.Run()
}
Now it's time to compile and run this bad boy. You'll need mongod
running on a standard port, then execute:
src/signatures/ $ go install
src/signatures/ $ signatures
[martini] listening on :3000 (development)
Sweet! I can create a signature:
$ curl -i -X POST \
-H "Content-Type: application/json"
-d '{"first_name": "Cervantes", "last_name": "Foreman", "age": 21, "email": "cervantesforeman@zilch.com", "message": "Dolore irure proident."}' \
localhost:3000/signatures
HTTP/1.1 201 Created
Content-Type: application/json; charset=UTF-8
Date: Mon, 12 May 2014 06:41:49 GMT
{
"first_name": "Cervantes",
"last_name": "Foreman",
"email": "cervantesforeman@zilch.com",
"age": 21,
"message": "Dolore irure proident."
}
And I can grab a list of all signatures:
$ curl -i localhost:3000/signatures
HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Date: Mon, 12 May 2014 06:43:24 GMT
[
{
"first_name": "Cervantes",
"last_name": "Foreman",
"email": "cervantesforeman@zilch.com",
"age": 21,
"message": "Dolore irure proident."
}
]
Unit Testing the Server
Ginkgo provides a command-line executable to generate unit tests.
src/signatures/signatures/ $ ginkgo bootstrap
Generating ginkgo test suite bootstrap for test in:
signatures_suite_test.go
src/signatures/signatures/ $ ginkgo generate server
Generating ginkgo test for Server in:
server_test.go
Ginkgo uses signatures_suite_test.go
to kick off the unit tests, which are contained in server_test.go
.
Defining Factory Objects with gory
My unit tests will be creating a bunch of Signature
objects, both valid and invalid. I don't want to write a bunch of object creation boilerplate in my unit tests, though, so I used an object factory.
// signatures/signatures_test_suite.go
package signatures_test
import (
. "github.com/modocache/signatures/signatures"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
"fmt"
"github.com/modocache/gory"
"testing"
)
/*
Ginkgo generated this function
to kick off our unit tests. Hook into it
to define factories.
*/
func TestSignatures(t *testing.T) {
defineFactories()
RegisterFailHandler(Fail)
RunSpecs(t, "Signatures Suite")
}
/*
Define two factories: one for a valid signature,
and one for an invalid one (too young).
*/
func defineFactories() {
gory.Define("signature", Signature{},
func(factory gory.Factory) {
factory["FirstName"] = "Jane"
factory["LastName"] = "Doe"
factory["Age"] = 27
factory["Message"] = "I agree!"
factory["Email"] = gory.Sequence(
func(n int) interface{} {
return fmt.Sprintf("jane-doe-%d@example.com", n)
})
})
gory.Define("signatureTooYoung", Signature{},
func(factory gory.Factory) {
factory["FirstName"] = "Joey"
factory["LastName"] = "Invalid"
factory["Age"] = 10
factory["Email"] = "joey-invalid@example.com"
})
}
Writing the Tests
Now it's time to actually write the unit tests.
Note that when I built this application, I tested throughout its development. I don't recommend you save your tests for the very end. I've done so in this post so as to present the material more clearly.
Testing HTTP in Go is a snap, thanks to the httptest
package in the Go standard library. For more information, this blog post by Pivotal Labs is spectacular.
The gist is that we can use an instance of *httptest.ResponseRecorder
to access responses from our server.
// signatures/server_test.go
package signatures_test
import (
. "github.com/modocache/signatures/signatures"
"bytes"
"encoding/json"
"github.com/modocache/gory"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
"net/http"
"net/http/httptest"
)
/*
Convert JSON data into a slice.
*/
func sliceFromJSON(data []byte) []interface{} {
var result interface{}
json.Unmarshal(data, &result)
return result.([]interface{})
}
/*
Convert JSON data into a map.
*/
func mapFromJSON(data []byte) map[string]interface{} {
var result interface{}
json.Unmarshal(data, &result)
return result.(map[string]interface{})
}
/*
Server unit tests.
*/
var _ = Describe("Server", func() {
var dbName string
var session *DatabaseSession
var server Server
var request *http.Request
var recorder *httptest.ResponseRecorder
BeforeEach(func() {
// Set up a new server, connected to a test database,
// before each test.
dbName = "signatures_test"
session = NewSession(dbName)
server = NewServer(session)
// Record HTTP responses.
recorder = httptest.NewRecorder()
})
AfterEach(func() {
// Clear the database after each test.
session.DB(dbName).DropDatabase()
})
Describe("GET /signatures", func() {
// Set up a new GET request before every test
// in this describe block.
BeforeEach(func() {
request, _ = http.NewRequest("GET", "/signatures", nil)
})
Context("when no signatures exist", func() {
It("returns a status code of 200", func() {
server.ServeHTTP(recorder, request)
Expect(recorder.Code).To(Equal(200))
})
It("returns a empty body", func() {
server.ServeHTTP(recorder, request)
Expect(recorder.Body.String()).To(Equal("[]"))
})
})
Context("when signatures exist", func() {
// Insert two valid signatures into the database
// before each test in this context.
BeforeEach(func() {
collection := session.DB(dbName).C("signatures")
collection.Insert(gory.Build("signature"))
collection.Insert(gory.Build("signature"))
})
It("returns a status code of 200", func() {
server.ServeHTTP(recorder, request)
Expect(recorder.Code).To(Equal(200))
})
It("returns those signatures in the body", func() {
server.ServeHTTP(recorder, request)
peopleJSON := sliceFromJSON(recorder.Body.Bytes())
Expect(len(peopleJSON)).To(Equal(2))
personJSON := peopleJSON[0].(map[string]interface{})
Expect(personJSON["first_name"]).To(Equal("Jane"))
Expect(personJSON["last_name"]).To(Equal("Doe"))
Expect(personJSON["age"]).To(Equal(float64(27)))
Expect(personJSON["message"]).To(Equal("I agree!"))
Expect(personJSON["email"]).To(
ContainSubstring("jane-doe"))
})
})
})
Describe("POST /signatures", func() {
Context("with invalid JSON", func() {
// Create a POST request using JSON from our invalid
// factory object before each test in this context.
BeforeEach(func() {
body, _ := json.Marshal(
gory.Build("signatureTooYoung"))
request, _ = http.NewRequest(
"POST", "/signatures", bytes.NewReader(body))
})
It("returns a status code of 400", func() {
server.ServeHTTP(recorder, request)
Expect(recorder.Code).To(Equal(400))
})
})
Context("with valid JSON", func() {
// Create a POST request with valid JSON from
// our factory before each test in this context.
BeforeEach(func() {
body, _ := json.Marshal(
gory.Build("signature"))
request, _ = http.NewRequest(
"POST", "/signatures", bytes.NewReader(body))
})
It("returns a status code of 201", func() {
server.ServeHTTP(recorder, request)
Expect(recorder.Code).To(Equal(201))
})
It("returns the inserted signature", func() {
server.ServeHTTP(recorder, request)
personJSON := mapFromJSON(recorder.Body.Bytes())
Expect(personJSON["first_name"]).To(Equal("Jane"))
Expect(personJSON["last_name"]).To(Equal("Doe"))
Expect(personJSON["age"]).To(Equal(float64(27)))
Expect(personJSON["message"]).To(Equal("I agree!"))
Expect(personJSON["email"]).To(
ContainSubstring("jane-doe"))
})
})
Context("with JSON containing a duplicate email", func() {
BeforeEach(func() {
signature := gory.Build("signature")
session.DB(dbName).C("signatures").Insert(signature)
body, _ := json.Marshal(signature)
request, _ = http.NewRequest(
"POST", "/signatures", bytes.NewReader(body))
})
It("returns a status code of 400", func() {
server.ServeHTTP(recorder, request)
Expect(recorder.Code).To(Equal(400))
})
})
})
})
Running the Tests
Our tests get us pretty great coverage:
src/signatures/ $ ginkgo -r --randomizeAllSpecs -cover
Running Suite: Signatures Suite
===============================
Random Seed: 1399880786 - Will randomize all specs
Will run 8 of 8 specs
...
Ran 8 of 8 Specs in 5.055 seconds
SUCCESS! -- 8 Passed | 0 Failed | 0 Pending | 0 Skipped PASS
coverage: 93.5% of statements
Ginkgo ran in 8.278165839s
Test Suite Passed
Note that you need MongoDB running in order to execute the tests.
Where to Go from Here
You can check out the source code for this project on GitHub. Try adding some features and sending a pull request:
- Prevent users with the same email from submitting multiple signatures, even if they use "dudebro+foo@example.com"
- Add a route to show a single signature, using its email address as a key
For more information on unit testing and continuous integration in Go, check out another post of mine: Continuous Integration in Go: Ginkgo & Coveralls.
Tweet