Testing in Go

This post is based on a presentation I gave for the Boston Go group in May. You can watch the video here.

For me, Go is like coming back to formal programming after the crazy world of Ruby for a little while. It reminds me of C with all the sharp corners sanded off; Go is such a simple language.

Some of my favorite things in the programming world are tests, web applications, and databases. Half of all my talks are about tests, in fact. So let’s talk about testing in Go.

The most basic example of testing in Go just needs whatever package you’re working in -- you don’t change it -- you import “testing,” and then any function that you start with Test is going to get run by Go’s testing harness. It’ll take one parameter (a testing T), and you can use that to call the different testing functions in Go.

package basics
import (
    "encoding/hex"
    "testing"
)
func TestTruth(t *testing.T) {
    if true != true {
        t.Error("everything I know is wrong")
    }
}

So here, I’m just making sure that true is equal to true. Instead of doing something in the affirmative and saying “true is equal to true, or we should crash,” I’m saying “If true isn’t true then there’s an error.” Generally, you talk about things in the worst case with Go, which I think flows with the way you write Go: calling methods and checking errors and doing something if something goes wrong.

Let’s talk about some of the testing functions on T. You can call errors, you can fail the tests, you can see if it failed. Fatal will make an error message, but it will also stop the test right where it is. You can also just log debugging stuff, so when you turn your verbosity up, you can see all your messages. You can leave them in there for yourself if you ever have to debug a test.

func TestBroken(t *testing.T) {
    if true != false {
        t.Error("expected", true, "got", false)
    }
    if 1+1 != 4 {
        t.Fatal("Can't proceed!", 1+1, 4)
    }
}

But 99 percent of the time, the only ones you’ll care about are Error and Fatal. Fatal is just an error plus a fail. So Error is like you’re making a big list of everything that’s going wrong on this test; you can have a whole ton of errors, and this will list them all. But with Fatal, it will stop. You can’t even do your other checks, so you have to pretty much bail out of the test at this point.

Often, I’ll just use Fatal because the tests depend on the pieces you’ve created as you go along. But sometimes, if you have a big object with a ton of fields, it’s nice to be able to do a lot of error statements and see exactly which ones worked and which ones didn’t while you’re trying to work on some database persistence, for example.

$ go test -v ./basics/
=== RUN TestBroken
--- FAIL: TestBroken (0.00s)
    package_test.go:13: expected true got false
    package_test.go:17: Can’t proceed! 2 4
FAIL
exit status 1
FAIL github/com/ngauthier/testing-go-presentation/basics    0.006s

So when you run this, the error will say expected true got false, and then it will continue on to the fatal that says Can’t proceed! 2 4

The neat thing here is that you can put as many arguments as you want into Error and Fatal, and it’ll stringify them nicely. It’s a really nice interface.

func TestError(t *testing.T) {
    dest := make([]byte, 0)
    if _, err := hex.Decode(dest, []byte{8}); err != nil {
        t.Error(err)
    }
}

$ go test -v -run Error ./basics/
=== RUN TestError
--- FAIL: TestError (0.00s)
    package_test.go:27: encoding/hex: odd length hex string
FAIL
exit status 1
FAIL github/com/ngauthier/testing-go-presentation/basics    0.004s

A really common pattern in Go (and therefore in tests) is errors. In this case, I’m going to decode some hex, and I put in some invalid bytes that aren’t going to decode into hex. So when we run this, we get an error. You’ll do the same thing as you do with a normal error check in Go, but then you just give that error directly to Error. It’ll print your error out with a nice little line number where the error came out.

$ go test -v -run Error ./basics/
    -run regexp
        Run only those tests and examples matching the regular expression.
    -v
        Verbose output: log all tests as they are run. Also print all text from Log and Logf calls even if the test succeeds.

You can give it a run with a regex, which can just be a string if you want to just run an individual test. That can help when you’re refactoring, and you make a little change, and it breaks everything, and you have to go hunt down individual pieces and just want to focus on a piece of the puzzle.

To level up our example, let’s talk about what a lot of people do with Go, which is HTTP servers.

type App struct {
    Message string
}
func (a *App) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    w.Write([]byte(a.Message))
}

Here’s an app that stores a message. I’m going to implement ServeHTTP on my app. That means that this app is going to function as an http.handler, so Go’s HTTP server will be able to accept the app. All the app does is write the message out, no matter the parameters you give it. Pretty dumb server.

func TestHello(t *testing.T) {
    app := &App{Message: "hello"}
    s := httptest.NewServer(app) // http provides its own test objects
    defer s.Close()
    resp, err := http.Get(s.URL) // test from the outside
    if err != nil {
        t.Error(err)
    }
    if body, err := ioutil.ReadAll(resp.Body); err != nil {
        t.Error(err)
    } else if string(body) != "hello" {
        t.Error("expected", "hello", "got", body)
    }
}

I’m going to test this by creating the app, then I’m going to use the HTTP test package’s NewServer, which is a lot like http.server. However, they’ve done a couple of nice things for you -- you don’t have to specify the port that you want to run it on, for example. If there’s a collision, they’ll re-randomize the port for you. So you get your own little server that’s going to work well in a test scenario.

Then I’m going to get on that server’s URL; we’re really testing from the outside. If you were to boot up a server, serving my app, and somebody hit it as an HTTP client and actually connected to that socket and made that request, what are you going to get? This is very different than if I had said app.servehttp and then given it a response writer and a request object.

Now I’m going to convert the body to a string and compare that to hello, which is what I initialized the message with. If that wasn’t hello, I’m going to do my error with expected and got. You’ll notice that I still made expected, hello, and got as three separate strings with commas. It’s nice to keep that format.

func fakeApp(msg string) *httptest.Server {
    app := &App{Message: msg}
    return httptest.NewServer(app)
}
func get(t *testing.T, s *httptest.Server, path string) string {
    resp, err := http.Get(s.URL + path)
    if err != nil {
        t.Error(err)
    }
    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        t.Error(err)
    }
    return string(body)
}

Let’s talk about factories and helpers. Tests are code too; they’re not just scripts that run alongside your code. It helps to treat your tests like real code, and you’ll see this in the standard library tests too.

In this case, I wrote two helpers to make this easier on me. This is my fake app, which is going to run an HTTP test server. I’ll initialize the app with whatever message I passed to the server, and I’ll return an HTTP test version of the server.

The second one I wrote is get. You give it your testing instance and a server and a path, and it makes a get request to that server and path, checks for error. It reads the body and checks for an error and returns the string version of that body. So this is going to do the whole get for me and send it back for any path on my server.

func TestHelpedHello(t *testing.T) {
    s := fakeApp("hello")
    defer s.Close()
    if body := get(t, s, "/"); body != "hello" {
        t.Error("expected", "hello", "got", body)
    }
}

Once I have these two, my test gets a lot easier to read. In this case, my get is doing some error checks for me. I don’t just have a lot of little helpers that don’t do anything. I’ve made get do some assertions to make sure that when I got something back from get, it was going to work.

So that was all about testing individual components. Let’s walk through an example of testing interactions between components -- that’s usually where all the interesting testing stuff comes from. When you have two big, complicated pieces working together, that’s when everything becomes difficult.

In this case, I’m using a simple example of a train and an engine.

type Engine struct {
    speed int
}
func NewEngine() *Engine {
    return &Engine{}
}
func (e *Engine) Speed() int {
    return e.speed
}
func (e *Engine) Accel() {
    e.speed += 10
}
func (e *Engine) Decel() {
    e.speed -= 10
}

The engine is going to store private speed. It’ll have a constructor that’s going to return a pointer to an engine struct; Accel is going to increment speed by 10; Decel is going to decrement speed by 10.

func TestEngine(t *testing.T) {
    e := NewEngine()
    expectSpeed(t, e, 0)
    e.Accel()
    expectSpeed(t, e, 10)
    e.Decel()
    expectSpeed(t, e, 0)
}

To test the engine, we make a new engine. I expect speed to be 0. I’ll accelerate it, expect my speed to be 10; I’ll decelerate it and expect to be back at 0.

func expectSpeed(t *testing.T, e *Engine, s int) {
    actual := e.Speed()
    if actual != s {
        t.Fatal("expected", s, "got", actual)
    }
}

expectSpeed will compare the actual to the speed you gave it and give a fatal if the speed was not equal to what was expected. This is another example of having a helper, so you don’t have to repeat these four lines every single time.

Note: As your software gets more complicated, you’re probably going to want some helpers to set things up. But it should also alert you -- if you have a lot of helpers, it means your code is hard to use because it requires a lot of set up. So at that point, it may be time to break things into packages.

type Train struct {
    engine *Engine
}
func NewTrain() *Train {
    return &Train{
        engine: NewEngine(),
    }
}
func (t *Train) Go() {
    t.engine.Accel()
    t.engine.Accel()
}
func (t *Train) Stop() {
    t.engine.Decel()
    t.engine.Decel()
}
func (t *Train) Speed() int {
    return t.engine.Speed()
}

Now I’ve got a private engine on the train. My constructor that returns a train is going to call the engine constructor to give us a new engine. Go just accelerates twice, Stop is going to decelerate twice, and Speed will return the speed.

The thing that I really care about is how Train and Engine are going to work together and how they’re constructed.

func expectTrainSpeed(t *testing.T, tr *Train, s int) {
    actual := tr.Speed()
    if actual != s {
        t.Fatal("expected", s, "got", actual)
    }
}
func TestTrain(t *testing.T) {
    tr := NewTrain()
    tr.Go()
    expectTrainSpeed(t, tr, 20)
    tr.Stop()
    expectTrainSpeed(t, tr, 0)
}

The train test is going to be pretty simple -- looks pretty similar to what we had before.

The train actually has to have a real engine, and it actually does stuff to the engine. That’s the only way our test is going to pass -- they are integrated together. We’re testing acceleration by inspecting speed.

However, we’re not looking at how the train and the engine are interacting with each other. If we really unit tested the train, we’d give it a fake engine that it was going to play around with. We’d tell the train to go, and then we’d see what the train tried to do to the engine. Notice that such a test would only test Go. It’s not testing Go and Stop and Speed like the integration test was.

Obviously this example is really contrived. If your code’s that simple, there’s no reason to go to those lengths. But what if Engine was some remote resource that you had to call over a JSON API through your own Train, and it was hard to set up and requires a bunch of authentication keys and has to be running on a port and all that junk? Then you’d really want to start faking out your engine. After all, we don’t really care about the engine, we just want to make sure the train does the right thing with the engine.

type Speeder interface {
    Speed() int
}
type Engine interface {
    Speeder
    Accel()
    Decel()
}
type engine struct {
    speed int
}
func NewEngine() Engine {
    return &engine{}
}
func (e *engine) Speed() int {
    return e.speed
}
func (e *engine) Accel() {
    e.speed += 10
}
func (e *engine) Decel() {
    e.speed -= 10
}

To be able to work with a fake engine, we have to refactor our engine behind an interface. To do that, I tossed in the Speeder interface. The big change is that instead of being a struct, Engine is now an interface type. It’s a speeder, and it will provide acceleration and deceleration.

Our actual engine is going to become a private struct engine. People outside of the engine package will not be able to access our engine struct. They’re only going to be able to see the interface. Our constructor is going to make an instance of the private struct engine and then return it as an engine interface. So we’re making an engine and returning it as an Engine, and Go is going to cast it over for us. It’ll allow it if our engine implements the Engine interface.

To implement that interface, we have to implement Speed, and we have to implement Accel and Decel.

So we never expose any struct attributes in our public interface. We’re pretty safe, and this refactor will be pretty easy. So now we can pass the engine around.

func expectSpeed(t *testing.T, speeder Speeder, speed int) {
    actual := speeder.Speed()
    if actual != speed {
        t.Fatal("expected", speed, "got", actual)
    }
}
func TestEngine(t *testing.T) {
    e := NewEngine()
    expectSpeed(t, e, 0)
    e.Accel()
    expectSpeed(t, e, 10)
    e.Decel()
    expectSpeed(t, e, 0)
}

Now, we have a totally public-facing interface for Engine. This test is pretty much the same as when Engine was a struct. It’s just now with an interface, which means all we can do is access methods on it.

type FakeEngine struct {
    AccelCalls int
    DecelCalls int
}
func NewFakeEngine() *FakeEngine {
    return &FakeEngine{}
}
func (e *FakeEngine) Accel() {
    e.AccelCalls += 1
}
func (e *FakeEngine) Decel() {
    e.DecelCalls += 1
}
func (e *FakeEngine) Speed() int {
    return 0
}

Now let’s make a fake engine. We want to implement the engine interface with it. This is pretty much the dumbest fake I could make -- there are way more interesting ways to do these, but the goal here isn’t so much to implement an engine as it is to keep track of what the interactions were like. The real goal here is to test Train’s behavior and test it without needing a real engine.

type Train struct {
    engine Engine
}
func NewTrain() *Train {
    return &Train{
        engine: NewEngine(),
    }
}

We have to do a little refactoring on Train. The only thing that changes is Engine. Since it’s an interface, you don’t want to pass pointers to interfaces around. It gets very confusing very fast. So you always just pass the interface around. You won’t actually know if it’s a pointer or not, which is a really fun thing about Go.

So our engine is now just an instance of the Engine interface. We also call the constructor as we normally would, so no big changes there.

func trainWithFakeEngine() (*Train, *FakeEngine) {
    t := NewTrain()
    e := NewFakeEngine()
    t.engine = e
    return t, e
}
func TestTrainGo(t *testing.T) {
    tr, e := trainWithFakeEngine()
    tr.Go()
    if e.AccelCalls != 2 {
        t.Error("expected", 2, "got", e.AccelCalls)
    }
}

Notice that my test’s name has changed. It used to be TestTrain, and now we’re testing TrainGo. We’re going to call my constructor that gives us a train and a fake engine, we’re going to call TrainGo, and then we’re expecting that the engine had Accel called twice. It’s like if you took a train, set it up in a warehouse, plugged something else into the port that controls the engine, and just made sure it said “acceleration” twice. As opposed to, you know, putting an actual train on an actual track and seeing if it runs into anything.

Next, I’m going to do what’s called setter injection. I take the engine, and I inject that dependency into Train. I can do this because I’m in the same package. Because this test is in the same package along with Train, I can access private methods. I can screw around with objects in ways that our users aren’t allowed to do.

func trainWithFakeEngine() (*Train, *FakeEngine) {
    t := NewTrain()
    e := NewFakeEngine()
    t.engine = e
    return t, e
}
func TestTrainStop(t *testing.T) {
    tr, e := trainWithFakeEngine()
    tr.Stop()
    if e.DecelCalls != 2 {
        t.Error("expected", 2, "got", e.DecelCalls)
    }
}

It's usually a good idea to use an interface for anything you’re going to be exposing publicly. The only time you wouldn’t want to do that is when you have, for example, data objects that are often going to be value objects. Such as anything that represents a request payload. People are just going to interact with its fields, not really call methods on it.

But usually interfaces are the way to go, for a few reasons:

• Future flexibility. If you make a really big package based on structs and calling structs and refactoring all that, it is a huge pain to refactor it over to an interface. But if you do an interface from the start, all you have to do is keep track of the methods on that interface. It forces you to keep it clean too.

• You have to expose functionality with methods. You can’t let people just access struct fields by making them public. This is due to the uniform access principle, which says that you should work with data the same way no matter how it’s stored or implemented. By forcing yourself to use methods like getters and setters, you don’t have to refactor all the code that uses what you’ve written down the road.

• Easy to fake. It’s super easy to swap in a fake, and the library doesn’t have a clue.

• A user can write their own versions of something and use your code.

With concurrent tests, we have to first test for coordination. We’re going to have at least two pieces of code -- two Go routines -- moving at the same time, and we need to see how they coordinate with each other.

type Engine interface {
    Speeder
    Accel()
    Decel()
    Stop()
    Run()
}

We’re going to add run and stop to our engine. Like turning a key, it’s sort of running off to the side while we’re sending our Accel and Decel instructions to it. When we’re done, we’ll stop it, or turn it off.

type accelCommand struct {
    done chan bool
}
func newAccelCommand() *accelCommand {
    return &accelCommand{done: make(chan bool)}
}
type engine struct {
    speed    int
    stop     chan bool
    done     chan bool
    accelcmd chan *accelCommand
}
func newEngine() *engine {
    return &engine{
        accelcmd: make(chan *accelCommand),
        stop:     make(chan bool),
        done:     make(chan bool),
    }
}
func NewEngine() Engine {
    return newEngine()
}

We’re going to create a new version of Accel, a blocking version. To do that, we’re going to use a channel. My engine will now have a stop channel, a done channel (to say that it has successfully stopped), and an acceleration command channel.

The first thing you’ll do is instantiate an acceleration command. Then you’ll send that command down the acceleration command channel. Then you’ll wait on the command’s done channel.

func (e *engine) Run() {
    for {
        select {
        case c := <-e.accelcmd:
            e.speed += 10
            time.Sleep(100 * time.Millisecond)
            close(c.done)
        case <-e.stop:
            close(e.done)
            return
        }
    }
}

To go along with the block Accel, we have to have an engine runloop. Our engine’s run is going to be selecting in two channels. If we get a stop, we’ll close the done channel and return out of my runloop. If we get an acceleration command, we’re going to increment our speed, and then I’ll close my command’s done channel to say we’ve successfully accelerated, and I’ll go back into my loop.

So if we want to accelerate, we have to have Run running at the same time. If we try to call Accel, and we’re not running our engine somewhere, it’s going to deadlock immediately.

func (e *engine) Stop() {
    e.stop <- true
    <-e.done
}

All that stop does is send a true down the stop channel and waits for done. It needs to make sure you’re not in the middle of acceleration. If you are, it waits for acceleration to finish, then it needs to go back into the select, and then it will grab the stop and receive that. This ensures you don’t get cut off in the middle of something.

func TestEngineAccel(t *testing.T) {
    e := newEngine()
    done := make(chan bool)
    go func() {
        e.Accel()
        close(done)
    }()
    c, ok := <-e.accelcmd
    if c == nil || !ok {
        t.Fatal("expected acceleration command")
    }
    close(c.done)
    <-done
}

This is what concurrent tests look like for me. I’ll do my setup -- instantiate my engine. Then I’ll have my own done channel. I’m going to background Accel but have my foreground code be doing the testing.

When Accel returns, I’ll close my done. In this case, I’m unit testing Accel; I’m not even testing the train’s own acceleration with the train’s own runloop here. I’m just making sure my accelerator sends the right command. Now if this were a more complicated command, like something that’s going off to an API, you’d want a unit test like this to make sure that the message sent by Accel was appropriate.

We’re unit testing only private implementation here, and that comes with its own tradeoffs. This test will be very brittle, along with the code it’s testing. If you want to change any of the private implementations, you have to change the test to go with it. These tests are useful for development, less useful for regression, and not useful at all for refactoring.

func TestEngine(t *testing.T) {
    e := NewEngine()
    go e.Run()
    defer e.Stop()
    expectSpeed(t, e, 0)
    e.Accel()
    expectSpeed(t, e, 10)
    e.Decel()
    expectSpeed(t, e, 0)
}

This is the kind of test that you really want to keep around. It’ll work off a public interface without concerning itself with private implementation. So this is testing concurrent code, but you barely even know it because you’re going to call Run and Stop. It’s super easy to run, and it tests a whole bunch of code.

So, how do we combine the fake engine idea with the concurrent code idea?

We could mark some timestamps, or to go a little further, we could have log of events that we had. For concurrency tools in Go, we have Go routines, channels, and sync. We could use locks.

For our example, the accelerator could wait on lock, like the Accel of my fake could wait on a lock. My doAccel private method could then unlock that lock. That way, my engine could block on Accel until exactly when my test decided to unlock Accel and let my code proceed.

The other one was the channel. So, Accel could wait on some performing acceleration channel, and then my doAccel could send a message saying to go ahead and perform the acceleration.

There really isn't a trick to any of this. The same code, the same libraries you have with Go are available in tests. Remember that tests are also code. You can use all your tricks, all your design patterns, all of your strategies in your tests the same way you can in your code.

If you want to dig a little further, check out my post “Creating Fakes in Go with Channels.”

• Slides

• Code

• Video

[youtube https://www.youtube.com/watch?v=v\_wz6E3uFRg&w=560&h=315\]