5 Ways To Speed Up Go Tests

A slow test-suite/build is one of the most frustrating hurdles to productivity for software engineers. Here are a few tips to speed things up with Go…

1. Run Tests in Parallel

Go runs each test in its own Goroutine, but tests in the same package are not run in parallel by default. Simply adding t.Parallel() at the start of your test will ask the testing package to do this for you.

2. Cache Results and Dependencies in CI

Whilst Go already caches test results and dependencies locally, this often isn’t useful in CI - at least not by default. There are often tools/plugins out there that can help with this. It’s especially easy with GitHub Actions - actions/setup-go will cache your test results and dependencies between builds, even across multiple runner operating systems.

- uses: actions/setup-go@v3
  with:
    cache: true
    cache-dependency-path: go.sum

This can speed up CI builds dramatically!

3. Remove time.Sleep()

Where possible, you should replace any calls to time.Sleep() in both your tests and in the code being tested. Instead, use concurrency tools like channels or WaitGroups.

For example:

func DoMyThing(job Job) {

    go otherThing.Start()

    // wait for otherThing to be ready
    time.Sleep(time.Second)

    otherThing.DoWork(job)
}

Could be replaced with:

func DoMyThing() {

    var wg sync.WaitGroup
    otherThing.Start(wg)

    // wait for otherThing to be ready
    wg.Wait()

    otherThing.DoWork(job)
}

This not only has the benefit of saving time when otherThing starts up in less time than a second, it also means things don’t break when it takes more than a second.

4. Use Interfaces To Avoid Excessive I/O

Tests which rely on excessive I/O are often among the slowest ones.

Let’s take a basic example:

func AnalyseFile(path string) error {
  f, err := os.Open(path)
  if err != nil {
    return err
  }
  defer f.Close()

  data, err := ioutil.ReadAll(f)
  if err != nil {
    return err
  }

  return doSomeWork(data)
}

The above code takes a file path, reads in the contents of the file, and then does some work with the data. If we want to test this with a lot of different inputs, it’d be much faster to use an interface to abstract away the I/O, such as:

func AnalyseFile(r io.Reader) error {
  data, err := ioutil.ReadAll(r)
  if err != nil {
    return err
  }

  return doSomeWork(data)
}

This allows us to write a self-contained test which requires no extra I/O:

func TestAnalyseFile(t *testing.T) {
    err := AnalyseFile(bytes.NewReader(`here is my content`))
    if err != nil {
      t.Errorf("unexpected error: %v", err)
    }
}

This becomes even more effective when refactoring code which accesses filesystems - perhaps recursively searching for particular file types within a directory - to use the io.fs interfaces instead. This allows the use of an in-memory filesystem, instead of passing in a directory name to be analysed.

5. Write a TestMain() Function

It’s often required to set up test dependencies before running tests, and tear them down afterwards. Instead of doing this at an individual test level, it’s often possible to set things up once before all the tests, and tear them down afterwards.

You should be careful with stateful dependencies, as they may be shared across tests, and require recreating each time.

You can do this using a TestMain function. This is a kind of wrapper that runs your tests - if it exists, your tests won’t run by default, instead you take responsibility for running them inside this function.

For example:

func TestMain(m *testing.M) {

    // set up our dependencies
    setup()

    // run our tests
    exitCode := m.Run()

    // tear down our dependencies
    teardown()

    // exit based on the test results
    os.Exit(exitCode)
}

Note you need to set the exit code manually - otherwise go test will exit with 0 - even if your tests fail!

It’s also possible to nest your tests to achieve a similar result, for example:

func TestEverything(t *testing.T) {
  setup()
  defer teardown()
  t.Run("a sub test", func(t *testing.T) {
      t.Run("a sub sub test", func(t *testing.T) {
          // do stuff here...
      })
  })
  t.Run("another sub test", func(t *testing.T) {
     // do stuff here...
  })
}

This works best when combined with table-driven tests.