TL;DR⌗

Separate integration tests into separate *_integration_test.go files
Add // +build integration to the top of these files to not run them be default when using go test
Use a .env file to store sensitive info required by your integration tests
Use go test -tags=integration to run your tests

Updates⌗

2020-10-06 - Updated with suggestions for additional VSCode settings

Overview⌗

Golang makes testing extremely easy. In most cases, all that’s needed is to create a file called appended with _test.go, then write a test function. For example, given a function like the following:

go add.go


func Sum(i, j int) int {
  return i + j
}

We can write a test function like this:

go add_test.go


func TestSum(t *testing.T) {
  sum := Sum(2, 2)
  if sum != 4 {
    t.Errorf("Expected %v, got %v", 4, sum)
  }
}

However, let’s imagine that we discovered a brand new web API that will take care of all our mathematical needs. Let’s imagine this server lives at math.example.com, and the summation API can be called like so:

GET http://math.example.com/add?a=2&b=2&authtoken=abcdef123

{
 "result": 4
}

To migrate to this API, we need to create a function like this:

go add.go


type MathClient struct {
  Token string
  Host  string
}

type AddResult struct {
  Result int `json:"result"`
}

func (c *MathClient) APISum(i, j int) (int, error) {
  query := fmt.Sprintf("http://%s/add?a=%v&b=%v&authtoken=%v", c.Host, i, j, c.Token)

  response, err := http.Get(query)
  if err != nil {
    return 0, err
  }
  defer response.Body.Close()

  data, err := ioutil.ReadAll(response.Body)
  if err != nil {
    return 0, err
  }

  a := AddResult{}
  err = json.Unmarshal(data, &a)
  if err != nil {
    return 0, err
  }

  return a.Result, nil
}

We also need to create a new test function:

go add_test.go



func TestAPISum(t *testing.T) {
  client := MathClient{
    Token: "abcdef123",
    Host:  "math.example.com",
  }

  sum, err := client.APISum(2, 2)
  if err != nil {
    t.Errorf("No error expected, got %v", err)
  }

  if sum != 4 {
    t.Errorf("Expected %v, got %v", 4, sum)
  }
}

Since this tests how we integrate with an external API rather than being fully self-isolated, we can consider it an integration test rather than a basic unit test.

The Problem⌗

With these modifications, we can now rely on this external service while still testing that our package works as expected. However, this immediately introduces a couple of problems into our testing suite:

Our test suite is now dependent on another service. Our test suite will start failing if the API goes down, we have a network connection issue, or we hit an API rate-limit.
Since network connections take significantly longer than Golang functions, these API calls could start slowing down our test suite significantly.
We have to start managing potentially sensitive configurations for our test suite. Right now we have a hardcoded authentication token sitting in our test suite, which is lousy security practice.

While there are ways we could potentially stub out hardcoded API responses and remove our dependency on the external service, there’s a lot of benefit to perform integration tests against the real API. However, we need to separate these fragile, expensive tests so they can be run less frequently. We also need to separate out any sensitive information from the test itself.

Recommendations⌗

1. Separate out the integration tests into their own files⌗

This can be whatever naming convention you’d prefer, but my preference is *_integration_test.go. In this case, we would move the entire contents of function TestAPISum into a new file add_integration_test.go.

2. Fence off integration tests with build tags⌗

Golang build tags are a means of separating which code gets included in our builds. We can use these to fence off our integration tests by adding the following to the first line of the integration test file:

go add_integration_test.go


// +build integration

Now we have different options when running go test:

go test # Run all unit tests in our package
go test -tags=integration # Run all unit and integration tests in our package
go test -tags=integration -run ^TestAPISum$ # Run a specific integration test

This means we can run our unit tests as often as we like, while only running the integration test suite as-needed.

3. Use a .env file to hold our sensitive configuration info⌗

.env files are a great way to store sensitive info without exposing it in our git repository. First, make sure you have the file excluded in your .gitignore:

txt .gitignore


.env

Then add your sensitive info as key-value pairs:

txt .env


AUTH_TOKEN=abcdef123

To make this available for our tests, we first need to load this .env file in the shell environment, then read the appropriate environment variables. One way of doing this is via the github.com/joho/godotenv module. We can alter our integration test like so:

go add_integration_test.go


func TestAPISum(t *testing.T) {
  err := godotenv.Load()
  if err != nil {
    t.Fatalf("could not load .env file: %v", err)
  }

  token := os.Getenv("AUTH_TOKEN")

  client := MathClient{
    Token: token,
    Host:  "math.example.com",
  }

  ...

4. (For VSCode) Make gopls aware of our integration tags⌗

The gopls language server has historically had some trouble with build tags. To ensure that your new build tag is recognized by it, you can add this settings to your settings.json:

json settings.json


{
  "gopls.env": {
    "GOFLAGS": "-tags=integration"
  }
}

VSCode will complain about the gopls.env setting not existing, but according to this comment, this is a setting that gopls will respect.

5. (For VSCode) Add Debug Configurations for Easy Debugging⌗

Normally, debugging tests with VSCode is simple with the official Golang extension. Above each test are run test and debug test buttons:

Golang test buttons

To debug a test, all we need to do is set a breakpoint and hit debug test. However, if we click this button on one of our integration tests, it won’t work. This is because VSCode doesn’t know about our custom build tag, and it will skip over any tests protected with this tag. Fortunately, we have a couple of options to work around this.

Option 1: Make VSCode Go tools aware of the integration tag⌗

If you’re fine with always running your integration tests along with your unit tests, you can just stick this into your settings.json:

json settings.json


{
  "go.toolsEnvVars": {
    "GOFLAGS": "-tags=integration"
  }
}

Option 2: Create a custom launch definition⌗

Another option is to stick a new definition in our launch.json that looks like the following:

json launch.json


{
  // Use IntelliSense to learn about possible attributes.
  // Hover to view descriptions of existing attributes.
  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Debug Golang test",
      "type": "go",
      "request": "launch",
      "mode": "test",
      "program": "${fileDirname}",
      "buildFlags": "-tags=integration",
      "args": ["-test.run", "${selectedText}"]
    }
  ]
}

This configuration uses the ${selectedText} macro to restrict which function gets launched, and -tags=integration to enable our integration tests.

To debug our integration test:

Set the breakpoint at the desired location
Highlight the test name
Go to the Run menu
Run the Debug Golang test profile

Debugging Golang integration tests in VSCode

Now we can step through our integration test like normal.

Additional Resources⌗

Working example code can be found at https://gitlab.com/terrabitz/blog-examples/-/tree/master/golang_integration_testing

Golang Integration Testing