API Integration Testing

Different approaches to integration testing API interactions.

Use real API

Test against an offsite, client-maintained, sandboxed API.

Pros

  • testing against a real api can provide better feeback from tests
  • always stays in sync (hopefully)
  • least amount of work to setup

Cons

  • api changes can break your tests, even without any code changes
  • slow tests are evil
  • difficult to start each test with known, consistent state

Setup mock API

Test against an onsite, contractor-maintained fake API. API may be on office network as a shared service, or each developer may have separate copy.

Pros

  • testing against a "real" api can provide better feeback from tests
  • can be faster than using the real api
  • easier to keep tests and api syncronized

Cons

  • mock api may be out of sync with real api
  • contractors now have two projects to maintain
  • still pretty slow
  • still difficult to reset state before each test

Mock all the things

Build mock objects in tests for all external dependencies.

Pros

  • fast tests
  • tests do not fail when api changes

Cons

  • hard to know what is being tested (integration test?)
  • also difficult to inject a mock into server process (from test process)
  • have to assume mocked responses are correct
  • still need to hit actual api to verify tests
  • tests do not fail when api changes

Write custom connection layer (with in-memory store)

Build mock connection that gets swapped in when testing.

Pros

  • only mock what needs to be mocked
  • fast tests
  • tests do not fail when api changes
  • easy to change (basic input/output)

Cons

  • have to assume mocked responses are correct
  • still need to hit actual api to verify tests
  • tests do not fail when api changes