TDD and BDD for APIs: testing web applications that talk to each other

Two apps, one contract, and nobody checking it

Here is a situation that will sound familiar to anyone who has worked on more than one application. Two web applications need to talk to each other. Maybe it is a shop talking to a payments service. Maybe it is a main platform sending orders to a warehouse system. Whatever the setup, there are two separate codebases, often two separate teams, and an API contract sitting between them. When that contract breaks, both sides break with it.

I have spent years building systems where several applications coordinate through APIs. This article explains why testing between applications is different from testing inside one, the four approaches people take, how to write the tests in Ruby on Rails and in Django, how contract testing with Pact works, and why good API tests end up being the most reliable documentation you have. It builds on TDD and BDD in Ruby on Rails, which covers the basics of test first development.

Why testing between applications is different

When you test a Rails or Django app on its own, you control everything. Models, controllers, views and tests live in one codebase. Change something, and the suite tells you straight away whether you broke it.

When app A calls app B's API, that guarantee is gone:

• Separate releases. App A ships on Monday, app B on Thursday. What worked on Monday can break on Thursday.
• Separate teams. The people building app A may not know the people building app B changed something. The news arrives by chat message, by email, or as a production error.
• Separate test suites. App A's tests pass. App B's tests pass. Together, in production, they fail.
• An unreliable network. A function call inside one app does not time out halfway. An API call can.
• Old versions. App A may still call an endpoint that app B deprecated months ago and never removed.

I have seen all of these cause production incidents. The worst was a payments integration where the provider changed a field name in its response from transaction_id to txn_id. Their tests passed. Our tests passed, because we were mocking their responses from the old documentation. It took three days of failed payments before anyone noticed.

Worked example: how a rename slips through

The example is invented. Lavitt's Quay Coffee is a coffee shop in Cork whose online shop (Rails) sends orders to Raheen Fulfilment, an invented warehouse in Limerick whose system is written in Django. When a shipment is created, the shop reads tracking_number from the warehouse's response and emails it to the customer.

1. The shop's tests stub the warehouse with a fixed response containing tracking_number. They pass.
2. The warehouse renames the field to tracking_code and updates its own tests. They pass.
3. The warehouse deploys on Thursday. Nothing in either pipeline has compared the two sides.
4. From Thursday, the shop reads nil and emails every customer an empty tracking link.

Every test was green at every step. Nothing was wrong inside either app. The thing that broke was the agreement between them, and nobody was testing the agreement.

Four ways to test an integration

There are four broad approaches. Each has its place, and most production systems combine them.

End to end integration tests

Start both applications, point them at each other, and run tests through the full flow. This tests the real integration, catches problems a mock would miss, and gives high confidence when it passes.

The costs are real too. It is slow, because both applications need to be running with seeded databases. It is flaky, because network hiccups, timing and leftover test data cause failures that come and go. It is expensive to keep integration environments alive. And the feedback comes late, often only after a deploy to staging. Use it as a final check before a release, not as your main strategy.

Mock everything

The opposite extreme: each application replaces the other's API with a stub and tests in isolation. It is fast, reliable and runs on any laptop.

The catch is drift. The mock says the API returns user_id, the real API returns userId, your tests pass and production fails. Everything looks green while you are testing against an imagined version of the other app. Mocks are right for fast unit tests, never as the only line of defence.

Contract testing

The middle ground: write down a contract that both sides agree to, then test each side against the contract on its own. It is nearly as fast as mocking, it catches breaking changes before a deploy, and it works across languages, so a Rails app can hold a contract with a Django app.

It takes some learning, it needs somewhere to share the contracts, and it only tests the interface: each side's business logic still needs its own tests. It is the right main strategy for an integration between applications you control, with a few end to end tests on top.

Consumer driven contracts

A specific flavour of contract testing, described by Ian Robinson in 2006 as a pattern for evolving services.¹ The consumer (the app calling the API) states what it needs. The provider (the app serving the API) proves it can meet those needs.

Consumers only check what they actually use, so a provider can see exactly which change would break which consumer, and the contracts themselves record who depends on what. It needs buy in from both teams, and with many consumers the coordination grows. The Pact documentation is frank about where it does not fit: public APIs, where the consumers cannot be individually identified, and integrations where the other team will not use Pact.²

Worked example: three fields out of fifty

Say Raheen Fulfilment's shipment response has 50 fields. Lavitt's Quay Coffee's shop uses three of them: shipment_id, status and tracking_number. Its consumer contract names only those three.

1. The warehouse wants to remove estimated_dispatch, a field from an old feature. No consumer contract mentions it, so the provider's verification still passes. It can remove the field safely.
2. The warehouse wants to rename tracking_number. Lavitt's Quay's contract mentions it, so verification fails, on the warehouse's own pipeline, before the deploy.
3. The warehouse team now knows exactly which consumer would break, and can talk to that one team instead of announcing a change to everyone.

That is the whole value of the approach in three steps. The rename from the earlier example would have been caught at step two.

TDD for APIs in Rails

Let me show how I approach API testing in Rails, test first, for an API that another application will consume. Request specs are the right tool: both the Rails and the RSpec teams discourage testing controllers directly and recommend request specs, which start with an HTTP request and end with the response.³

# spec/requests/api/v1/orders_spec.rb
require "rails_helper"

RSpec.describe "Orders API", type: :request do
  let(:user) { create(:user) }
  let(:headers) { { "Authorization" => "Bearer #{user.api_token}", "Accept" => "application/json" } }

  describe "GET /api/v1/orders/:id" do
    let(:order) { create(:order, user: user, total_cent: 4999) }

    it "returns the order details" do
      get "/api/v1/orders/#{order.id}", headers: headers

      expect(response).to have_http_status(:ok)
      expect(response.media_type).to eq("application/json")
      json = response.parsed_body
      expect(json["id"]).to eq(order.id)
      expect(json["total_cent"]).to eq(4999)
      expect(json["status"]).to eq("pending")
    end

    it "includes the line items" do
      create(:line_item, order: order, quantity: 2)
      get "/api/v1/orders/#{order.id}", headers: headers

      expect(response.parsed_body["line_items"].length).to eq(1)
      expect(response.parsed_body["line_items"][0]["quantity"]).to eq(2)
    end

    it "rejects a request without a token" do
      get "/api/v1/orders/#{order.id}"
      expect(response).to have_http_status(:unauthorized)
    end

    it "answers 404 for someone else's order" do
      stranger = create(:user)
      get "/api/v1/orders/#{order.id}", headers: { "Authorization" => "Bearer #{stranger.api_token}" }
      # 404, not 403, so the response does not reveal that the order exists
      expect(response).to have_http_status(:not_found)
    end
  end

  describe "POST /api/v1/orders" do
    let(:product) { create(:product, price_cent: 1999) }
    let(:valid_params) do
      {
        order: {
          line_items: [{ product_id: product.id, quantity: 2 }],
          shipping_address: { line1: "1 Lavitt's Quay", city: "Cork", eircode: "T12 X2A4" }
        }
      }
    end

    it "creates the order and works out the total" do
      expect {
        post "/api/v1/orders", params: valid_params, headers: headers, as: :json
      }.to change(Order, :count).by(1)

      expect(response).to have_http_status(:created)
      expect(response.parsed_body["total_cent"]).to eq(3998)
    end

    it "refuses an order with no items" do
      post "/api/v1/orders", params: { order: { line_items: [] } }, headers: headers, as: :json

      expect(response).to have_http_status(422) # 422 Unprocessable Content
      expect(response.parsed_body["errors"]).to include("line_items")
    end
  end
end

The "someone else's order" test matters more than it looks. An API that returns an object just because the caller knows its id is the first risk on the OWASP API Security Top 10, Broken Object Level Authorization.⁴ The status for a validation failure is 422, which HTTP now names "Unprocessable Content".⁵ I write the number in the test, because Rack 3.1 renamed the symbol for it and deprecated the old one.⁶

Worked example: the total in cent

The test says two of a product at €19.99 come to 3998. Work it through: €19.99 is 1,999 cent, and 2 times 1,999 is 3,998 cent, which is €39.98. The API sends whole cent as an integer, never a floating point number of euro, for the same reason the TDD article holds money as BigDecimal: floating point cannot store most decimal fractions exactly. Writing the test first forced that decision before any code existed.

Testing API versions

When several applications depend on your API, versions matter. A test per version pins down what each one promises:

# spec/requests/api/versioning_spec.rb
require "rails_helper"

RSpec.describe "API versions", type: :request do
  let(:user) { create(:user) }
  let(:order) { create(:order, user: user) }
  let(:headers) { { "Authorization" => "Bearer #{user.api_token}" } }

  it "keeps snake case keys in v1" do
    get "/api/v1/orders/#{order.id}", headers: headers
    expect(response.parsed_body).to include("total_cent", "created_at")
  end

  it "uses camel case keys and adds the delivery estimate in v2" do
    get "/api/v2/orders/#{order.id}", headers: headers
    expect(response.parsed_body).to include("totalCent", "createdAt", "estimatedDeliveryDate")
  end
end

The v1 test is the promise to every consumer still on v1. If someone "tidies up" v1 to camel case, it fails.

OpenAPI documentation from the tests, with rswag

OpenAPI is the standard format for describing an HTTP API. It grew out of Swagger 2.0, which was donated to the OpenAPI Initiative in 2015, and the current version is 3.2.1.⁷ In Rails, the rswag gem lets you write request specs in a form that also produces an OpenAPI file:

# spec/requests/api/v1/orders_openapi_spec.rb
require "swagger_helper"

RSpec.describe "Orders API", type: :request do
  path "/api/v1/orders/{id}" do
    get "Retrieves an order" do
      tags "Orders"
      produces "application/json"
      parameter name: :id, in: :path, type: :integer
      parameter name: "Authorization", in: :header, type: :string, required: true

      response "200", "order found" do
        schema type: :object,
               properties: {
                 id: { type: :integer },
                 total_cent: { type: :integer },
                 status: { type: :string, enum: %w[pending confirmed shipped delivered] },
                 created_at: { type: :string, format: "date-time" }
               },
               required: %w[id total_cent status]

        let(:user) { create(:user) }
        let(:id) { create(:order, user: user).id }
        let(:Authorization) { "Bearer #{user.api_token}" }

        run_test!
      end

      response "404", "order not found" do
        let(:user) { create(:user) }
        let(:id) { 0 }
        let(:Authorization) { "Bearer #{user.api_token}" }

        run_test!
      end
    end
  end
end

run_test! builds the request, sends it, and checks the actual status code and, where there is a schema, the response body against it.⁸ Running rake rswag:specs:swaggerize then writes the OpenAPI file from the same specs.⁸ Because the specs must pass, the document describes what the API actually does. One setting is worth turning on: by default extra fields in a response are allowed, and openapi_no_additional_properties makes the check strict.⁸

The costs: the specs get longer with all the schema in them, OpenAPI's schema language takes some learning, and the suite slows down a little if you write every test this way. I use rswag for the endpoints other teams consume, and plain request specs for the rest.

TDD for APIs in Django

Django has its own mature ecosystem, built around Django REST Framework (DRF). Its APITestCase gives you an APIClient with a force_authenticate method for tests.⁹ The same order API, from the warehouse's side of the world:

# tests/test_orders_api.py
from django.urls import reverse
from rest_framework import status
from rest_framework.test import APITestCase

from orders.models import LineItem, Order
from products.models import Product
from users.models import User


class OrderApiTests(APITestCase):
    def setUp(self):
        self.user = User.objects.create_user(email="priya@example.com", password="a long test password")
        self.client.force_authenticate(user=self.user)
        self.product = Product.objects.create(name="House blend 1kg", price_cent=1999)

    def test_get_order_returns_details(self):
        order = Order.objects.create(user=self.user, total_cent=4999, status="pending")
        response = self.client.get(reverse("order-detail", kwargs={"pk": order.id}))

        self.assertEqual(response.status_code, status.HTTP_200_OK)
        self.assertEqual(response.data["total_cent"], 4999)
        self.assertEqual(response.data["status"], "pending")

    def test_get_order_includes_line_items(self):
        order = Order.objects.create(user=self.user, total_cent=3998)
        LineItem.objects.create(order=order, product=self.product, quantity=2, price_cent=1999)
        response = self.client.get(reverse("order-detail", kwargs={"pk": order.id}))

        self.assertEqual(response.data["line_items"][0]["quantity"], 2)

    def test_get_order_without_auth_is_rejected(self):
        self.client.force_authenticate(user=None)
        order = Order.objects.create(user=self.user, total_cent=100)
        response = self.client.get(reverse("order-detail", kwargs={"pk": order.id}))

        # 401 with token authentication, 403 with session authentication
        self.assertEqual(response.status_code, status.HTTP_401_UNAUTHORIZED)

    def test_someone_elses_order_is_not_found(self):
        stranger = User.objects.create_user(email="sam@example.com", password="another long password")
        order = Order.objects.create(user=stranger, total_cent=100)
        response = self.client.get(reverse("order-detail", kwargs={"pk": order.id}))

        self.assertEqual(response.status_code, status.HTTP_404_NOT_FOUND)

    def test_create_order_works_out_the_total(self):
        data = {
            "line_items": [{"product_id": self.product.id, "quantity": 2}],
            "shipping_address": {"line1": "1 Henry Street", "city": "Limerick", "eircode": "V94 X2A4"},
        }
        response = self.client.post(reverse("order-list"), data, format="json")

        self.assertEqual(response.status_code, status.HTTP_201_CREATED)
        self.assertEqual(response.data["total_cent"], 3998)

    def test_create_order_without_items_fails(self):
        response = self.client.post(reverse("order-list"), {"line_items": []}, format="json")

        self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
        self.assertIn("line_items", response.data)

One detail differs from Rails, and it trips people up. Which status an unauthenticated request gets depends on the authentication scheme: DRF returns 401 when the first authentication class can send a WWW-Authenticate header (token authentication can), and 403 when it cannot (session authentication).¹⁰ The test above assumes token authentication. Decide which you use, and pin it in a test, because consumers handle the two differently. Note also that DRF answers a validation failure with 400, where the Rails API answered 422. Neither is wrong, but a consumer talking to both needs to know, which is exactly the kind of fact a contract records.

OpenAPI documentation with drf-spectacular

The Django counterpart to rswag is drf-spectacular. It generates an OpenAPI 3.0.3, 3.1 or 3.2 schema from your DRF views and serializers.¹¹

# orders/views.py
from drf_spectacular.utils import OpenApiExample, extend_schema
from rest_framework import status, viewsets
from rest_framework.response import Response

from .models import Order
from .serializers import OrderCreateSerializer, OrderSerializer


class OrderViewSet(viewsets.ModelViewSet):
    serializer_class = OrderSerializer

    def get_queryset(self):
        # Only the caller's own orders, which is what makes the 404 test pass
        return Order.objects.filter(user=self.request.user)

    @extend_schema(
        summary="Retrieve an order",
        responses={200: OrderSerializer},
        examples=[
            OpenApiExample(
                "An order",
                value={"id": 123, "total_cent": 4999, "status": "pending"},
                response_only=True,
            )
        ],
    )
    def retrieve(self, request, *args, **kwargs):
        return super().retrieve(request, *args, **kwargs)

    @extend_schema(summary="Create an order", request=OrderCreateSerializer, responses={201: OrderSerializer})
    def create(self, request, *args, **kwargs):
        serializer = OrderCreateSerializer(data=request.data)
        serializer.is_valid(raise_exception=True)
        order = serializer.save(user=request.user)
        return Response(OrderSerializer(order).data, status=status.HTTP_201_CREATED)

The difference from rswag is where the truth comes from. drf-spectacular describes the code (the views and serializers), so the schema follows the code automatically. rswag describes what the tests observed. drf-spectacular's schema can therefore claim a shape the view does not actually return, if a view bypasses its serializer, and the tests are what catch that.

Contract testing with Pact

Now the most useful approach for APIs between separate applications. Pact describes itself as a code first tool for testing HTTP and message integrations with contract tests, and it is consumer driven: the contract is produced while the consumer's own tests run.¹²

What Pact does, step by step

1. The consumer writes tests against a Pact mock server that describe the requests it will make and the responses it expects.
2. Running those tests writes a contract, a JSON file called a pact.
3. The pact is shared with the provider, usually through a Pact Broker.
4. The provider replays every request in the pact against its real application and checks that the responses match. Before each request it sets up the data the consumer assumed, the provider state (for example "a product with SKU HOUSE1KG exists").¹³
5. If every interaction passes, the two sides agree, without either one starting the other.

Read it from top to bottom. The consumer half ends when the pact is published. The provider half starts from that file. The last box is the safety net: the broker keeps a matrix of which consumer and provider versions have been verified against each other, and its can-i-deploy tool answers whether a given version is safe to release.¹⁴

The consumer side, in Rails

Pact for Ruby is now on version 2, built on the shared Pact core written in Rust, and supports the latest Pact specification. The older pure Ruby DSL (Pact.service_consumer, Pact.like) belongs to version 1.¹⁵ A consumer test for the shop's warehouse client in the version 2 style:

# spec/pact/providers/raheen_fulfilment/shipments_spec.rb
require "pact/rspec"

RSpec.describe "Shipments client", :pact do
  has_http_pact_between "lavitts-quay-shop", "raheen-fulfilment", opts: { mock_port: 3001 }

  let(:client) { WarehouseClient.new(base_url: "http://localhost:3001", token: "test-token") }
  let(:interaction) { new_interaction }

  context "when creating a shipment" do
    let(:interaction) do
      super()
        .given("a product exists", sku: "HOUSE1KG")
        .upon_receiving("a request to create a shipment")
        .with_request(
          method: :post,
          path: "/api/v1/shipments",
          headers: { "Content-Type" => "application/json", "Authorization" => match_any_string("Bearer test-token") },
          body: {
            order_id: match_any_integer(4812),
            items: match_each({ sku: match_any_string("HOUSE1KG"), quantity: match_any_integer(2) }),
            shipping_address: { eircode: match_any_string("T12 X2A4"), country: "IE" }
          }
        )
        .will_respond_with(
          status: 201,
          body: {
            shipment_id: match_any_string("SHP4812"),
            status: "pending",
            tracking_number: match_any_string("TRK000123")
          }
        )
    end

    it "returns the shipment with its tracking number" do
      interaction.execute do
        result = client.create_shipment(
          order_id: 4812,
          items: [{ sku: "HOUSE1KG", quantity: 2 }],
          shipping_address: { eircode: "T12 X2A4", country: "IE" }
        )
        expect(result.tracking_number).to eq("TRK000123")
      end
    end
  end

  context "when the shipment does not exist" do
    let(:interaction) do
      super()
        .given("no shipment exists", shipment_id: "MISSING")
        .upon_receiving("a request for a missing shipment")
        .with_request(method: :get, path: "/api/v1/shipments/MISSING")
        .will_respond_with(status: 404, body: { error: match_any_string("Shipment not found") })
    end

    it "raises a not found error" do
      interaction.execute do
        expect { client.shipment("MISSING") }.to raise_error(WarehouseClient::NotFound)
      end
    end
  end
end

The matchers matter. match_any_string("TRK000123") says "any string, for example this one", so the provider is free to return any tracking number but must return a string under that exact key. status: "pending" with no matcher means exactly that value. The contract records only the three response fields the shop uses, which is the "three out of fifty" example in code.

The provider side, in Django

The warehouse verifies the pact against its real Django application. In current versions of Pact for Python, the Verifier is built step by step: where the provider runs, where the pacts come from, how provider states are set up, and where results are published.¹⁶

# tests/test_pact_provider.py
import os

import pytest
from pact import Verifier

from orders.models import Product, Shipment


def product_exists(action, parameters):
    if action == "setup":
        Product.objects.get_or_create(sku=parameters["sku"], defaults={"name": "House blend 1kg", "stock": 100})


def no_shipment_exists(action, parameters):
    if action == "setup":
        Shipment.objects.filter(shipment_id=parameters["shipment_id"]).delete()


@pytest.mark.django_db(transaction=True)
def test_provider_honours_consumer_pacts(live_server):
    verifier = (
        Verifier("raheen-fulfilment")
        .add_transport(url=live_server.url)
        .broker_source("https://pact-broker.raheenfulfilment.example", token=os.environ["PACT_BROKER_TOKEN"])
        .state_handler({"a product exists": product_exists, "no shipment exists": no_shipment_exists}, teardown=True)
    )
    if "CI" in os.environ:
        verifier.set_publish_options(version=os.environ["GIT_COMMIT"], branch=os.environ["GIT_BRANCH"])
    verifier.verify()

The state handlers run in the same process as the provider (here live_server runs Django in a thread of the test process), which the Pact documentation requires when handlers touch the database directly.¹⁶ The provider states are parameterised (sku, shipment_id) rather than baked into the name, which the Pact documentation recommends.¹⁶

Worked example: the rename, caught

Replay the earlier story with Pact in place. The warehouse renames tracking_number to tracking_code and pushes the change.

1. The warehouse's pipeline runs test_provider_honours_consumer_pacts. The verifier fetches Lavitt's Quay's pact from the broker.
2. It sets up "a product exists" with SKU HOUSE1KG and replays the POST request against the real app.
3. The response contains tracking_code and no tracking_number. The pact says tracking_number must be a string. Verification fails.
4. The warehouse build goes red on the branch, before anything is deployed, with a message naming the consumer and the missing key.
5. Had someone forced the deploy anyway, can-i-deploy would have answered no, because that provider version has no successful verification against the consumer version in production.

Three days of broken tracking emails become one failed build on the warehouse's side.

Pact's strengths: it works across languages, it catches breaking changes before deployment, and consumers only pin what they use. Its costs: real learning, a broker to run (open source, or a hosted one), provider states that need care as they multiply, and no coverage of business logic, only of the interface.

What can go wrong

Mock drift

You stub the other API. It works. Months pass. The API changes, your stubs do not, your tests stay green and production breaks. The fix is contract testing, or at the very least recording real responses from the provider's sandbox now and then and comparing them with your stubs.

False confidence from staging

The integration tests pass on staging. You deploy, and production fails. Staging has different data, different load and different configuration. An integration test proves the code can work, not that it will. Integration tests are necessary but not sufficient: combine them with contract tests, thorough request specs and monitoring in production.

Testing only the happy path

What happens when the other API returns a 500, times out, or sends something that is not JSON? With WebMock you can make a stub do each of those, including to_timeout:¹⁷

# spec/services/warehouse_client_spec.rb
require "rails_helper"

RSpec.describe WarehouseClient do
  let(:url) { "https://api.raheenfulfilment.example/api/v1/shipments" }
  let(:client) { WarehouseClient.new(base_url: "https://api.raheenfulfilment.example", token: "t") }
  let(:order) { create(:order) }

  it "reports the provider being down" do
    stub_request(:post, url).to_return(status: 500, body: "Internal Server Error")
    result = client.create_shipment_for(order)
    expect(result).to be_failure
    expect(result.error).to eq("Warehouse API unavailable")
  end

  it "reports a timeout" do
    stub_request(:post, url).to_timeout
    expect(client.create_shipment_for(order).error).to eq("Warehouse API timeout")
  end

  it "reports a response that is not JSON" do
    stub_request(:post, url).to_return(status: 201, body: "not json")
    expect(client.create_shipment_for(order).error).to eq("Invalid response from Warehouse API")
  end
end

How the shop should behave when the warehouse is slow or down, with retries in a background job and a short timeout, is covered in background workers in Rails.

Testing against production APIs

I have seen teams run their test suites against real production APIs. You might create real orders, charges or shipments. Your tests depend on someone else's uptime. You can hit rate limits. And you put live credentials into your CI system. Use the provider's sandbox, contracts or recorded responses. Never production.

Switching off authentication in tests

# Dangerous: the test no longer touches authentication at all
before do
  allow_any_instance_of(ApiController).to receive(:authenticate!).and_return(true)
end

That test passes, and tells you nothing about whether authentication works. Then you deploy, and real clients cannot log in. Create real test tokens and send them, as the request specs above do.

API tests as documentation

Here is the reward for all of this: your API tests become the most reliable documentation you have. Hand written API documentation starts to decay the day it is published. Someone changes an endpoint and forgets the docs. Field names drift. Required parameters become optional. Before long the documentation actively misleads.

Documentation generated from tests cannot drift that way. If the test says the endpoint returns user_id and it returns userId, the build fails. The same idea is behind Cucumber feature files as living documentation, applied to an API.

In Rails with rswag:

# Write the OpenAPI file from the specs
bundle exec rake rswag:specs:swaggerize

In Django with drf-spectacular:

# Write the OpenAPI schema
python manage.py spectacular --file schema.yml

# urls.py: serve the schema and an interactive Swagger UI
from django.urls import path
from drf_spectacular.views import SpectacularAPIView, SpectacularSwaggerView

urlpatterns = [
    path("api/schema/", SpectacularAPIView.as_view(), name="schema"),
    path("api/docs/", SpectacularSwaggerView.as_view(url_name="schema"), name="swagger-ui"),
]

A pact is documentation of a different kind: not what the API offers, but what each consumer actually relies on. Reading the warehouse's broker tells you that the Cork shop depends on shipment_id, status and tracking_number, and on nothing else. No hand written page ever told anyone that.

Putting it together: a testing strategy in five layers

Based on years of doing this, here is my recommended approach for APIs between applications:

1. Unit tests. Serializers, validators and business logic in isolation. Fast, reliable, on every commit.
2. Request specs. Real HTTP requests to your own API with outside dependencies stubbed. Authentication, authorisation, validation and response formats. On every commit.
3. Contract tests. Pact between the applications you control, on every commit, with can-i-deploy blocking a release that breaks a contract.
4. A few integration tests. End to end on staging, before major releases, for the critical paths only.
5. Monitoring in production. Error rates, response times and failed calls, with alerts. This catches what tests cannot.

Worked example: one change through the layers

Lavitt's Quay Coffee adds gift messages to orders, which means a new optional gift_message field sent to the warehouse.

1. A unit test checks that the message is trimmed and limited to 200 characters.
2. A request spec checks that the shop's own API accepts and returns the field.
3. The consumer pact gains gift_message in the shipment request. The warehouse's verification fails, because its API rejects unknown fields. That is the conversation the two teams needed to have, and it happens before either deploys.
4. Once the warehouse supports the field, one staging test places a real gift order through both systems.
5. In production, an alert watches the rate of failed shipment calls in the first days after release.

Each layer caught, or was ready to catch, a different kind of problem. Unit tests catch logic errors, contract tests catch interface mismatches, integration tests catch environment problems, and monitoring catches what happens in the real world.

The tests are the agreement

Testing APIs between separate applications is different from testing inside one. Separate releases, separate teams and an unreliable network demand their own strategies. Contract testing, and consumer driven contracts with Pact in particular, give the best balance of speed and safety, backed by thorough request specs and a small number of integration tests. And as a bonus, the tests become documentation that stays accurate because it runs on every build.

The effort pays for itself every time a breaking change fails a build instead of a customer's order, every time a new developer learns an API by reading its tests, and every time you refactor without holding your breath.

Footnotes

1. Ian Robinson, Consumer driven contracts: a service evolution pattern, martinfowler.com, 12 June 2006, https://martinfowler.com/articles/consumerDrivenContracts.html

2. Pact, What is Pact good for, Pact documentation, https://docs.pact.io/getting_started/what_is_pact_good_for

3. RSpec, rspec-rails README (request specs), GitHub, https://github.com/rspec/rspec-rails

4. OWASP, API1:2023 Broken Object Level Authorization, OWASP API Security Top 10, 2023, https://owasp.org/API-Security/editions/2023/en/0xa1-broken-object-level-authorization/

5. R. Fielding, M. Nottingham and J. Reschke, RFC 9110 HTTP Semantics, section 15.5.21, IETF, 2022, https://www.rfc-editor.org/rfc/rfc9110.html

6. Rack, Changelog, version 3.1.0 (status descriptions for 422 updated, obsolete status symbols deprecated), GitHub, 11 June 2024, https://github.com/rack/rack/blob/main/CHANGELOG.md

7. OpenAPI Initiative, OpenAPI Specification, latest version and revision history, https://spec.openapis.org/oas/latest.html

8. rswag, README (run_test!, swaggerize, openapi_no_additional_properties), GitHub, https://github.com/rswag/rswag

9. Django REST framework, Testing, https://www.django-rest-framework.org/api-guide/testing/

10. Django REST framework, Authentication, unauthorized and forbidden responses, https://www.django-rest-framework.org/api-guide/authentication/

11. T. Franzel, README of the drf spectacular project, GitHub, https://github.com/tfranzel/drf-spectacular

12. Pact, Introduction, Pact documentation, https://docs.pact.io/

13. Pact, Provider states, Pact documentation, https://docs.pact.io/getting_started/provider_states

14. Pact, Can I deploy, Pact Broker documentation, https://docs.pact.io/pact_broker/can_i_deploy

15. Pact Foundation, pact-ruby README, GitHub, https://github.com/pact-foundation/pact-ruby

16. Pact Foundation, Pact Python, Provider testing, GitHub, https://github.com/pact-foundation/pact-python/blob/main/docs/provider.md

17. WebMock, README (to_timeout), GitHub, https://github.com/bblimke/webmock