Contract testing vs end-to-end testing
Choose contract testing to catch breaking changes in each application programming interface (API) and webhook schema on every pull request, without a partner bank or processor sandbox. Choose end-to-end testing when a test must prove that money moves correctly across the ledger, the card processor, and the bank settlement file, because no single contract covers that chain.
Contract testing and end-to-end testing compared
The table compares both test types on a fintech integration where a product calls a partner bank, a Banking-as-a-Service (BaaS) platform, or a card processor, and receives webhooks and bank files back.
| Factor | Contract testing | End-to-end testing |
|---|---|---|
| What the test checks | Contract testing checks that one side of an integration sends and accepts the requests, responses, and event payloads written in an agreed contract. | End-to-end testing checks that a complete business flow produces the correct result in every real system the flow passes through. |
| Systems the test needs | Contract testing runs one service at a time, with the other side replaced by a stub built from the contract. | End-to-end testing needs every system in the flow deployed together, including the partner bank or processor test environment. |
| Source of expected results | Contract testing reads an OpenAPI document, a JSON Schema for webhook payloads, or a consumer-driven contract file. | End-to-end testing reads expected balances, statuses, and file lines from a test case written for one business flow. |
| Defects the test catches | Contract testing catches a renamed field, a changed type, a new enum value, or an undocumented status code before release. | End-to-end testing catches amounts, statuses, and references that disagree between the ledger, the processor record, and the bank file. |
| Point in the pipeline | Contract testing runs on each pull request, with no call to a partner bank or processor. | End-to-end testing runs after deployment to a shared environment and waits for asynchronous events such as settlement files and returns. |
| Dependence on partner sandboxes | Contract testing keeps running when a partner bank or processor sandbox is down or rate-limited. | End-to-end testing depends on a partner bank or processor sandbox, and a sandbox can differ from production in outcomes, timing, and files. |
| Failure diagnosis | A failed contract test names the operation, the field, and the side of the integration that broke the contract. | A failed end-to-end test shows a wrong final state, and finding the defect needs a trace across every system in the flow. |
| Evidence about money movement | Contract testing cannot show that a payment posted once to the ledger and settled for the correct amount. | End-to-end testing shows that one payment produced one ledger posting, one processor record, and one matching bank file line. |
| Maintenance trigger | A contract test changes when the contract changes, and a specification diff lists each breaking change. | An end-to-end test changes when any system in the flow changes its data, timing, or test environment. |
When to choose contract testing
- Partner bank and BaaS APIs that your product consumes. The Pact documentation lists "Testing APIs where the consumers cannot be individually identified (eg. public APIs)" and "Situations where you cannot control the data being used to generate the provider's responses" among the cases Pact is not good for (What is Pact good for?, checked on 2026-09-22). When a partner bank does not verify your contract in its own pipeline, the contract for its side is its published specification plus responses recorded from its sandbox. The API contract test suite in BaaS and partner bank integration testing is built that way and fails your build when the partner's documented response no longer matches what your code parses.
- Your own platform API with known consumers. Pact is "a code-first consumer-driven contract testing tool", and its contract "is generated during the execution of the automated consumer tests" (Pact docs introduction, checked on 2026-09-22). Consumer-driven contracts fit internal services and software development kits (SDKs) your teams own, where each consumer team can publish what it depends on. For a public API with unknown integrators, the suite in API and webhook testing is generated from the OpenAPI document instead.
- Webhook payload schemas. The OpenAPI Specification 3.2.1 (published 10 September 2026, checked on 2026-09-22) defines the
webhooksfield as "The incoming webhooks that MAY be received as part of this API and that the API consumer MAY choose to implement." A contract test validates each event type against that schema, so a card transaction event whoseamountfield changes from an integer to a string fails before your handler parses it in production. Receiver behaviour under retries and duplicates is a separate test, covered in the webhook reliability guide. - Partner API version upgrades. A diff of the old and new partner specifications lists each removed field, new required field, and narrowed enum before any end-to-end run, which shortens the upgrade to the operations that changed.
When to choose end-to-end testing
- Money movement across the ledger, the processor, and the bank file. Each hop can pass its contract while the flow loses money. A processor sends
amount: 1050in minor units, the ledger schema accepts an integer, both contracts pass, and the ledger posts 1,050.00 instead of 10.50. Only a test that follows one payment from authorisation to the ledger posting and the settlement file line compares the three amounts. - Outcomes that arrive later in a file. An ACH (Automated Clearing House) return reaches the product in a later bank file, after the product may have released the funds. The test injects the return file and asserts the reversal posting, as the Nacha return code guide shows per return code. Bank statements in ISO 20022 camt.053 format need the same treatment; the camt.053 reconciliation guide matches statement entries to ledger postings.
- Retries that cross system boundaries. A timeout after processor approval followed by a client retry returns two valid responses, and each response passes its contract. The defect is a second capture or a second ledger posting, which only a count across systems finds; the payment idempotency guide gives the request sequences and the SQL for that count.
- Sandboxes that differ from production. A processor or card scheme sandbox that returns scripted outcomes for fixed test amounts, settles instantly, or produces no settlement file hides timing and file defects. Our practice is to run end-to-end tests against the sandbox for the online path and against bank files generated from the bank's file specification for the settlement path, so each part of the flow runs against the closest available source.
Cost and timeline factors
Contract testing cost follows the interfaces:
- Specification quality. A partner specification without error bodies or enum lists needs recorded sandbox responses to fill the gaps, which adds work per operation.
- Number of operations and event types. Each operation and each webhook event type adds schema checks and stubs.
- Consumer-driven setup. Pact lists as a fit "a small enough number of consumers for a given provider that the provider team can manage an individual relationship with each consumer team", so each consumer adds a contract to publish, verify, and keep current.
- Version cadence. Each partner specification release needs a diff and an update of the stubs in the same cycle.
End-to-end testing cost follows the money flows:
- Environment access. Partner sandbox credentials and test programme setup follow the partner's onboarding process, and the partner sets that lead time.
- Test data across systems. One synthetic customer needs matching records in the ledger, the processor, and the bank file. Recorded partner traffic can contain account data, so it is masked before reuse.
- Asynchronous waits. Settlement cycles and return windows set how long a flow takes to complete, and simulated file drops shorten the run.
- Scope. Cost grows with each combination of money flow and payment rail, such as card capture, ACH debit, and wire.
Questions
What is consumer-driven contract testing?
Consumer-driven contract testing is contract testing in which each consumer of an application programming interface (API) records the requests it sends and the responses it depends on in a contract file, and the provider verifies that file against its own build before release. The Pact documentation states that the contract is generated during the execution of the automated consumer tests.
What is end-to-end testing?
End-to-end testing runs a complete business flow through every real system it touches, such as a card payment from authorisation to the ledger posting and the line in the settlement file. An end-to-end payment test passes only when amounts, statuses, and references agree in each system, including states that arrive later, such as a returned debit.
Scope your integration tests
Send your partner API specifications, webhook event list, and the money flows you run, and we return a plan that marks which checks run as contract tests and which run end to end.
Scope your testing