Fulmenflux

Blog / Reliability

Data contracts: stop letting upstream deploys break your dashboards

A data contract is a schema, an owner and a test at the boundary. Here is the smallest version that actually prevents incidents.

Cristian Curteanu 2 min read

The usual incident: an application team renames a column, nothing in their tests fails, and three hops downstream a revenue dashboard quietly reports zero. Support hears about it from a customer.

A data contract is an agreement between whoever produces a dataset and whoever depends on it. It does not need a platform to be useful. The smallest version has four parts.

The minimum viable contract

  1. A schema with types and nullability, stored in the producer’s repo.
  2. A named owner who gets paged when the contract is violated.
  3. Semantics: what a row means, its grain, and what “late” is.
  4. A check at the boundary that runs in the producer’s CI and again when data lands.
dataset: orders_events
owner: team-checkout
grain: one row per order state change
fields:
  - name: order_id
    type: string
    required: true
  - name: amount_cents
    type: integer
    required: true
freshness: 15m

Test where data crosses a team boundary

Tests deep inside a dbt project catch bugs late. Put the first check where the data enters your platform, and fail loudly there: reject or quarantine the batch, alert the producer, keep the bad rows out of the models downstream.

Lineage answers “who is affected?”

When a contract changes, you want the list of dependent models and dashboards in seconds, not a day of grepping. Column-level lineage from your orchestrator and dbt manifests is usually enough to start.

Start with the five tables that hurt

Don’t contract the whole warehouse. Pick the datasets behind your revenue reporting and customer-facing features, write the four parts above, and add the CI check. Expand when the first incident is prevented.

Tags

More from the blog