What API Rate Limits Teach You

The failure that matters

A hard failure is fine — you see it, you fix it. The dangerous case is a partial load: half the campaigns returned, the job marked successful, and a reporting table that is quietly wrong for a week.

Patterns that hold up

None of these are clever. All of them are the difference between trusted and untrusted data.

• Exponential backoff with jitter — synchronised retries are their own denial of service.
• Idempotent writes keyed on a natural business key, so a replay never duplicates.
• Atomic swaps: load into a staging table, promote only on a complete run.
• Row-count and freshness assertions before promotion, not after.
• A dead-letter record for every dropped request, with enough context to replay it.

The organisational half

Technical retries do not help if nobody is told. Every pipeline needs one owner and one alert channel, and the alert must say what is stale — not that a job exited non-zero.