Back to all articles
APIs as a ProductPart 08
ObservabilityAPI productsPartner integrationsProduct metrics

APIs as a Product, Part 8: Partner Health, the Numbers I Watched Every Week

Janahan Thavendran3 min read

TL;DR

  • A healthy platform-wide average can hide one partner failing on a large share of its calls.
  • Review every number per partner: errors split into ours and theirs, latency both ways, sync lag, version lag, webhook delivery and adoption.
  • Falling traffic and partners stuck in sandbox are health signals too; treat a drifting partner as a customer to call.
In this article

APIs as a Product, Part 8 of 10 · No number without a partner's name next to it.

Picture a dashboard that says the API is healthy. The overall error rate is well inside what was agreed. Latency is normal. Nobody has been paged.

Then a merchant calls to ask why orders from one partner have dried up. Split the same numbers by partner and the picture changes: that partner is failing on a large share of its calls. Small enough that the overall average barely moves, big enough that, for the merchants who rely on that channel, the API is broken.

Averages are comfortable. They are also where your worst partner hides.

Every number gets a partner's name

That is why our weekly review started with a view per partner, not platform-wide numbers. We used Datadog for the monitoring, but the tool mattered less than the rule: no number without a partner's name next to it.

Number
Error rate, split into our errors and theirs
What it caught
Whether a problem was ours to fix or a partner's integration to help with
Number
Latency, both directions
What it caught
Slow responses from us, and partners slow to confirm orders, which led to timeouts and retries
Number
Sync lag
What it caught
Time from a change in the core to confirmation that each consumer had received it
Number
Version lag
What it caught
Traffic each consumer still sent to old versions
Number
Webhook delivery
What it caught
Partners whose endpoints were down or rejecting our events
Number
Adoption
What it caught
Partners stuck in sandbox, live partners whose traffic was falling, features nobody used

The split between our errors and theirs mattered most. A spike in server errors was our problem. A spike in bad requests from one partner usually meant they had shipped a new app version. Both needed action, but from different people.

Sync lag was the number I watched most closely, because nobody complained about it until it was too late. A menu change that took hours to reach a partner meant customers could order something the kitchen had stopped making. That never showed up as an error. It showed up as a refund.

Adoption is a health number too

The quietest warning sign was a partner whose traffic slowly declined. No errors, no complaints, just fewer calls each week. Sometimes that was seasonal. Sometimes it meant the partner had moved their users to another provider and hadn't told us yet. We only caught it because we were looking at each partner's trend, not the total.

The same went for partners who got a sandbox key and never went live. Each one was a sales conversation and an integration effort that had stalled somewhere. Looking at where they stopped told us more about our docs and onboarding than any survey.

None of these numbers were hard to collect. The hard part was looking at them every week, and treating a partner whose numbers were drifting as a customer to call, not a line on a chart.

A healthy platform with an unhealthy partner is not a healthy platform. It is a platform that hasn't noticed yet.


Related reading: The Shared Platform, Part 5: Measuring a Platform Nobody Sees and Platform at Scale, Part 6: You Can't Fix What You Can't See.