How to Alert on Model Provider Outages Before Users Notice

When a model provider has an incident, the first sign is often a user saying the assistant is broken. Status pages help, but they tend to update after problems start and may not reflect the specific model or region you use. Your own monitoring can detect trouble sooner.

Step 1: Alert on real traffic first

Your production requests are the best signal. Alert when, over five minutes:

• The error rate for model calls rises above a threshold, such as 5 percent.
• Timeouts or overload responses, such as HTTP 529 from Anthropic's API or 503 responses elsewhere, increase sharply.
• Time to first token at the 95th percentile doubles compared with the same hour the previous day.

Break these down by provider, model and region, so the alert says where the problem is.

Step 2: Add synthetic probes

Real traffic is quiet at night and at weekends. Run a tiny scheduled probe every minute or two from a couple of regions:

start = time.monotonic()
response = client.messages.create(
    model=MODEL,
    max_tokens=5,
    messages=[{"role": "user", "content": "Reply with OK"}],
)
record("probe_latency_seconds", time.monotonic() - start, model=MODEL)
record("probe_success", 1 if response.content else 0, model=MODEL)

Probes this small cost almost nothing. Alert when several consecutive probes fail, not on a single failure.

Step 3: Subscribe to status updates

Subscribe to the provider's status page by email, RSS or webhook, and post updates into the channel your on-call team watches. When your alert and a status update arrive together, you know it isn't your code.

Step 4: Connect alerts to actions

Every alert should link to the runbook step it triggers: fail over to another region, switch to a backup model, turn on a degraded mode or show users a clear message.

Step 5: Tell users quickly

A banner saying the assistant is temporarily slow or unavailable prevents a flood of support tickets. Trigger it automatically from the alert where you can.

Things to watch

• Your own bugs. A bad deployment can look like a provider outage. Show the last deployment time on the alert.
• Rate limits. HTTP 429 responses usually mean you've hit your own limits, not that the provider is down. Alert on them separately.
• Probe credentials. Give probes their own key, so a revoked production key shows up as a production alert rather than a probe failure.

Set up the error-rate alert on real traffic today, then add probes for the models your most important features rely on.