> ## Documentation Index
> Fetch the complete documentation index at: https://aidocs.ethanbragdon.icu/llms.txt
> Use this file to discover all available pages before exploring further.

# Provider Health Runbook

> Investigate Starfire AI model or search provider degradation using health, latency, errors, rate limits, routing, and incident scope.

# Provider health runbook

Use this runbook when model, search, embedding, or other upstream provider behavior appears degraded.

## 1. Confirm scope

Determine whether the issue affects:

* one model
* several models on one provider
* one route/region
* one workload such as streaming or tools
* every provider

A single-user failure is not enough to declare a provider outage.

## 2. Review provider metrics

Inspect available signals such as:

* request volume
* success rate
* latency
* timeout rate
* rate-limit responses
* provider server errors
* token/request throughput
* fallback frequency

## 3. Compare routes

If the model has multiple routes, determine whether the problem is tied to one route or to the model/provider generally.

## 4. Check capability-specific failures

A provider can work for basic text while failing on streaming, tool calls, vision, or large contexts. Segment diagnostics by workload before disabling the entire provider.

## 5. Test a known request

Use an authorized diagnostic/test request rather than a user's private prompt. Confirm expected response, latency, and capability behavior.

## 6. Decide mitigation

Possible mitigations include:

* remove an unhealthy route from normal traffic
* use a configured capability-compatible fallback
* temporarily reduce rollout
* disable one affected model
* wait for provider recovery while surfacing a clear user error

## 7. Create/update incident

When impact is broad or sustained, attach the provider/model components to an incident and record mitigation/recovery evidence.

## 8. Verify recovery

Do not restore broad traffic based on one successful request. Confirm error rate and latency return to acceptable behavior.

<Warning>
  A fallback that lacks the request's required capabilities can turn a provider outage into incorrect product behavior. Preserve the request contract first.
</Warning>