Docs/Fallback

Fallback

诺玛AI's fallback mechanism automatically switches to a backup model when the primary model is unavailable, keeping your service uninterrupted.

How it works#

The request is sent to the primary model
If the primary model returns an error (5xx, timeout, rate limit, etc.)
The models in the fallback list are tried in order automatically
The first successful response is returned

Per-request fallback#

Configure fallback for a single request via the provider.fallback parameter:

Code

from openai import OpenAI

client = OpenAI(
    base_url="https://as.apinoma.com/v1",
    api_key="<YOUR APINOMA_API_KEY>"
)

response = client.chat.completions.create(
    model="openai/gpt-4o",  # primary model
    messages=[{"role": "user", "content": "Hello"}],
    extra_body={
        "provider": {
            "fallback": [
                "anthropic/claude-sonnet-4.6",  # First backup
                "google/gemini-pro-compatible-flash-lite-preview"              # Second backup
            ]
        }
    }
)

# Inspect the model actually used
print(response.model)

Trigger conditions#

Fallback is triggered in the following cases:

Condition

Description

HTTP 5xx

Server error

Request timeout

Model response timed out

429 rate limit

Upstream model hit its rate limit

Model unavailable

Provider maintenance or offline

Fallback is NOT triggered in these cases:

Condition

Description

HTTP 4xx (except 429)

Client error — the request needs to be fixed

Content filtering

Content the model refused to generate

Combine with routing#

The fallback mechanism can be combined with provider routing:

Code

response = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Hello"}],
    extra_body={
        "provider": {
            "routing": "latency",    # Latency-first routing
            "fallback": [             # Fallback list
                "anthropic/claude-sonnet-4.6",
                "google/gemini-pro-compatible-flash-lite-preview"
            ]
        }
    }
)

Recommended fallback setup#

For recommended models, see the model page.

Best practices#

Pick backup models with comparable capability — to keep output quality consistent after a fallback
Fall back across providers — avoid having models from the same provider go down at once
Set 2–3 backups — enough to handle most failure scenarios
Monitor fallback frequency — if fallbacks happen often, consider switching your primary model

Last updated on June 23, 2026