Provider routing

诺玛AI supports a multi-provider architecture: the same Models can be served through different provider nodes. With routing strategies, you control how requests are distributed.

Routing strategies#

Strategy

Description

Use case

`priority`

In priority order (default)

Stability first

`cost`

Lowest cost first

Batch processing, cost-sensitive

`latency`

Lowest latency first

Real-time chat, user interaction

`balanced`

Load balancing

High-concurrency scenarios

Usage#

Configure the routing strategy via the `provider.routing` extension parameter:

from openai import OpenAI
 
client = OpenAI(
    base_url="https://as.apinoma.com/v1",
    api_key="<YOUR APINOMA_API_KEY>"
)
 
response = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Hello"}],
    extra_body={
        "provider": {
            "routing": "cost"  # Lowest cost first
        }
    }
)

Strategy details#

`priority` — priority routing (default)

Distributes requests in 诺玛AI's preset provider priority order, favoring the most stable nodes first.

`cost` — cost first

Automatically selects the cheapest available provider node. Best for latency-insensitive scenarios such as batch processing and data labeling.

`latency` — latency first

Selects the provider node with the lowest response latency. Best for real-time chat scenarios that need fast responses.

`balanced` — load balancing

Spreads requests evenly across all available provider nodes. Best for high-concurrency scenarios, avoiding overload on any single node.

Best practices#

• Use `latency` for real-time chat — shorter wait times for users
• Use `cost` for batch jobs — lower overall cost
• Default to `priority` in production — ensures stability
• Combine with fallback — the routing strategy can be used together with the `fallback` parameter

You can also set a global default routing strategy in the 诺玛AI console, so you do not need to specify it on every request.

Last updated on June 23, 2026