Skip to content
FluxCDNFluxCDN
esc
On this page

Docs

Routing controls

A request names a model, not a vendor. The router turns that name into a ranked list of concrete endpoints and serves the first one that works. You can constrain that choice per request, but never widen it beyond what your organization policy allows.

Aliases and strategies

An alias picks both the model and the ranking strategy. Naming a model directly keeps the balanced strategy unless you override it.

AliasStrategyBehaviour
router/autobalancedTrades capability, price and endpoint redundancy. The default, and the right choice unless you have measured a reason to change it.
router/fastfastestLowest time to first token among healthy endpoints. Ignores price.
router/cheapcheapestLowest total cost that still meets the capability requirements of the request. Ignores latency.
router/reliablereliablePrefers the endpoint with the strongest recent health record, then the most peers to fall back to.

Hard filters

Filters run before ranking. An endpoint that fails any of them is never considered, no matter how cheap or fast it is.

Capability
Endpoint must support what the request needs — tools, JSON schema, vision, streaming.
Context
Prompt must fit the endpoint's context window, not just the model's nominal one.
Region
If a region is pinned, only endpoints in that region qualify. `global` does not count.
Retention
With zero-retention required, providers without a no-training guarantee are dropped.
Policy
Organization allow/deny lists for providers and models.
Health
Endpoints with an open circuit breaker are excluded until it half-opens.

Constraining a request

The routing object is a FluxCDN extension to the OpenAI request shape. Anything you set is intersected with your organization policy — the stricter value always wins.

POST /v1/chat/completions
{
  "model": "router/auto",
  "messages": [{"role": "user", "content": "…"}],
  "routing": {
    "strategy": "cheapest",
    "region": "eu",
    "require_zero_retention": true,
    "providers": ["anthropic", "google"],
    "max_cost_micros": 50000
  }
}

When nothing qualifies

If the filters exclude every endpoint, the request fails with upstream_unavailable. It is never quietly served by something outside your constraints — a region pin that silently fell back to another region would be worse than an error, because you would not know it happened.

Failover and retries

A failure before the first token is safe to retry on the next-ranked endpoint, and the router does so inside the same request. A failure after tokens have streamed is not retried — replaying a partially delivered response would corrupt it.

Every attempt is recorded. The response reports how many were made in routing.attempts and the x-fluxcdn-attempts header, and the full chain is visible on the request in Activity.