Docker Compose: cascade routing in one line
Prompts the deterministic classifier scores as simple are routed to the cheaper sibling of the model you asked for, inside the same provider, with a shadow-parity breaker. Here is the change for Docker Compose, the switch that turns it on, and the arithmetic on gpt-6.1-sol.
Docker Compose reaches TokenZero with one environment or settings change and keeps your openai key. The cascade sibling of gpt-6.1-sol is gpt-6-luna at $0.00035 per reference call against $0.00700; if every call were simple that is $665 a month on 100,000 calls, and a 99.8% parity floor with shadow evaluation decides whether a pair stays enabled.
· Source: pricing table 2026-10-26; overhead from the published benchmark on /benchmarks
At a glance
- Framework
- Docker Compose (YAML)
- Capability
- cascade routing
- Change
- compose.yaml: environment / settings
- Request shape
- any official SDK via environment
- Turn it on
- X-TokenZero-Cascade: auto
- Workspace setting
- cascade: auto, shadowEval: true
- Reference model
- gpt-6.1-sol ($2.00 in / $10.00 out per 1M, table 2026-10-26)
- Cascade saving if every simple call moves
- $665/month to gpt-6-luna (computed)
- Gateway overhead (measured)
- 1.52 ms p50, 3.42 ms p95
- Parity floor
- 99.8%
- Free tier
- $500/month routed spend
Reference workload on gpt-6.1-sol: 2,000 input (1,500 static) / 300 output tokens, 100,000 calls per month, table 2026-10-26
| Scenario | Per call | Per month | Delta |
|---|---|---|---|
| List price, no gateway | $0.00700 | $700 | — |
| Static prefix cached (prompt compression) | $0.00415 | $415 | $285 (41%) |
| 20% exact-cache repeats | $0.00560 | $560 | $140 (20%) |
| All calls cascaded to gpt-6-luna | $0.00035 | $35 | $665 (95%) |
| Budget guardrail at $500/month | — | capped at $500.00 | 402 after 71,428 calls |
The change
1. Point Docker Compose at the gateway (compose.yaml):
services:
app:
environment:
OPENAI_API_KEY: ${OPENAI_API_KEY}
+ OPENAI_BASE_URL: https://tokenzero.sh/v1
+ ANTHROPIC_BASE_URL: https://tokenzero.shEvery official SDK reads these variables, so the container image does not change.
2. Turn on cascade routing. Per request: X-TokenZero-Cascade: auto (default advise: the recommendation is returned in X-TokenZero-Cascade-Recommendation and nothing moves). Per workspace (applies to every call without touching code): PATCH /v1/workspaces/settings {cascade: auto, shadowEval: true} or the dashboard toggle.
3. Prove it. npx -y @tokenzero/cli doctor checks the gateway, the signer, the pricing table and your key; then watch routed share and parity per pair on the scorecard.
Questions people ask
How do I point Docker Compose at TokenZero?
Set the base URL where Docker Compose configures its provider (compose.yaml) to https://tokenzero.sh/v1 and keep your openai key; nothing else changes.
What does cascade routing save on gpt-6.1-sol?
The cascade sibling of gpt-6.1-sol is gpt-6-luna at $0.00035 per reference call against $0.00700; if every call were simple that is $665 a month on 100,000 calls, and a 99.8% parity floor with shadow evaluation decides whether a pair stays enabled.
Does cascade routing change the model's output?
Only when a prompt is scored simple and the pair is above the 99.8% parity floor; shadow evaluation re-runs a sample on the original model, and a pair that drops below the floor is auto-disabled. Everything else goes to the model you asked for.
What does TokenZero cost?
Free up to $500 per month of routed spend; then 15% of verified savings with a 3x-ROI-or-$0 rule, capped plans available. Gateway overhead measured at 1.52 ms p50.