TokenZero vs LiteLLM Proxy
LiteLLM Proxy is a Python proxy that normalises many providers behind one OpenAI-shaped endpoint with keys, budgets and logging. The matrix below says what each product is and how it bills; the only numbers are TokenZero's own measurements and prices.
TokenZero and LiteLLM Proxy differ in kind: LiteLLM Proxy is self-hosted LLM proxy; TokenZero is a BYOK gateway whose engines reduce the provider bill and sign a receipt per call. Measured overhead is 1.52 ms p50; the cascade holds a 99.8% parity floor. On 100,000 gpt-6.1-sol calls, prefix caching saves $285 a month, $242 net of the 15% fee.
· Source: LiteLLM Proxy documentation at https://docs.litellm.ai/docs/proxy/enterprise read 2026-10-26; TokenZero overhead from /benchmarks; prices from table 2026-10-26
At a glance
- Compared with
- LiteLLM Proxy (self-hosted LLM proxy)
- TokenZero overhead
- 1.52 ms p50 / 3.42 ms p95 (measured, /benchmarks)
- BYOK
- TokenZero: always · LiteLLM Proxy: yes
- Receipts
- TokenZero: signed per call · LiteLLM Proxy: none
- Billing
- TokenZero: 15% of verified savings · LiteLLM Proxy: open source; enterprise licence
- Reference saving (gpt-6.1-sol, 100k calls)
- $285/month from prefix caching (computed)
- Net after 15% fee
- $242/month (computed)
- Parity floor
- 99.8%
- Free tier
- $500/month routed
- Vendor page read
- 2026-10-26
TokenZero vs LiteLLM Proxy: what each one is (vendor page read 2026-10-26)
| Dimension | TokenZero | LiteLLM Proxy |
|---|---|---|
| Category | BYOK savings gateway | self-hosted LLM proxy |
| Hosting | hosted | self-hosted (enterprise tier available) |
| Bring your own key | always; keys pass through and are never stored | yes |
| Open source | CLI, MCP server, audit core and GitHub Action Apache-2.0; gateway hosted | MIT |
| How it bills | free up to $500/month routed; then 15% of verified savings, 3x-ROI-or-$0, capped plans | open source; enterprise licence |
| Caching | prompt-cache formatting + exact and semantic response cache | response cache (Redis) |
| Routing | cascade to the cheaper sibling inside one provider, 99.8% parity floor, advisory by default | load balancing, fallbacks and routing strategies you configure |
| Savings receipts | Ed25519-signed per call, hash-chained, public key published | none |
| Gateway overhead | 1.52 ms p50 / 3.42 ms p95 (measured, /benchmarks) | 16.25 ms p50 measured on the same box (/benchmarks) |
| Data retention | metadata only; ZDR mode available | see vendor docs |
| Net saving, 100k gpt-6.1-sol calls/month (prefix caching only) | $242 after fee | — |
What you keep when you move from LiteLLM Proxy
OpenAI, Anthropic and Gemini request shapes; LiteLLM can keep running in front of TokenZero.
- Hosted, nothing to run: no database, no Redis, no config file.
- Measured on the same box and mock upstream: TokenZero adds 1.52 ms p50 against LiteLLM Proxy 16.25 ms p50 (method on /benchmarks).
- Smaller surface by construction: a Node 24 TypeScript gateway with zero runtime npm dependencies (node:http, node:crypto, node:sqlite), stateless BYOK pass-through with no stored provider keys, and ZDR mode that stores no prompt bodies; there is no central credential database to protect.
- Signed receipts and a hash-chained ledger instead of a spend log.
The change
OPENAI_BASE_URL=https://tokenzero.sh/v1
ANTHROPIC_BASE_URL=https://tokenzero.sh
# keep your provider keys exactly as they areThen npx -y @tokenzero/cli doctor to confirm the gateway, signer and pricing table, and see the switch guide.
Questions people ask
What is the difference between TokenZero and LiteLLM Proxy?
LiteLLM Proxy is a Python proxy that normalises many providers behind one OpenAI-shaped endpoint with keys, budgets and logging. TokenZero is a BYOK gateway: you keep OpenAI, Anthropic and Gemini request shapes; LiteLLM can keep running in front of TokenZero, and every call comes back with a signed receipt. Hosted, nothing to run: no database, no Redis, no config file.
What does switching from LiteLLM Proxy to TokenZero save?
It depends on your prompts; on the reference workload (100,000 calls of 2,000 input with a 1,500-token static prefix and 300 output on gpt-6.1-sol, table 2026-10-26) prefix caching alone cuts $700 to $415 a month, $285 saved and $242 net after the 15% fee; exact-cache hits and cascade routing add to that. The receipts show the real figure.
Can I keep LiteLLM Proxy and add TokenZero?
Usually yes: chain them by pointing LiteLLM Proxy's upstream at https://tokenzero.sh (or TokenZero's X-TokenZero-Upstream at LiteLLM Proxy); the receipt still prices the request at list and records what the gateway did.
How is the 15% fee computed?
On verified savings only: list-price baseline minus provider-reported actual cost, per signed receipt. A cycle whose savings are below 3x the fee is billed $0; Pro Cap and Scale Cap add monthly ceilings. The first $500 of routed spend each month is free.