switch guide · updated 2026-10-26

Switch from LiteLLM Proxy to TokenZero in one line

LiteLLM Proxy is a self-hosted Python proxy that normalises many providers behind one OpenAI-shaped endpoint. TokenZero is a BYOK savings gateway: you keep OpenAI, Anthropic and Gemini request shapes (TokenZero forwards each provider's native shape unchanged), and every request comes back with a signed receipt for what it saved.

The change

OPENAI_BASE_URL=https://tokenzero.sh/w//v1
ANTHROPIC_BASE_URL=https://tokenzero.sh/w/
# provider keys stay in your code exactly as they are; nothing else changes

Create the workspace first:

curl -s -X POST https://tokenzero.sh/v1/signup -d '{}'   # workspace + key, no form
npx -y @tokenzero/cli init                                # or: audits the repo, writes .env

What differs from LiteLLM Proxy

  • Hosted, nothing to run: no database, no Redis, no config file.
  • Measured overhead on the same box and the same mock upstream: TokenZero +1.52 ms p50 vs LiteLLM Proxy +16.25 ms p50 (see /benchmarks for the method and version).
  • Signed receipts and a hash-chained ledger instead of a spend log.

What TokenZero does

  • Bring your own key. Provider keys pass through untouched; TokenZero never holds them.
  • Four engines. Prompt-cache formatting (Anthropic breakpoints, OpenAI prompt_cache_key / 12h retention), exact and semantic response cache, calibrated cascade routing with a shadow-parity breaker, and a signed savings ledger.
  • Measured, not claimed. Gateway overhead and the audit corpus are published on /benchmarks with the scripts that produced them.
  • Free up to $500/month of routed spend; paid plans are a share of verified savings with a 3x-ROI-or-$0 rule.

Verified, not reported

Every request through the gateway gets an Ed25519-signed receipt with list-price baseline, actual provider cost and the difference, chained per workspace (seq, prev_receipt_hash), verifiable offline with npx -y @tokenzero/cli verify --chain.

Questions

Can I keep LiteLLM in front of TokenZero?
Yes: point LiteLLM's api_base for each provider at the TokenZero base URL; TokenZero adds caching, routing advice and receipts underneath.

Is there a migration tool?
No tool is needed: change the base URL (or the /w/<key>/ prefix) and keep your provider keys. Run npx -y @tokenzero/cli audit first to see what the gateway will act on.