playbook · updated 2026-10-26

Prompt cache misses

Cached-input tokens stay at 0 even though the system prompt never changes.

Why it happens

Provider caches match an exact prefix: any variation before the static part (timestamps, user ids, reordered tools), a prefix under the minimum size (1,024 tokens on most models, 512 on Opus 5 / Haiku 5.5), or on Anthropic a missing cache_control breakpoint, defeats it.

Fixes

  1. Read X-TokenZero-Cache-Format: it names what the formatter did and why a prefix is too small.
  2. Put tools first, then the system prompt, then dynamic content; keep the static part byte-identical. The CAPC step minifies tool schemas deterministically so identical tools always produce identical prefixes.
  3. On OpenAI pre-5.6 models the gateway sets prompt_cache_key from the static prefix; on GPT-5.6+ Responses, X-TokenZero-Prompt-Cache-TTL: 12h keeps the cache warm longer.
  4. Run npx -y @tokenzero/cli audit: uncached static prompts are the first finding it reports, with the line.

Start here

curl -s -X POST https://tokenzero.sh/v1/signup -d '{}'   # workspace + key, no form
npx -y @tokenzero/cli init                                # or: audits the repo, writes .env
OPENAI_BASE_URL=https://tokenzero.sh/w//v1
ANTHROPIC_BASE_URL=https://tokenzero.sh/w/
# provider keys stay in your code exactly as they are; nothing else changes

Questions

How do I know the cache worked?
The receipt's usage.cached count and X-TokenZero-Savings-USD; both come from the provider's own usage numbers, not an estimate.