vercel/ai
120 source files scanned · 150 LLM call site(s) · ~0% of LLM spend looks recoverable (static heuristics · repo looks like an app)
$140per 10k runs today (claude-opus-5-5, ~2000 prompt tokens)
$140per 10k runs with fixes applied
[](https://tokenzero.sh/audit/vercel/ai)
Models referenced
claude-opus-5-5 ×5 (cheaper sibling: claude-sonnet-5-5)gemini-3.1-pro-preview ×4gpt-6-astra ×2 (cheaper sibling: gpt-6.1-sol)gpt-4.1-mini ×2claude-sonnet-5-5 ×1 (cheaper sibling: claude-haiku-5-5)
Findings
| Severity | Where | Finding | Est. |
|---|---|---|---|
| low | examples/ai-functions/src/stream-text/anthropic/compaction.ts:41 | ~1,203 static prompt tokens re-billed at full price on every call This call sends a large fixed prompt but never marks a cache breakpoint, so Anthropic bills all 1203 tokens at the full input rate each time instead of the cache-read rate (95% cheaper). Sent by the call at examples/ai-functions/src/stream-text/anthropic/compaction.ts:36. Fix: Route through TokenZero (breakpoints are injected automatically) or add `cache_control: {type: "ephemeral"}` to the last system block and the last tool. | 71% |
| low | examples/ai-functions/src/generate-text/anthropic/compaction.ts:41 | ~1,203 static prompt tokens re-billed at full price on every call This call sends a large fixed prompt but never marks a cache breakpoint, so Anthropic bills all 1203 tokens at the full input rate each time instead of the cache-read rate (95% cheaper). Sent by the call at examples/ai-functions/src/generate-text/anthropic/compaction.ts:36. Fix: Route through TokenZero (breakpoints are injected automatically) or add `cache_control: {type: "ephemeral"}` to the last system block and the last tool. | 71% |
| low | examples/ai-e2e-next/app/api/chat/anthropic-compaction/route.ts:44 | ~1,203 static prompt tokens re-billed at full price on every call This call sends a large fixed prompt but never marks a cache breakpoint, so Anthropic bills all 1203 tokens at the full input rate each time instead of the cache-read rate (95% cheaper). Sent by the call at examples/ai-e2e-next/app/api/chat/anthropic-compaction/route.ts:258. Fix: Route through TokenZero (breakpoints are injected automatically) or add `cache_control: {type: "ephemeral"}` to the last system block and the last tool. | 71% |
| low | examples/ai-functions/src/stream-text/openai/compaction.ts:38 | ~1,203 static prompt tokens sent on every call; only a stable prefix gets the cache discount A fixed block of ~1203 tokens goes out with every call. gpt-6-astra discounts cached input 90%, but only for a byte-identical prefix of at least 1024 tokens that is routed to the same cache; anything volatile (dates, ids, user text) placed before or inside it, or traffic spread across cache shards, means paying full price. Sent by the call at examples/ai-functions/src/stream-text/openai/compaction.ts:33. Fix: Route through TokenZero to get a stable `prompt_cache_key`, or move dynamic content below the static prefix and set `prompt_cache_key` yourself. | 15% |
| low | examples/ai-functions/src/generate-text/openai/compaction.ts:38 | ~1,203 static prompt tokens sent on every call; only a stable prefix gets the cache discount A fixed block of ~1203 tokens goes out with every call. gpt-6-astra discounts cached input 90%, but only for a byte-identical prefix of at least 1024 tokens that is routed to the same cache; anything volatile (dates, ids, user text) placed before or inside it, or traffic spread across cache shards, means paying full price. Sent by the call at examples/ai-functions/src/generate-text/openai/compaction.ts:33. Fix: Route through TokenZero to get a stable `prompt_cache_key`, or move dynamic content below the static prefix and set `prompt_cache_key` yourself. | 15% |
| info | examples/ai-functions/src/stream-text/anthropic/context-management.ts:10 | Tool definitions sent without a cache breakpoint Tool schemas render before the system prompt in the cached prefix. Without a breakpoint on the last tool they are billed in full on every call. Fix: TokenZero marks the last tool automatically; manually, add `cache_control` to the last tool definition. | 8% |
Fix all of it with one line
OPENAI_BASE_URL=https://tokenzero.sh/v1 ANTHROPIC_BASE_URL=https://tokenzero.sh
Keys pass through untouched. Cache formatting and exact caching are on by default; cascade routing stays opt-in. How it works