Vercel AI SDK Core (Node): semantic caching in one line
Byte-identical repeats are served from the exact cache at $0 and near-duplicates from the opt-in semantic cache. Here is the change for Vercel AI SDK Core (Node), the switch that turns it on, and the arithmetic on claude-sonnet-5-5.
Vercel AI SDK Core (Node) reaches TokenZero with one base-URL argument and keeps your anthropic key. Every byte-identical repeat is served at $0: at a 20% repeat share, 100,000 reference calls on claude-sonnet-5-5 cost $560 instead of $700 a month; the semantic cache extends that to near-duplicates when you opt in.
· Source: pricing table 2026-10-26; overhead from the published benchmark on /benchmarks
At a glance
- Framework
- Vercel AI SDK Core (Node) (TypeScript)
- Capability
- semantic caching
- Change
- lib/llm.ts: base URL argument
- Request shape
- Anthropic Messages
- Turn it on
- X-TokenZero-Semantic-Cache: true
- Workspace setting
- semantic_cache: true, cache_ttl: 3600
- Reference model
- claude-sonnet-5-5 ($2.00 in / $10.00 out per 1M, table 2026-10-26)
- Exact-cache saving at a 20% repeat share
- $140/month (computed)
- Gateway overhead (measured)
- 1.52 ms p50, 3.42 ms p95
- Parity floor
- 99.8%
- Free tier
- $500/month routed spend
Reference workload on claude-sonnet-5-5: 2,000 input (1,500 static) / 300 output tokens, 100,000 calls per month, table 2026-10-26
| Scenario | Per call | Per month | Delta |
|---|---|---|---|
| List price, no gateway | $0.00700 | $700 | — |
| Static prefix cached (prompt compression) | $0.00415 | $415 | $285 (41%) |
| 20% exact-cache repeats | $0.00560 | $560 | $140 (20%) |
| All calls cascaded to claude-haiku-5-5 | $0.00035 | $35 | $665 (95%) |
| Budget guardrail at $500/month | — | capped at $500.00 | 402 after 71,428 calls |
The change
1. Point Vercel AI SDK Core (Node) at the gateway (lib/llm.ts):
-import { anthropic } from '@ai-sdk/anthropic';
+import { createAnthropic } from '@ai-sdk/anthropic';
+const anthropic = createAnthropic({ baseURL: 'https://tokenzero.sh/v1', apiKey: process.env.ANTHROPIC_API_KEY });
const { text } = await generateText({ model: anthropic('claude-sonnet-5-5'), prompt });The Anthropic provider appends /messages itself, so the base URL ends in /v1.
2. Turn on semantic caching. Per request: X-TokenZero-Semantic-Cache: true (exact cache is automatic for temperature-0 or seeded requests; X-TokenZero-Cache: always forces it). Per workspace (applies to every call without touching code): PATCH /v1/workspaces/settings {semantic_cache: true, cache_ttl: 3600} or the dashboard toggle.
3. Prove it. npx -y @tokenzero/cli doctor checks the gateway, the signer, the pricing table and your key; then watch cache_hit_rate_pct and savings from engine cache on the scorecard.
Questions people ask
How do I point Vercel AI SDK Core (Node) at TokenZero?
Pass base_url / baseURL = https://tokenzero.sh where Vercel AI SDK Core (Node) builds its client (lib/llm.ts) and keep your anthropic key; nothing else changes.
What does semantic caching save on claude-sonnet-5-5?
Every byte-identical repeat is served at $0: at a 20% repeat share, 100,000 reference calls on claude-sonnet-5-5 cost $560 instead of $700 a month; the semantic cache extends that to near-duplicates when you opt in.
Does semantic caching change the model's output?
The exact cache returns the provider's own earlier response for a byte-identical request; the semantic cache is opt-in and only matches within the distance you accept. Non-deterministic requests are never cached unless you force it.
What does TokenZero cost?
Free up to $500 per month of routed spend; then 15% of verified savings with a 3x-ROI-or-$0 rule, capped plans available. Gateway overhead measured at 1.52 ms p50.