Pydantic AI: prompt compression in one line
The static prefix is formatted for the provider cache (Anthropic breakpoints, OpenAI prompt_cache_key or 12h retention) and tool schemas are minified losslessly before they are sent. Here is the change for Pydantic AI, the switch that turns it on, and the arithmetic on gpt-6.1-sol.
Pydantic AI reaches TokenZero with one base-URL argument and keeps your openai key. On gpt-6.1-sol a cached input token costs $0.100 per million instead of $2.00, so 100,000 calls with a 1,500-token static prefix drop from $700 to $415 a month, 41% less, before any routing. Gateway overhead is 1.52 ms at p50, measured; every call returns an Ed25519-signed receipt.
· Source: pricing table 2026-10-26; overhead from the published benchmark on /benchmarks
At a glance
- Framework
- Pydantic AI (Python)
- Capability
- prompt compression
- Change
- agent.py: base URL argument
- Request shape
- OpenAI chat / Responses
- Turn it on
- X-TokenZero-Cache-Strategy: static
- Workspace setting
- cache_strategy: static, prompt_cache_ttl: 1h
- Reference model
- gpt-6.1-sol ($2.00 in / $10.00 out per 1M, table 2026-10-26)
- Cache-formatting saving on the reference workload
- $285/month (41%) (computed)
- Gateway overhead (measured)
- 1.52 ms p50, 3.42 ms p95
- Parity floor
- 99.8%
- Free tier
- $500/month routed spend
Reference workload on gpt-6.1-sol: 2,000 input (1,500 static) / 300 output tokens, 100,000 calls per month, table 2026-10-26
| Scenario | Per call | Per month | Delta |
|---|---|---|---|
| List price, no gateway | $0.00700 | $700 | — |
| Static prefix cached (prompt compression) | $0.00415 | $415 | $285 (41%) |
| 20% exact-cache repeats | $0.00560 | $560 | $140 (20%) |
| All calls cascaded to gpt-6-luna | $0.00035 | $35 | $665 (95%) |
| Budget guardrail at $500/month | — | capped at $500.00 | 402 after 71,428 calls |
The change
1. Point Pydantic AI at the gateway (agent.py):
-agent = Agent('openai:gpt-6.1-sol')
+from pydantic_ai.models.openai import OpenAIChatModel
+from pydantic_ai.providers.openai import OpenAIProvider
+agent = Agent(OpenAIChatModel('gpt-6.1-sol', provider=OpenAIProvider(base_url='https://tokenzero.sh/v1')))Structured outputs and tool calls pass through unchanged; tool schemas are minified losslessly by the gateway.
2. Turn on prompt compression. Per request: X-TokenZero-Cache-Strategy: static (default) · X-TokenZero-Prompt-Cache-TTL: 1h or 12h. Per workspace (applies to every call without touching code): PATCH /v1/workspaces/settings {cache_strategy: static, prompt_cache_ttl: 1h} or the dashboard toggle.
3. Prove it. npx -y @tokenzero/cli doctor checks the gateway, the signer, the pricing table and your key; then watch input_tokens_cached_pct and compaction tokens saved on the scorecard.
Questions people ask
How do I point Pydantic AI at TokenZero?
Pass base_url / baseURL = https://tokenzero.sh/v1 where Pydantic AI builds its client (agent.py) and keep your openai key; nothing else changes.
What does prompt compression save on gpt-6.1-sol?
On gpt-6.1-sol a cached input token costs $0.100 per million instead of $2.00, so 100,000 calls with a 1,500-token static prefix drop from $700 to $415 a month, 41% less, before any routing.
Does prompt compression change the model's output?
No. Cache formatting only marks or keys the static prefix and tool minification only collapses whitespace in descriptions; names, enums, defaults and examples are never touched.
What does TokenZero cost?
Free up to $500 per month of routed spend; then 15% of verified savings with a 3x-ROI-or-$0 rule, capped plans available. Gateway overhead measured at 1.52 ms p50.