allenai/olmocr
120 source files scanned · 32 LLM call site(s) · ~4% of LLM spend looks recoverable (static heuristics)
$55per 10k runs today (gpt-5.1, ~2000 prompt tokens)
$52.8per 10k runs with fixes applied
[](https://tokenzero.sh/audit/allenai/olmocr)
Models referenced
gpt-5.1 ×3 (cheaper sibling: gpt-5-mini)claude-3-7-sonnet-20250219 ×2gpt-4o-2024-08-06 ×2gpt-4o ×1 (cheaper sibling: gpt-4o-mini)o4-mini-2025-04-16 ×1o3-2025-04-16 ×1
Findings
| Severity | Where | Finding | Est. |
|---|---|---|---|
| high | olmocr/bench/miners/mine_length_gpt_simple.py:127 | gpt-5.1 used for a task whose prompt reads like extraction/classification Full-size pricing ($1.25/M in, $10/M out) on a constrained task. gpt-5-mini costs ~80% less per call at this shape. Fix: Opt into TokenZero cascade routing (`X-TokenZero-Cascade: auto`) so this call is routed to gpt-5-mini only when the classifier is confident (an opt-in shadow parity monitor is coming; nothing is promised), or switch the model here behind an eval. | 80% |
| high | olmocr/bench/miners/check_old_scans_math.py:59 | Retired model "o4-mini-2025-04-16" The provider shut this model down on 2026-10-23; requests fail or are remapped. Recommended replacement: gpt-5.6-terra. Fix: Move to gpt-5.6-terra (eval first); TokenZero's cascade can route to it, and the gateway warns in X-TokenZero-Warning until you do. | 0% |
| medium | olmocr/synth/claude_client.py:27 | LLM call inside a loop Per-item calls repeat the full prompt prefix for every element. If items are independent and latency-tolerant, the provider Batch API is 50% cheaper, and OpenAI Flex processing (`service_tier: "flex"`) gives the same 50% on ordinary calls without restructuring; if the prefix is shared, caching pays off immediately. Fix: Batch where possible, or set `service_tier: "flex"` on OpenAI calls that can wait; otherwise keep the shared prefix cache-formatted (TokenZero does this automatically). | 25% |
| medium | olmocr/synth/claude_client.py:47 | LLM call inside a loop Per-item calls repeat the full prompt prefix for every element. If items are independent and latency-tolerant, the provider Batch API is 50% cheaper, and OpenAI Flex processing (`service_tier: "flex"`) gives the same 50% on ordinary calls without restructuring; if the prefix is shared, caching pays off immediately. Fix: Batch where possible, or set `service_tier: "flex"` on OpenAI calls that can wait; otherwise keep the shared prefix cache-formatted (TokenZero does this automatically). | 25% |
| medium | olmocr/bench/miners/mine_multi_column.py:87 | LLM call inside a loop Per-item calls repeat the full prompt prefix for every element. If items are independent and latency-tolerant, the provider Batch API is 50% cheaper, and OpenAI Flex processing (`service_tier: "flex"`) gives the same 50% on ordinary calls without restructuring; if the prefix is shared, caching pays off immediately. Fix: Batch where possible, or set `service_tier: "flex"` on OpenAI calls that can wait; otherwise keep the shared prefix cache-formatted (TokenZero does this automatically). | 25% |
| medium | olmocr/bench/miners/mine_blank_pages_gpt.py:93 | "gpt-4o-2024-08-06" retires on 2027-01-14 The provider announced on 2026-10-14 that this model retires on 2027-01-14 (80 days). Recommended replacement: gpt-6-luna. Fix: Move to gpt-6-luna (eval first); TokenZero's cascade can route to it, and the gateway warns in X-TokenZero-Warning until you do. | 0% |
| medium | olmocr/bench/miners/check_headers_footers.py:56 | "o3-2025-04-16" retires on 2026-12-11 The provider announced on 2026-06-11 that this model retires on 2026-12-11 (46 days). Recommended replacement: gpt-5.6-sol. Fix: Move to gpt-5.6-sol (eval first); TokenZero's cascade can route to it, and the gateway warns in X-TokenZero-Warning until you do. | 0% |
| low | olmocr/bench/miners/mine_length_gpt_simple.py:127 | Repeat-prone task with no response-cache opt-in Classification and extraction calls often see identical inputs. Without a determinism signal no proxy can safely serve repeats from cache. Fix: If identical inputs should get identical answers, send `X-TokenZero-Cache: always` on these calls (or `temperature: 0` / a `seed` on models that still accept them). Repeats are then served in <10 ms at $0. | 10% |
| low | olmocr/bench/runners/run_claude.py:33 | Tool definitions sent without a cache breakpoint Tool schemas render before the system prompt in the cached prefix. Without a breakpoint on the last tool they are billed in full on every call. Fix: TokenZero marks the last tool automatically; manually, add `cache_control` to the last tool definition. | 8% |
| low | olmocr/bench/miners/mine_length_gpt_simple.py:127 | "gpt-5.1" retires on 2027-04-01 The provider announced on 2026-10-01 that this model retires on 2027-04-01 (157 days). Recommended replacement: gpt-6-sol. Fix: Move to gpt-6-sol (eval first); TokenZero's cascade can route to it, and the gateway warns in X-TokenZero-Warning until you do. | 0% |
| info | scripts/pii/chatgpt_tag_dolmadocs_v2.py:338 | Repeat-prone task with no response-cache opt-in Classification and extraction calls often see identical inputs. Without a determinism signal no proxy can safely serve repeats from cache. Fix: If identical inputs should get identical answers, send `X-TokenZero-Cache: always` on these calls (or `temperature: 0` / a `seed` on models that still accept them). Repeats are then served in <10 ms at $0. | 10% |
| info | scripts/pii/chatgpt_tag_dolmadocs_v1.py:334 | ~424 static prompt tokens sent on every call; only a stable prefix gets the cache discount A fixed block of ~424 tokens goes out with every call. This provider discounts cached input 90%, but only for a byte-identical prefix of at least 1024 tokens that is routed to the same cache; anything volatile (dates, ids, user text) placed before or inside it, or traffic spread across cache shards, means paying full price. Sent by the call at scripts/pii/chatgpt_tag_dolmadocs_v1.py:370. Fix: Route through TokenZero to get a stable `prompt_cache_key`, or move dynamic content below the static prefix and set `prompt_cache_key` yourself. | 0% |
| info | scripts/pii/chatgpt_tag_dolmadocs_v2.py:306 | ~338 static prompt tokens sent on every call; only a stable prefix gets the cache discount A fixed block of ~338 tokens goes out with every call. This provider discounts cached input 90%, but only for a byte-identical prefix of at least 1024 tokens that is routed to the same cache; anything volatile (dates, ids, user text) placed before or inside it, or traffic spread across cache shards, means paying full price. Sent by the call at scripts/pii/chatgpt_tag_dolmadocs_v2.py:338. Fix: Route through TokenZero to get a stable `prompt_cache_key`, or move dynamic content below the static prefix and set `prompt_cache_key` yourself. | 0% |
| info | scripts/pii/autoscan_dolmadocs.py:251 | ~424 static prompt tokens sent on every call; only a stable prefix gets the cache discount A fixed block of ~424 tokens goes out with every call. This provider discounts cached input 90%, but only for a byte-identical prefix of at least 1024 tokens that is routed to the same cache; anything volatile (dates, ids, user text) placed before or inside it, or traffic spread across cache shards, means paying full price. Sent by the call at scripts/pii/autoscan_dolmadocs.py:287. Fix: Route through TokenZero to get a stable `prompt_cache_key`, or move dynamic content below the static prefix and set `prompt_cache_key` yourself. | 0% |
| info | olmocr/bench/miners/mine_multi_column.py:61 | Unrecognised model id "claude-3-7-sonnet-20250219" Not in the current pricing table: it may be retired or renamed. Retired models return errors or are silently remapped by some providers. Fix: Check the provider model list; TokenZero flags unpriced models in `X-TokenZero-Warning`. | 0% |
Fix all of it with one line
OPENAI_BASE_URL=https://tokenzero.sh/v1 ANTHROPIC_BASE_URL=https://tokenzero.sh
Keys pass through untouched. Cache formatting and exact caching are on by default; cascade routing stays opt-in. How it works