LLM tokenizer and compressor
Count tokens for five providers and preview the lossless whitespace compaction the gateway applies to tool schemas.
The sample two-tool schema is 262 tokens on the OpenAI tokenizer, 382 on Anthropic's and 276 on Gemini's; the gateway's lossless compaction (whitespace runs inside descriptions only) brings the OpenAI count to 143, 119 tokens (45%) saved on every call that carries it. Names, enums, defaults and examples are never touched.
· Source: pricing table 2026-10-26 and the gateway's own tokenizer and benchmark files; your inputs never leave the browser except token counts, which are not stored
At a glance
- Tokenizers
- openai, anthropic, gemini, xai, deepseek
- Sample tokens (openai)
- 262 (computed)
- Sample tokens (anthropic)
- 382 (computed)
- Sample tokens (gemini)
- 276 (computed)
- Sample tokens (xai)
- 276 (computed)
- Sample tokens (deepseek)
- 276 (computed)
- After compaction (OpenAI)
- 143 (computed)
- Saved per call
- 119 tokens (45%) (computed)
- What compaction changes
- whitespace runs inside description strings
- What it never changes
- names, enums, defaults, examples, patterns, required lists
Sample tool schema: tokens per provider tokenizer, before and after lossless compaction
| Tokenizer | Before | After | Saved |
|---|---|---|---|
| openai | 262 | 143 | 119 |
| anthropic | 382 | 209 | 173 |
| gemini | 276 | 151 | 125 |
| xai | 276 | 151 | 125 |
| deepseek | 276 | 151 | 125 |
Tokenizer and compressor
Questions people ask
Is the compaction lossless?
Yes. Only runs of whitespace inside description strings are collapsed to one space; the words, their order, and every other field are unchanged, and the cache-tier guard skips compaction when it would drop a prefix below the model's cache minimum.
Why do the five counts differ?
Each provider tokenizes differently; the gateway uses a per-provider factor calibrated on the official tokenizers so audits, receipts and this tool agree with the provider's usage numbers.
Is my text stored?
No. It is counted, optionally compacted, returned and discarded.
What does TokenZero cost?
Free up to $500 per month of routed spend; then 15% of verified savings with a 3x-ROI-or-$0 rule. Gateway overhead is 1.52 ms p50, measured.