Measure tokens, model costs and cache well

The API bill doubled and traffic is the same. Finance asks which feature spends the most, and the only answer is a spreadsheet that divides characters by four. A pilot agent cost several times its budget. In all three cases the problem is the same: someone estimated instead of measuring.

Measure, don’t estimate

Every response carries the token usage (usage) field with the real numbers:

Field	What it counts
input_tokens	Input that didn’t come from the cache
cache_creation_input_tokens	Input written to the cache on this request
cache_read_input_tokens	Input read from the cache
output_tokens	All output, thinking included
output_tokens_details.thinking_tokens	The part of the output that was thinking

Store it on every call with a feature tag (summaries, chat, reports) and you’ll know exactly who spends what.

To stop an input before paying for it there is token counting: it takes the same request as the Messages API, returns the estimated input tokens and is free. It’s the right control for rejecting the 900-page PDF before sending it, not a character limit. Count with the model you’ll use: the newest models use a different tokenizer, and the same text can produce quite a few more tokens.

The cost of a loop

A cost model for an agent built as “one request with the prompt and the visible answer” always falls short, because:

• every iteration is a request, and it resends the full history, which grows with every tool result;
• tool definitions are billed as input on every request;
• thinking is billed as output even when hidden;
• max_tokens caps each request, not the whole task.

To estimate, multiply by the average number of iterations and add the growing history; then validate it with a pilot and the real usage.

Cache well

Prompt caching reuses an identical prefix across requests: reads cost much less than normal input and lower latency; writes cost somewhat more. What you need to know:

• The prefix goes in order: tools, system prompt, messages. Everything before a cache breakpoint, marked with cache_control, is cached. There are up to four, or automatic caching with a single field on the request.
• Stable content on top, variable content below. A timestamp, the user’s name or a request ID at the start of the system prompt changes the prefix, and there is never a cache hit (cache read).
• Changing tools, the effort level, the thinking configuration or the speed invalidates the cache.
• It’s short-lived: five minutes by default, refreshed on every use; there is a one-hour option for more spaced-out traffic. Below a minimum length, the prompt isn’t cached (with no error).
• It doesn’t free context window: it lowers cost and latency, but the tokens still take up space.

To diagnose, look at usage: if cache_read_input_tokens is zero on requests that should repeat the prefix, something variable ended up before the breakpoint.

The usual traps

• Estimating tokens by dividing characters and not storing usage.
• Asking the model to report how many tokens it used.
• Lowering max_tokens everywhere to “stay within budget”.
• Counting a loop as a single request or leaving out hidden thinking.
• Putting the date or user data before the content you want cached.
• Believing the cache solves a context that no longer fits.

On the exam

If the question is “who spends” or “how much will it cost”, look for usage stored per feature and token counting before sending. If the bill went up without more traffic, suspect the cache: a change that put something variable at the start of the prompt. And for an underestimated agent cost, look for the iterations that resend the history and the thinking billed as output.