
I treat token cost the way I treat error rates and latency: measured continuously, attributed to whatever caused it and checked against the real bill. A monthly invoice is one number, a month late, and by the time it shows that something is wrong the money has been spent.
What the invoice did not tell us
A platform we run had a bug in which every turn that used tools took 44 seconds to produce its first token. The cause was a prompt cache invalidated on every turn: roughly 74,000 cache-write tokens per turn and no cache reads. I wrote up how we found the cause in What broke the prompt cache. What matters here is how we noticed at all. It was not from the invoice, which was a single figure and a month behind. It was from latency, weeks after the meter had started running.
Cost tracked as an afterthought means finding a five-figure bug from a latency complaint, weeks late. Cost tracked as an operating metric means finding it on a graph the same day.
Why is one cost figure not enough?
The cost tracker we built covers every coding agent we run and every model we serve. For each agent it reports five kinds of the same dollar: API-equivalent cost, metered API cost, subscription-included cost, estimated cost where a provider does not itemize, and unknown cost where we cannot yet tell.
The categories exist because the dollars differ. We run agents on subscriptions as well as on metered APIs, so the same unit of work can cost nothing at the margin on one route and real money on another. A cost model that cannot show that averages two different economics into one number that describes neither.
Cache pricing adds another axis. A cache write at the five-minute tier and one at the one-hour tier are billed at different rates, and we carry them as separate line items. Averaging them hides exactly the kind of anomaly that produced the 44-second bug.
We also use several model providers on purpose. Each itemizes usage differently, bills cache behavior differently and exposes billing data in its own shape. The tracker has to normalize them into one internal schema before it can even ask whether an estimate was close.
Where the numbers come from
None of the figures come from asking an agent how much it spent. An agent’s account of its own usage is a self-report, and I do not accept those for billing any more than for code, for the reasons in Agent self-reports are not evidence. Every figure traces back to the usage field the provider returned on the response, or to the line the provider billed.
The estimates do not grade themselves either. On a schedule, the tracker reconciles its running estimates against the actual invoices. If the two drift apart past a threshold, we treat that as a bug in the estimator and fix it.
Attribution goes three levels deep: per account, per agent, per machine. One account spending ten times what comparable accounts spend is rarely a heavy user. It is usually something misbehaving in that account’s path, the same way one server using ten times the CPU of its peers is a leak. With that per-account view in place, we would have caught the cache bug two weeks sooner.
The ceiling has to sit inside the run
The cost data is available from the command line, through a tool server agents can call, through an API and on a dashboard. A dashboard alone would not be enough. The agent spending the tokens should be able to check its budget in the middle of a task, the way a person checks a balance before a purchase. An agent that learns what it spent from a human reading a graph afterwards has already spent it.
The same idea runs one layer down. The framework we use to turn a goal into a task graph and fan it out across many agents carries a budget governor as a core component. It converts token usage to dollars through a pricing table, adds up spend across every agent in the run in real time, and halts the run when it crosses a ceiling set in advance. It is a hard limit in the literal sense of Hard limits are governance: a number set before the run starts.
A governor that exists only on a dashboard, read after the run has finished, is a receipt.
Agents need this more than ordinary services do, because a stuck agent does not fail loudly. It retries a failing tool call, or reads the same file five times trying to understand an error, or starts a sub-agent that gets stuck on the same problem. A person doing that work would get bored or give up. An agent keeps going until something outside it stops it. Alerts on the ceiling are retried, because a webhook that fires once into a server error has told nobody anything.
The discipline comes down to three questions that usage data should answer on any day: which account costs the most per unit of output, which agent has the worst cache-read ratio, and whether the estimate matches last month’s invoice to within a few percent.


