Cost to Run an Agentic Tool-Calling Loop with LLM APIs

An agent loop is not one API call — it is a chain of them, and each link re-sends everything before it. Step 1 sends the system prompt plus tool schemas (2,000 tokens). Step 2 sends all of that again plus the model's first tool call (350 tokens) and the tool's result (700 tokens). By step 12 a single request is carrying 13,550 input tokens, and the run as a whole has billed 107,900 input tokens against only 4,700 output tokens. Counting each message once — the intuition most budgets are built on — gives 15,100 for the whole run, about 7.5× too low. This is why agent bills surprise people: the transcript is re-read on every step, so input tokens grow with the square of the step count while output tokens grow linearly. It is also why prompt caching matters more here than in any other workload: at the defaults, 86% of those input tokens are a re-read of a prefix that never changed.

Formula: input_tokens = (N + 1) × S + (A + T) × N × (N + 1) / 2 and output_tokens = N × A + F. N is the number of tool-calling steps, S the system-and-tool-schema prefix, A the assistant tokens generated per step, T the tool-result tokens appended per step, and F the final answer. The N × (N + 1) / 2 term is the re-sent transcript — it is what makes the loop quadratic. Cost is then (input_tokens × input_price + output_tokens × output_price) × runs / 1,000,000.

On the caching lens: with incremental prompt caching only the newest turn is fresh input; everything earlier is a cache read, so cache-eligible tokens are input_tokens − (S + N × (A + T)).We apply a discount only where the provider publishes a cached-input rate for that model; models without one are priced at their normal input rate and marked in the table, never silently discounted.

All prices are per million tokens, sourced directly from official provider pricing pages. No fabricated numbers — every model in the table links to its detail page, which cites the official pricing source.

Why does an agentic loop cost so much more than a single API call?

Because the conversation is stateless. Every step re-sends the full transcript so far — system prompt, tool schemas, every prior tool call and every prior tool result — and you are billed for all of it again. Doubling the number of steps therefore does not double the cost: at the defaults above, going from 12 steps to 24 raises input tokens about 3.4×. The calculator computes the exact figure for the loop shape you enter.

How do you calculate the token cost of a tool-calling agent?

For a run of N steps with a system-and-tools prefix of S tokens, A assistant tokens generated per step and T tool-result tokens appended per step: total input = (N+1)×S + (A+T)×N×(N+1)/2, and total output = N×A + the final answer. The quadratic term is the re-sent transcript. Multiply by your number of runs and by each model's per-million-token prices to get the dollar figure — which is exactly what the table above does, for every current model.

How much does prompt caching save on an agent loop?

More than on any other workload, because the re-sent prefix is identical every step. With incremental caching, only the newest turn is fresh input; everything before it is a cache read. At the defaults that is 86% of all input tokens, and cached input is typically billed at 10–25% of the normal input rate. Switch on the caching lens above to see each model's cached cost at its own published cached-input rate — the calculator only applies a discount where the provider publishes one, and marks the models that do not.

Which LLM is cheapest for running agents?

Not necessarily the one with the lowest headline price. Agent loops are overwhelmingly input-heavy — about 96% of tokens at the defaults — so input pricing and the cached-input rate dominate, and a model with cheap output but expensive input ranks worse here than its sticker price suggests. The table above ranks every current model by total cost for the loop shape you enter, cheapest first.

How many steps does a typical agent run take?

It depends entirely on the task. A single lookup-and-answer is 1–3 steps; a research or refactoring task is commonly 10–30; a long autonomous run can exceed 100. Because cost scales quadratically, the step count is the most expensive number in your whole configuration — capping it is usually a larger saving than switching models. Set the steps field above to your own observed median rather than trusting a default.