When an agent bill comes in at ten times the estimate, the instinct is to look for one big bug. There almost never is one. What actually happens is that four independent multipliers stack on top of each other, and each one looks reasonable in isolation. Anthropic’s own engineering team found that multi-agent systems use about 15x as many tokens as a chat interaction, and even a single agent uses about 4x (Source: Anthropic engineering). Multiply that by model choice, hidden reasoning tokens, and SDK retry defaults, and a 10x overrun stops being mysterious.

This guide does two things. First, it puts exact numbers on each multiplier using worked arithmetic you can check by hand. Second, it builds a framework-agnostic budget_guard.py sketch step by step, so you finish with hard caps, a retry policy, and alerts you can wrap around any model client.

One caveat up front: every dollar figure below is a price as listed on the cited pages at the 2025-06 snapshot (two OpenAI launch prices date to 2024). Verify against the live pricing pages before relying on any of them.

The Four Multipliers

Multiplier 1: Model choice, up to 100x

The same workload priced on different models differs by two orders of magnitude. At launch, gpt-4o-mini was listed at $0.15 per million input tokens and $0.60 per million output (Source: OpenAI), while gpt-4o launched at $5/$15 (Source: OpenAI) and o1 at $15/$60 (Source: OpenAI pricing). That is a 100x spread between gpt-4o-mini and o1 on both input and output. Anthropic’s spread is narrower but real: Claude Sonnet at $3/$15 versus Claude Haiku at $0.80/$4 (Source: Anthropic pricing). Gemini 2.5 Pro lists at $1.25/$10 for prompts up to 200k tokens (Source: Google AI pricing).

Prices also move within a model family: gpt-4o launched at $5/$15 but was listed at $2.50/$10 on the pricing page at the 2025-06 snapshot (Source: OpenAI pricing) — a 2x change inside one model name. If your cost estimate was written once and never re-priced, it is already wrong.

Guard fix: a model allowlist. Escalation to a pricier model must be an explicit, logged decision, not a default.

Multiplier 2: Context growth, 16.75x

Agents accumulate context. Each tool result gets appended to the conversation and resent on every subsequent turn, so input tokens grow quadratically with turn count. In EX 2, turn 20 alone sends 62,000 input tokens, so the growth only shows up when you sum the run rather than read one call.

Guard fix: a per-call input token cap that drops the oldest tool results when the prompt exceeds it.

Multiplier 3: Hidden reasoning tokens

Reasoning and thinking tokens are billed at the output rate but do not appear in the response text. This is documented across all three major providers: OpenAI’s reasoning guide (Source: OpenAI docs), Anthropic’s extended thinking docs (Source: Anthropic docs), and Gemini’s thinking docs (Source: Google AI docs). If you estimate cost from the visible response, you systematically undercount output. If you cap output by visible length, your cap does not cap what you pay for.

Guard fix: a per-call output cap passed to the provider as max_tokens or a thinking budget, then reconciled against the usage object’s output figure, which includes reasoning tokens.

Multiplier 4: Retry and fan-out, up to 3x

The OpenAI and Anthropic Python SDKs both default to max_retries=2 with exponential backoff (Source: openai-python, Source: anthropic-sdk-python). AWS SDKs default to 3 max attempts in standard mode (Source: AWS SDK reference). One logical request therefore becomes up to three physical requests before failure surfaces. On top of that, agent fan-out — spawning subagents or parallel tool calls — multiplies volume again. Anthropic’s guidance to prefer workflows over autonomous agents when the task is predictable (Source: Anthropic) is partly a cost argument: predictable structure means bounded fan-out.

Rate limits interact with this: limits are enforced as RPM and TPM per organization, model, and tier, and exceeding them returns HTTP 429 (Source: OpenAI rate limits) — which triggers exactly the retries you did not budget for.

Guard fix: max_retries=1 with jittered backoff, plus a circuit breaker that stops the run after repeated consecutive failures.

Worked Arithmetic: Three Examples, Line by Line

All prices here are as listed at the 2025-06 snapshot; verify before relying on them.

EX 1 — Model choice on an identical workload

Workload: 200 calls, each with 10,000 input tokens and 500 output tokens. That is 2,000,000 input tokens and 100,000 output tokens per run.

gpt-4o-mini:  2.0M x $0.15/M = $0.30   (input)
              0.1M x $0.60/M = $0.06   (output)
              total = $0.36 per run

gpt-4o:       2.0M x $2.50/M = $5.00
              0.1M x $10.00/M = $1.00
              total = $6.00 per run   -> 16.7x gpt-4o-mini

o1:           2.0M x $15.00/M = $30.00
              0.1M x $60.00/M = $6.00
              total = $36.00 per run  -> 100x gpt-4o-mini

Now the hidden multiplier: add 2,000 thinking tokens per call at o1’s output rate.

200 calls x 2,000 reasoning tokens = 400,000 extra output tokens
400,000 x $60/M = $24.00 additional

That is a 66% cost increase on the o1 run ($24.00 on $36.00, the EX 1 arithmetic) from tokens you never see in the response text, which OpenAI bills as output (OpenAI reasoning guide).

EX 2 — Context growth over a 20-turn agent

Setup: system plus user prompt is 2,000 tokens. Each turn appends a 3,000-token tool result, and the entire conversation is resent every turn.

Input on turn i          = 2,000 + 3,000 x i
Total over 20 turns      = 40,000 + 3,000 x (1+2+...+20)
                         = 40,000 + 3,000 x 210
                         = 670,000 tokens
Naive estimate           = 20 calls x 2,000 tokens = 40,000 tokens
Overrun                  = 670,000 / 40,000 = 16.75x

Priced at gpt-4o-mini’s input rate:

Actual:   670,000 x $0.15/M = $0.1005 per run
Naive:     40,000 x $0.15/M = $0.006  per run
At 10,000 runs/day: $1,005/day vs $60/day

Even on the cheapest model in the table, unmanaged context growth turns a $60/day line item into a $1,005/day one, using the gpt-4o-mini listed input rate from EX 2.

EX 3 — The cache decision

Same 20-turn agent, but with a 50,000-token stable prefix (a large system prompt plus retrieved documents), across 200 calls. Anthropic prices cache writes at 1.25x base input and cache reads at 0.1x (Source: Anthropic prompt caching).

Uncached: 200 x 50,000 = 10,000,000 tokens x $3.00/M = $30.00

Cached:   write once:  50,000 x ($3.00 x 1.25)/M = 50,000 x $3.75/M = $0.1875
          200 reads:  10,000,000 x ($3.00 x 0.1)/M = 10,000,000 x $0.30/M = $3.00
          total = $3.1875 ≈ $3.19

Saving:   $30.00 / $3.19 ≈ 9.4x

For comparison, OpenAI’s cached input is 50% of base price and automatic for prompts of at least 1,024 tokens (Source: OpenAI), and Gemini’s cached tokens run about 0.25x the input rate plus storage cost (Source: Google AI docs).

The worked cost table

Example Workload Configuration Cost per run Multiplier vs baseline
EX 1 200 calls × (10k in / 500 out) gpt-4o-mini $0.36 1x (baseline)
EX 1 200 calls × (10k in / 500 out) gpt-4o $6.00 16.7x
EX 1 200 calls × (10k in / 500 out) o1 $36.00 100x
EX 1 o1 + 2,000 reasoning tokens/call o1, hidden output $60.00 167x vs gpt-4o-mini (incl. $24.00 hidden)
EX 2 20 turns, 3k tool result appended gpt-4o-mini input $0.1005 16.75x vs naive $0.006
EX 2 Same, at 10,000 runs/day gpt-4o-mini input $1,005/day vs $60/day naive
EX 3 200 calls, 50k stable prefix Sonnet, uncached $30.00 1x (baseline)
EX 3 200 calls, 50k stable prefix Sonnet, cached $3.19 9.4x saving

Why the Cap Must Live in Your Code

OpenAI’s documented spend controls are organization-level: usage tiers, rate limits, and an organization-wide monthly budget (Source: OpenAI help). There is no per-thread or per-agent cap in the API. That is the structural reason a guard has to exist in your application layer: the provider will stop you at the org level, after every agent in the fleet has already spent. Per-agent budgets are your code’s job.

Building budget_guard.py, Step by Step

The guard is framework-agnostic: it wraps any model client and exposes two hooks — preflight before a call and record after it. Append each block to a single budget_guard.py.

Step 1: The price table and allowlist

# budget_guard.py — framework-agnostic cost guard for LLM calls
# Prices: USD per 1M tokens, as listed at the 2025-06 snapshot.
# VERIFY against live pricing pages before relying on them.

PRICES = {
    "gpt-4o-mini":    {"in": 0.15,  "out": 0.60},
    "gpt-4o":         {"in": 2.50,  "out": 10.00},
    "o1":             {"in": 15.00, "out": 60.00},
    "claude-sonnet":  {"in": 3.00,  "out": 15.00},
    "claude-haiku":   {"in": 0.80,  "out": 4.00},
    "gemini-2.5-pro": {"in": 1.25,  "out": 10.00},
}

# Escalation to a pricier model must be explicit, never a default.
MODEL_ALLOWLIST = {"gpt-4o-mini", "claude-haiku"}

Step 2: Pre-call token estimation

You cannot cap what you cannot count before the call. OpenAI’s recommended path is local counting with tiktoken (Source: OpenAI Cookbook); Anthropic exposes a server-side count_tokens endpoint (Source: Anthropic API).

import tiktoken  # pip install tiktoken

_ENC = tiktoken.encoding_for_model("gpt-4o-mini")

def count_tokens(text: str) -> int:
    # For Claude models, prefer POST /v1/messages/count_tokens instead.
    return len(_ENC.encode(text))

def trim_oldest(parts: list[str], cap: int) -> list[str]:
    """Drop oldest tool results until the prompt fits the input cap.
    Keeps parts[0] (system) and parts[1] (latest user turn)."""
    while sum(count_tokens(p) for p in parts) > cap and len(parts) > 2:
        del parts[2]
    return parts

Step 3: Usage records and USD conversion

from dataclasses import dataclass

@dataclass
class UsageRecord:
    model: str
    input_tokens: int
    output_tokens: int   # includes reasoning/thinking tokens
    usd: float

def usd(model: str, in_tok: int, out_tok: int) -> float:
    p = PRICES[model]
    return in_tok / 1_000_000 * p["in"] + out_tok / 1_000_000 * p["out"]

Step 4: Hard caps

@dataclass
class BudgetConfig:
    run_usd_cap: float = 5.00           # enforced BEFORE call N+1
    call_input_token_cap: int = 8_000   # triggers trim of oldest tool results
    call_output_token_cap: int = 1_000  # pass as max_tokens on each call; preflight does not enforce it
    max_calls_per_run: int = 200        # loop cap
    max_fanout: int = 5                 # parallel subagent cap
    alert_thresholds: tuple = (0.50, 0.80, 1.00)

class BudgetExceeded(Exception):
    pass

class BudgetGuard:
    def __init__(self, cfg: BudgetConfig):
        self.cfg = cfg
        self.records: list[UsageRecord] = []
        self.calls = 0
        self._fired: set[float] = set()

    def spend(self) -> float:
        return sum(r.usd for r in self.records)

    def preflight(self, model, prompt_parts, max_output, fanout=1) -> float:
        if model not in MODEL_ALLOWLIST:
            raise BudgetExceeded(f"{model} not in allowlist; escalate explicitly")
        if self.calls + fanout > self.cfg.max_calls_per_run:
            raise BudgetExceeded("loop cap reached")
        if fanout > self.cfg.max_fanout:
            raise BudgetExceeded("fan-out cap reached")
        if sum(count_tokens(p) for p in prompt_parts) > self.cfg.call_input_token_cap:
            prompt_parts[:] = trim_oldest(prompt_parts, self.cfg.call_input_token_cap)
        est = usd(model, sum(count_tokens(p) for p in prompt_parts), max_output)
        if self.spend() + est * fanout > self.cfg.run_usd_cap:
            raise BudgetExceeded(
                f"run cap: ${self.spend():.4f} spent, est ${est:.4f} x {fanout}")
        return est

    def record(self, model, in_tok, out_tok) -> UsageRecord:
        rec = UsageRecord(model, in_tok, out_tok, usd(model, in_tok, out_tok))
        self.records.append(rec)
        self.calls += 1
        self._fire_alerts()
        return rec

    def _fire_alerts(self):
        frac = self.spend() / self.cfg.run_usd_cap
        for t in self.cfg.alert_thresholds:
            if frac >= t and t not in self._fired:
                self._fired.add(t)
                print(f"[budget] {int(t*100)}% of run cap used "
                      f"(${self.spend():.4f} / ${self.cfg.run_usd_cap:.2f})")

The run cap is checked in preflight — before call N+1 is issued — using the pre-call estimate. The record call then reconciles against actuals from the response’s usage object, which is where hidden reasoning tokens show up.

Step 5: Retry policy and circuit breaker

import random, time

class GuardedClient:
    """Wraps any client exposing client.call(model, parts, max_output)."""
    def __init__(self, client, guard: BudgetGuard, max_retries: int = 1,
                 breaker_threshold: int = 5, breaker_cooldown: float = 60.0):
        self.client, self.guard = client, guard
        self.max_retries = max_retries   # SDK default is 2 -> up to 3x volume
        self.breaker_threshold = breaker_threshold
        self.breaker_cooldown = breaker_cooldown
        self._failures = 0
        self._open_until = 0.0

    def call(self, model, prompt_parts, max_output):
        if time.time() < self._open_until:
            raise BudgetExceeded("circuit breaker open; cooling down")
        self.guard.preflight(model, prompt_parts, max_output)
        for attempt in range(self.max_retries + 1):
            try:
                resp = self.client.call(model, prompt_parts, max_output)
                self._failures = 0
                self.guard.record(model, resp.input_tokens, resp.output_tokens)
                return resp
            except Exception:
                self._failures += 1
                if self._failures >= self.breaker_threshold:
                    self._open_until = time.time() + self.breaker_cooldown
                    raise
                if attempt < self.max_retries:
                    time.sleep(0.5 * 2 ** attempt + random.uniform(0, 0.25))
        raise BudgetExceeded("retries exhausted")

The wrapper must be the only retry layer: create the underlying SDK client with max_retries=0, or SDK retries nest inside the wrapper’s retries and worst-case volume multiplies. With max_retries=1 here, the wrapper allows at most 2 attempts per logical call. The circuit breaker stops a failing dependency from burning budget in a loop.

Step 6: Golden tests replaying EX 1–3

def test_ex1_model_multiplier():
    g = BudgetGuard(BudgetConfig(run_usd_cap=1_000.00))
    for _ in range(200):
        g.record("gpt-4o-mini", 10_000, 500)
    assert abs(g.spend() - 0.36) < 1e-9        # $0.36/run
    g2 = BudgetGuard(BudgetConfig(run_usd_cap=1_000.00))
    for _ in range(200):
        g2.record("o1", 10_000, 500)
    assert abs(g2.spend() - 36.00) < 1e-9      # 100x

def test_ex2_context_growth():
    total_in = sum(2_000 + 3_000 * i for i in range(1, 21))
    assert total_in == 670_000
    assert abs(total_in / 40_000 - 16.75) < 1e-9
    assert abs(total_in / 1e6 * 0.15 - 0.1005) < 1e-9
    assert abs(0.1005 * 10_000 - 1_005.0) < 1e-9   # $1,005/day vs $60/day

def test_ex3_cache_decision():
    uncached = 10_000_000 / 1e6 * 3.00
    cached = 50_000 / 1e6 * 3.00 * 1.25 + 10_000_000 / 1e6 * 3.00 * 0.1
    assert abs(uncached - 30.00) < 1e-9
    assert abs(cached - 3.1875) < 1e-9             # 9.4x saving

The EX 1 test runs the guard against the price table. EX 2 and EX 3 are arithmetic checks on their own constants and do not exercise the guard or the price table, so a price change will not fail them. Treat them as documentation of the arithmetic, not as a regression net. The code is a sketch that was not executed in this run.

Cache and Batch: The Two Discounts Worth Wiring In

EX 3 showed a 9.4x saving from caching a stable prefix with Anthropic’s 0.1x reads and 1.25x writes. OpenAI’s 50% cached-input discount is automatic above 1,024 tokens, so structuring prompts with a stable prefix is free money there (Source: OpenAI). Gemini’s roughly 0.25x cached rate plus storage makes the same structuring worthwhile (Source: Google AI docs). Separately, OpenAI’s Batch API prices at 50% of the interactive rate (Source: OpenAI batch docs) — anything that does not need to run inside an interactive loop should not pay interactive prices. The guard’s cache policy hook is where you enforce “stable prefix first, cacheable prefix second.”

The Leak Table

Leak Mechanism Multiplier Guard fix
Model drift Pricier model on identical workload Up to 100x (EX 1) Model allowlist + explicit escalation
Context growth Tool results resent every turn 16.75x (EX 2) Per-call input cap; drop oldest tool results
Hidden reasoning tokens Billed as output, invisible in text +$24.00 on EX 1’s o1 run Output cap read from usage object, not rendered text
Retry / fan-out SDK default 2 retries; AWS 3 attempts Up to 3x request volume max_retries=1, jittered backoff, circuit breaker, fan-out cap

FAQ

Does prompt caching make context growth a non-issue? No. It makes it cheaper — 9.4x cheaper in EX 3 (Anthropic prompt caching) — but the write costs 1.25x base input on Anthropic, Gemini charges storage, and cache hits only apply to stable prefixes. A growing tail of fresh tool results is still billed at full rate every turn. Cache the prefix; cap the tail.

Why not just rely on the provider’s spend limits? Because they are organization-level. OpenAI’s documented controls are usage tiers, rate limits, and an org monthly budget (Source: OpenAI help); there is no per-agent or per-thread cap in the API. Org-level caps stop the fleet after the damage; per-run caps stop the run before the next call.

How do I estimate cost before making the call? Count tokens pre-call — tiktoken locally for OpenAI models (Source: OpenAI Cookbook) or Anthropic’s count_tokens endpoint (Source: Anthropic API) — then apply the price table. preflight in the guard does exactly this and raises before the request is sent.

Are reasoning tokens really billed as output even though I can’t see them? Yes, per all three vendors’ documentation: OpenAI (Source), Anthropic (Source), and Gemini (Source). In EX 1, 2,000 invisible thinking tokens per call added $24.00 to a $36.00 o1 run.

Should I move work to the Batch API? Anything that tolerates latency should. Batch pricing is 50% of interactive (Source: OpenAI batch docs), which halves the bill, while caching cut EX 3 by 9.4x. Interactive agent loops pay full rate by definition, which is why they need the guard.

The Bottom Line

A 10x overrun is not one bug; it is four multipliers stacking: model choice (up to 100x), context growth (16.75x in a 20-turn agent), hidden reasoning tokens billed as output, and retry/fan-out volume (up to 3x from SDK retry defaults alone). Provider controls stop at the organization level, so per-run caps must live in your code. The budget_guard.py above gives you five of them — an allowlist, a pre-call estimate, a per-run USD cap enforced before call N+1, a one-retry policy with a circuit breaker, and alerts at 50/80/100% — with golden tests that pin the guard to the worked arithmetic. Build the guard before you need it; the arithmetic says the bill arrives faster than the intuition does.

How This Guide Was Built

This guide is based on official vendor documentation and worked arithmetic; we did not run the guard against a live bill. Every dollar figure traces to a cited pricing page as listed at the 2025-06 snapshot (gpt-4o-mini and gpt-4o launch prices date to their 2024 announcements), and every behavioral claim — retry defaults, cache multipliers, reasoning-token billing, org-level spend controls — traces to the linked documentation. The worked cost table and the guard code appear nowhere else.

We deliberately excluded several things you may have seen elsewhere: viral anecdotes about specific customers receiving 10x bills (no primary source exists for any specific public incident); frequency data on retry storms (none is published); claims about built-in per-agent budget caps inside LangChain, LangGraph, AutoGen, or CrewAI (we could not verify them against primary documentation, so we built the cap at the client layer instead, where it is verifiable); and historical price-change series (only the single documented gpt-4o price change is cited). We also did not assert that N subagents multiply cost by N — the sourced figure is Anthropic’s 15x token multiplier for multi-agent systems, which is a different and defensible claim.

📖 Related Reads

  • ToolBrain — tool reviews, LLM comparisons, and AI workflow guides

Cross-links automatically generated from NiteAgent.

← Back to all posts