Bharat Bhavnasi San Francisco, CA, USA

The Cheapest Agent Upgrade Is a Stop Condition

• 5 min read
#agents #cost #reliability #orchestration #infrastructure

The most useful change you can make to a production agent this quarter is not a better model or a cleverer prompt. It is a number: the maximum number of loops the agent is allowed to run before it stops or escalates. The recent data is unambiguous that restraint is underpriced — teams keep reaching for capability when the cheaper, more reliable win is bounding the loop they already have.

That sounds like a cost story, and it is partly one. But the more interesting finding underneath it is that the loops you cut were mostly not buying you correctness. They were buying you tokens.

The loop is the cost, not the model

Glean’s token-efficiency writeup (June 3) collects the numbers that make this concrete. An agent makes 3–10x more model calls than a chatbot for a single request — plan, select a tool, execute, verify, respond. And because each call carries the full transcript, a session that opens at 5,000 input tokens can be shipping 80,000+ tokens by step 20, a point Stevens’ hidden-economics analysis makes the same way. Cost does not scale with the work; it scales with the conversation length, which grows roughly with the square of the loop count.

This is why per-token optimization, while real, has a ceiling. Prompt caching cuts input costs 41–80% on agentic workloads. Distillation and routing help. Trajectory compression — squeezing observations and actions before they re-enter context — helps more. All of those make each step cheaper. None of them make the agent take fewer steps. A 30-step run that should have been a 6-step run is still a 30-step run, just at a discount. The Reflexion pattern is the cautionary case: a 10-cycle self-reflection loop can consume roughly 50x the tokens of a single pass because the context grows quadratically with each cycle. Caching the input does not save you from the fact that you ran the loop ten times.

Bounding beats unleashing — at comparable quality

The phrase that should change how you build is from the Stevens work: agents with dynamic turn limits cost 24% less than unconstrained ones, with comparable output quality. Read that twice. The constrained agents were not worse. The extra turns the unconstrained agents took were, in aggregate, not improving the answer — they were re-deriving context, second-guessing, and padding the bill.

That reframes the turn limit from a crude safety valve into a quality-neutral default. The practical pattern has three parts:

  • A hard cap on review/revision cycles, with an escalation path when the cap is hit instead of a silent truncation.
  • A delta threshold: measure whether correctness or groundedness actually improved after each iteration, and stop when the improvement falls below a floor. Most of the time the second revision earns its keep and the fifth does not.
  • A session token budget — a contract on total input-plus-output the agent may consume in a window — so a misbehaving loop fails loudly and early rather than after it has spent $8.

None of this is glamorous, and that is the point. You are not making the agent smarter. You are refusing to pay for the part of its thinking that was never load-bearing.

Why this matters more when you run a fleet

A single bounded agent saves you a quarter of your bill. The reason restraint is suddenly a front-page concern is that nobody runs a single agent anymore. The dev-tools surface that trended in late June is almost entirely about running many agents at once. Orca is an “agent development environment” for driving a fleet of coding agents in parallel, each in its own git worktree, across whatever backend you bring. GStack packages a Claude Code setup as 23 opinionated tools that play org roles — CEO, eng manager, release manager, QA — so you can stand up a whole synthetic team with one install.

In that world an unbounded agent is not a 8problem;itisa8 problem; it is a 8 problem times ten, running concurrently, with the failure modes correlated. The per-agent turn limit and budget stop being a cost knob and become the unit of operational safety. A fleet you cannot bound is a fleet you cannot run in production, regardless of how good any single agent is. The orchestration layer’s first job is not coordination — it is admission control.

There is a darker corollary worth keeping in view: oversight is not free either. Adding verifier and critic loops to catch bad actions also adds loops, and recent work on tool-using agents has found that heavy safety mediation can block the overwhelming majority of non-compliant actions while still rarely producing a safe successful completion — you pay the loop cost for the guardrail and do not get the outcome. More verification is not automatically net-positive. The same discipline applies: bound it, measure whether it changed the result, and stop when it stops paying.

The throughline across all of this is the least fashionable instinct in a field obsessed with capability. The agent does not need to be able to do more. It needs to know when to stop.

Worth bookmarking

Share:
Suggest changes