Loop termination and runtime budget gates
An agent in production entered a tool-call loop and spent $4,000 in nine hours before anyone noticed. Design the controls so this cannot recur, without making the agent give up on legitimately long tasks.
The constraint. Observability is not control. Langfuse or LangSmith will show you the loop after the fact; neither stops it. The gate has to be in the execution path, and it has to be enforced by the runtime rather than requested in the prompt.
Layered controls, cheapest first.
| Layer | Control | Trips on |
|---|---|---|
| Per-step | Wall-clock and token cap per model call | Runaway single generation |
| Per-run | Cumulative token budget, iteration ceiling | The $4,000 case |
| Per-run | Repeated-state detector | Identical tool call with identical arguments twice |
| Per-tenant | Rolling spend limit over a time window | Many small runs, one abusive tenant |
| Global | Circuit breaker on aggregate spend rate | Provider pricing change, prompt regression |
The repeated-state detector is the specific fix. Hash each tool call — name plus normalized arguments. If the same hash appears twice in one run, the agent is not making progress. Second occurrence: inject an observation naming the repetition. Third: terminate. This catches the actual failure mode, which is rarely "too many steps" and almost always "the same step forever."
Degrade, do not just kill. A hard kill at the ceiling loses all work. Better: at 80% of budget, switch the system prompt to a summarization instruction and force a final answer with what has been gathered. At 100%, terminate and escalate with the partial state attached. The user gets something, and the state is inspectable.
Where the gate lives. In the gateway or the orchestrator, never in the agent. An agent asked to respect its own budget will exceed it — the instruction competes with the task, and the task wins. Enforce in middleware that counts tokens on every request and can refuse.
Stack. LiteLLM as the gateway for normalized token accounting and per-key budgets across providers. LangGraph's recursion_limit for the crude iteration ceiling, plus a custom reducer for the repeated-state hash. Temporal if you want the budget as workflow state that survives process death. Langfuse for the trace that tells you which prompt change caused it — after the gate has already stopped the bleeding.
Where answers fail. Answering "we monitor with Langfuse and set alerts." An alert at 3am is not a control.