Agent loop

The core of every AI agent: a model choosing actions in a loop until the task is done, plus the stop conditions, state rules and limits a production loop needs.

Contents

The agent loop is the minimal structure that makes a language model an agent. The model is called repeatedly. Each turn it either asks for one or more tool calls, which the harness executes and appends to the conversation, or it answers without calling a tool, which ends the loop. The Claude Agent SDK defines a turn the same way: one model output plus execution of its tool calls, with the loop ending on a response that has no tool calls (Agent SDK loop docs).

Contradiction: this page (from The Anatomy of a Modern AI Agent) says a turn with no tool calls ends the loop, and the sketch below returns on it. Anthropic Messages API documents turns that end without client tool calls but must not end the loop: pause_turn (resend the assistant content unchanged so server tools can continue) and max_tokens (truncated output), per the API reference and server tools docs. A production loop should branch on the stop reason, not only on the presence of tool calls. Still open as of 2026-11-14: framework docs keep the simple rule (Claude Agent SDK), while the AI SDK also ends its loop on any non-tool finish reason, so truncation handling is left to the caller.

python
while True:
    response = model.generate(messages, tools=tools)
    messages.append(response)
    if not response.tool_calls:
        return response.text
    for call in response.tool_calls:
        messages.append(run(call))

Key properties

  • No hard-coded control flow. There is no separate planner or state machine. The model decides each next step.
  • The harness only executes. It runs the requested tools, returns results, and enforces guardrails. Everything around the model call is harness engineering.
  • Quality lives outside the loop. How well the model chooses depends on its tools, its context window contents (context engineering) and its ability to check results (verification).

Contradiction: The Anatomy of a Modern AI Agent says agents have no hard-coded control flow. Google Cloud’s pattern guide treats predefined workflows (sequential, parallel, loop) as legitimate agent patterns that are cheaper and more predictable than letting a model decide. Synthesis: the open loop described here is the model-orchestrated case. Fixed workflows wrap agents in code-driven control flow. See agent design patterns.

ReAct is a named form of this loop in which the model explicitly writes a Thought before each Action and records each Observation, until it reaches an answer, an iteration limit or an error. Google’s guide recommends it for tasks that need continual re-planning and positions it as the step up when a single agent degrades as tools grow (Google Cloud). smolagents’ MultiStepAgent is a ReAct loop that logs each iteration as an ActionStep (smolagents guide). In LangChain 1.0, create_agent replaced langgraph.prebuilt.create_react_agent (LangGraph 1.0). See ReAct pattern.

Stopping the loop

A real loop needs more exits than “no tool calls”:

flowchart TD
    C["Model call"] --> S{"Why did it stop?"}
    S -->|"tool calls"| T["Run tools, append results"] --> L{"Limit hit?<br/>turns, budget, tokens"}
    L -->|"no"| C
    L -->|"yes"| E["Error result / exception"]
    S -->|"pause_turn"| P["Resend assistant content unchanged"] --> C
    S -->|"max_tokens"| M["Truncated: continue or fail"]
    S -->|"end_turn"| D["Final answer"]
    S -->|"handoff"| H["Switch agent, keep looping"] --> C

Synthesis: combines the stop reasons in the Messages API reference with the run outcomes in the OpenAI Agents SDK and Claude Agent SDK docs.

  • Stop reasons. Claude returns end_turn, max_tokens, stop_sequence, tool_use, pause_turn, refusal or model_context_window_exceeded (API reference). On pause_turn, resend the assistant content as-is with the same tools so server tools can continue (server tools docs).
  • Handoffs as a branch. The OpenAI Agents SDK Runner has three outcomes per turn: final output ends the run, a handoff switches the current agent, tool calls run and the loop repeats (running agents). Handoffs are themselves exposed to the model as tools such as transfer_to_refund_agent (handoffs docs).
  • Hard caps, per framework. None of these loops is safe unbounded:
FrameworkCapWhat happens at the cap
Claude Agent SDKmax_turns, max_budget_usd (default: no limit)ResultMessage subtype error_max_turns / error_max_budget_usd; the budget can overshoot by one response plus running subagents (docs)
OpenAI Agents SDKmax_turnsRaises MaxTurnsExceeded, unless an error_handlers entry returns a fallback (docs)
Vercel AI SDKstopWhen: isStepCount(n), hasToolCall(...), custom predicatesgenerateText defaults to one step; ToolLoopAgent defaults to isStepCount(20); WorkflowAgent.stream() has no default limit (docs)
Pydantic AIUsageLimits(request_limit, tool_calls_limit, token limits)Raises UsageLimitExceeded; tool_calls_limit is checked before execution, so an over-limit parallel batch runs none (docs)
LangChainModelCallLimitMiddleware, ToolCallLimitMiddleware (thread_limit, run_limit, exit_behavior)Per exit_behavior (docs)
  • Hooks can veto the stop. A Claude Code Stop hook can return decision: block to send the agent back to work, and a PreToolUse hook can allow, deny or rewrite a call before it runs (hooks docs). LangChain’s middleware offers the same seams (before_model, wrap_tool_call, jump_to: "end") (middleware docs). See agent hooks and middleware.

State the loop must carry

  • Thinking blocks. With extended thinking, a tool-use loop is one assistant turn. When sending tool results back, include that turn’s thinking blocks (and redacted_thinking) complete, unmodified and in order; modified blocks get a 400, and toggling thinking mid-turn silently disables it (thinking docs).
  • Cache breakpoints. The context grows every turn, so prompt caching is what keeps the loop affordable. On a miss, Anthropic’s cache looks back at most 20 blocks for an earlier entry, so long, growing conversations need an extra breakpoint. Changing thinking config or effort mid-conversation invalidates the cache (caching docs, thinking docs). The Claude Agent SDK caches stable prefixes (system prompt, tools, CLAUDE.md) automatically (Agent SDK).
  • Compaction. Near the context limit, older turns are summarised (context compaction). The Claude Agent SDK does this automatically and re-injects CLAUDE.md every request, so persistent rules belong there (Agent SDK). The API offers on-demand, threshold and self-written summarisation, and rule-based context editing as an alternative (compaction docs). Compaction alone did not carry a multi-session app build; Anthropic’s harness externalised state into a progress file, a JSON feature list and git (long-running harnesses).
  • Resume. Claude Agent SDK sessions are JSONL transcripts that can be continued, resumed by ID or forked (sessions docs); the OpenAI Agents SDK pauses for tool approval and resumes from a serialisable RunState (running agents). Durable execution goes further: an approval can wait on a durable sleep('7d') with no process kept alive (Vercel guide).

Failure modes

  • Runaway loops. Unbounded settings (no max_turns, isLoopFinished(), WorkflowAgent.stream()) can loop until money or context runs out; pair them with a step or budget cap (AI SDK docs, Agent SDK).
  • Hard stops lose work. Pydantic AI raises at request_limit even when the model already has what it needs (docs); a single-shot Claude Agent SDK query() raises after yielding an error result, so check ResultMessage.subtype first (Agent SDK).
  • Premature victory and one-shotting. Long-running agents declare a project done too early, or attempt too much and run out of context mid-feature (long-running harnesses).
  • Sycophantic flips. Assistants abandon correct answers when challenged (Sharma et al.). Synthesis: an automated “are you sure?” turn is not a verification step; check with tests or an independent call instead (LLM limitations).
  • Opaque runs. Tracing each turn helps: OpenTelemetry’s GenAI conventions define invoke_agent, plan and execute_tool spans and a gen_ai.conversation.compacted attribute (OTel GenAI). See OpenTelemetry GenAI.

Tools · Subagents, which nest the loop inside itself · Agent design patterns · ReAct pattern · Agent handoffs · Harness engineering

Going deeper

Sources

Suggest changes →