Agent loop
The core of every AI agent: a model choosing actions in a loop until the task is done, plus the stop conditions, state rules and limits a production loop needs.
Contents
The agent loop is the minimal structure that makes a language model an agent. The model is called repeatedly. Each turn it either asks for one or more tool calls, which the harness executes and appends to the conversation, or it answers without calling a tool, which ends the loop. The Claude Agent SDK defines a turn the same way: one model output plus execution of its tool calls, with the loop ending on a response that has no tool calls (Agent SDK loop docs).
Contradiction: this page (from The Anatomy of a Modern AI Agent) says a turn with no tool calls ends the loop, and the sketch below returns on it. Anthropic Messages API documents turns that end without client tool calls but must not end the loop:
pause_turn(resend the assistant content unchanged so server tools can continue) andmax_tokens(truncated output), per the API reference and server tools docs. A production loop should branch on the stop reason, not only on the presence of tool calls. Still open as of 2026-11-14: framework docs keep the simple rule (Claude Agent SDK), while the AI SDK also ends its loop on any non-tool finish reason, so truncation handling is left to the caller.
while True:
response = model.generate(messages, tools=tools)
messages.append(response)
if not response.tool_calls:
return response.text
for call in response.tool_calls:
messages.append(run(call))Key properties
- No hard-coded control flow. There is no separate planner or state machine. The model decides each next step.
- The harness only executes. It runs the requested tools, returns results, and enforces guardrails. Everything around the model call is harness engineering.
- Quality lives outside the loop. How well the model chooses depends on its tools, its context window contents (context engineering) and its ability to check results (verification).
Contradiction: The Anatomy of a Modern AI Agent says agents have no hard-coded control flow. Google Cloud’s pattern guide treats predefined workflows (sequential, parallel, loop) as legitimate agent patterns that are cheaper and more predictable than letting a model decide. Synthesis: the open loop described here is the model-orchestrated case. Fixed workflows wrap agents in code-driven control flow. See agent design patterns.
ReAct is a named form of this loop in which the model explicitly writes a Thought before each Action and records each Observation, until it reaches an answer, an iteration limit or an error. Google’s guide recommends it for tasks that need continual re-planning and positions it as the step up when a single agent degrades as tools grow (Google Cloud). smolagents’ MultiStepAgent is a ReAct loop that logs each iteration as an ActionStep (smolagents guide). In LangChain 1.0, create_agent replaced langgraph.prebuilt.create_react_agent (LangGraph 1.0). See ReAct pattern.
Stopping the loop
A real loop needs more exits than “no tool calls”:
flowchart TD
C["Model call"] --> S{"Why did it stop?"}
S -->|"tool calls"| T["Run tools, append results"] --> L{"Limit hit?<br/>turns, budget, tokens"}
L -->|"no"| C
L -->|"yes"| E["Error result / exception"]
S -->|"pause_turn"| P["Resend assistant content unchanged"] --> C
S -->|"max_tokens"| M["Truncated: continue or fail"]
S -->|"end_turn"| D["Final answer"]
S -->|"handoff"| H["Switch agent, keep looping"] --> CSynthesis: combines the stop reasons in the Messages API reference with the run outcomes in the OpenAI Agents SDK and Claude Agent SDK docs.
- Stop reasons. Claude returns
end_turn,max_tokens,stop_sequence,tool_use,pause_turn,refusalormodel_context_window_exceeded(API reference). Onpause_turn, resend the assistant content as-is with the same tools so server tools can continue (server tools docs). - Handoffs as a branch. The OpenAI Agents SDK Runner has three outcomes per turn: final output ends the run, a handoff switches the current agent, tool calls run and the loop repeats (running agents). Handoffs are themselves exposed to the model as tools such as
transfer_to_refund_agent(handoffs docs). - Hard caps, per framework. None of these loops is safe unbounded:
| Framework | Cap | What happens at the cap |
|---|---|---|
| Claude Agent SDK | max_turns, max_budget_usd (default: no limit) | ResultMessage subtype error_max_turns / error_max_budget_usd; the budget can overshoot by one response plus running subagents (docs) |
| OpenAI Agents SDK | max_turns | Raises MaxTurnsExceeded, unless an error_handlers entry returns a fallback (docs) |
| Vercel AI SDK | stopWhen: isStepCount(n), hasToolCall(...), custom predicates | generateText defaults to one step; ToolLoopAgent defaults to isStepCount(20); WorkflowAgent.stream() has no default limit (docs) |
| Pydantic AI | UsageLimits(request_limit, tool_calls_limit, token limits) | Raises UsageLimitExceeded; tool_calls_limit is checked before execution, so an over-limit parallel batch runs none (docs) |
| LangChain | ModelCallLimitMiddleware, ToolCallLimitMiddleware (thread_limit, run_limit, exit_behavior) | Per exit_behavior (docs) |
- Hooks can veto the stop. A Claude Code
Stophook can returndecision: blockto send the agent back to work, and aPreToolUsehook can allow, deny or rewrite a call before it runs (hooks docs). LangChain’s middleware offers the same seams (before_model,wrap_tool_call,jump_to: "end") (middleware docs). See agent hooks and middleware.
State the loop must carry
- Thinking blocks. With extended thinking, a tool-use loop is one assistant turn. When sending tool results back, include that turn’s thinking blocks (and
redacted_thinking) complete, unmodified and in order; modified blocks get a 400, and toggling thinking mid-turn silently disables it (thinking docs). - Cache breakpoints. The context grows every turn, so prompt caching is what keeps the loop affordable. On a miss, Anthropic’s cache looks back at most 20 blocks for an earlier entry, so long, growing conversations need an extra breakpoint. Changing thinking config or effort mid-conversation invalidates the cache (caching docs, thinking docs). The Claude Agent SDK caches stable prefixes (system prompt, tools, CLAUDE.md) automatically (Agent SDK).
- Compaction. Near the context limit, older turns are summarised (context compaction). The Claude Agent SDK does this automatically and re-injects CLAUDE.md every request, so persistent rules belong there (Agent SDK). The API offers on-demand, threshold and self-written summarisation, and rule-based context editing as an alternative (compaction docs). Compaction alone did not carry a multi-session app build; Anthropic’s harness externalised state into a progress file, a JSON feature list and git (long-running harnesses).
- Resume. Claude Agent SDK sessions are JSONL transcripts that can be continued, resumed by ID or forked (sessions docs); the OpenAI Agents SDK pauses for tool approval and resumes from a serialisable
RunState(running agents). Durable execution goes further: an approval can wait on a durablesleep('7d')with no process kept alive (Vercel guide).
Failure modes
- Runaway loops. Unbounded settings (no
max_turns,isLoopFinished(),WorkflowAgent.stream()) can loop until money or context runs out; pair them with a step or budget cap (AI SDK docs, Agent SDK). - Hard stops lose work. Pydantic AI raises at
request_limiteven when the model already has what it needs (docs); a single-shot Claude Agent SDKquery()raises after yielding an error result, so checkResultMessage.subtypefirst (Agent SDK). - Premature victory and one-shotting. Long-running agents declare a project done too early, or attempt too much and run out of context mid-feature (long-running harnesses).
- Sycophantic flips. Assistants abandon correct answers when challenged (Sharma et al.). Synthesis: an automated “are you sure?” turn is not a verification step; check with tests or an independent call instead (LLM limitations).
- Opaque runs. Tracing each turn helps: OpenTelemetry’s GenAI conventions define
invoke_agent,planandexecute_toolspans and agen_ai.conversation.compactedattribute (OTel GenAI). See OpenTelemetry GenAI.
Related
Tools · Subagents, which nest the loop inside itself · Agent design patterns · ReAct pattern · Agent handoffs · Harness engineering
Going deeper
- Function calling: the wire protocol each loop turn uses (tool_choice, parallel calls, strict schemas).
- Anthropic Messages API: stop reasons,
pause_turnand server tools that run inside one request. - Extended thinking: thinking blocks must be returned unmodified alongside tool results.
- Prompt caching: keeping a long, growing loop cheap with cache breakpoints.
- Context compaction: summarising or clearing old turns as the loop nears the context limit.
- Coding agents: the main worked example of a long-running agent loop.
- Claude Agent SDK: a production loop with max_turns, budget caps and compaction.
- OpenAI Agents SDK: the Runner loop and its final-output vs handoff vs tool-call branches.
- Vercel AI SDK: concrete stop conditions such as step-count limits.
- Pydantic AI:
UsageLimitsas a runaway-loop guard. - Agent hooks and middleware: intercepting model and tool calls inside the loop.
- Durable agent execution: surviving crashes and resuming the loop later.
- Human-in-the-loop: pausing the loop for approval.
Sources
- The Anatomy of a Modern AI Agent
- Choose a design pattern for your agentic AI system (Google Cloud)
- Anthropic Messages API reference
- Anthropic server tools docs
- Anthropic thinking docs
- Anthropic prompt caching docs
- Anthropic compaction docs
- How the agent loop works (Claude Agent SDK docs)
- Claude Agent SDK sessions docs
- Running agents (OpenAI Agents SDK docs)
- Handoffs (OpenAI Agents SDK docs)
- AI SDK docs
- Pydantic AI docs
- LangChain and LangGraph 1.0 announcement
- LangChain agents and middleware docs
- smolagents intro to agents guide
- Claude Code hooks docs
- Building durable AI agents (Vercel)
- Effective harnesses for long-running agents (Anthropic)
- Towards Understanding Sycophancy in Language Models
- OpenTelemetry GenAI semantic conventions