Agent design patterns

Reusable architectures for arranging one or many agents, and how to choose between them.

Contents

Agent design patterns are reusable architectures for organising an agentic system: how many agents there are, what each does, and who decides what happens next. That last point is the most useful way to tell them apart. In some patterns, code you wrote fixes the order of steps. In others, a model decides at runtime. Model-decided patterns adapt better but cost more calls, tokens and latency. Anthropic draws the same line in its own vocabulary: workflows run LLMs and tools through predefined code paths, while agents direct their own process and tool use, and agentic systems trade latency and cost for task performance (Anthropic).

Choosing a pattern

Start from the requirements, not the pattern:

  1. Task shape. Are the steps known in advance, or open-ended? Does a model need to direct the work?
  2. Latency. Is a fast, interactive response needed, or is a slower, more thorough result acceptable?
  3. Cost. Can the budget support several model calls per request?
  4. Human judgment. Are some decisions high-stakes, safety-critical or subjective?

If the task is predictable or fits in a single model call (summarising, translating, classifying), don’t use an agent at all. Otherwise, start with a single agent and add structure only when a requirement forces it. Revisit the choice as the workload changes.

OpenAI’s guide gives a positive fit test. Agents suit workflows that have resisted deterministic automation: complex decisions with exceptions (refund approval), unwieldy rule sets (vendor security review) and heavy unstructured data (insurance claims). If a use case does not clearly fit, a deterministic solution may be enough (OpenAI guide). Anthropic likewise says to find the simplest solution first, since an optimised single LLM call with retrieval and examples often suffices (Anthropic). Both favour growing incrementally: OpenAI reports that customers succeed more by maximising a single agent before splitting into several than by starting fully autonomous (OpenAI guide).

Synthesis: the guide gives criteria, not a flowchart. This is one way to read them as a decision path:

flowchart TD
  A["New task"] --> B{"Predictable, or one model call?"}
  B -- yes --> N["No agent: call the model directly"]
  B -- no --> C["Start with a single agent"]
  C --> D{"Struggling as tools and responsibilities grow?"}
  D -- no --> K["Keep the single agent"]
  D -- "needs constant re-planning" --> R["ReAct"]
  D -- "steps known in advance" --> W["Fixed workflow: sequential, parallel, loop"]
  D -- "steps depend on the input" --> O["Model-orchestrated: coordinator, hierarchical, swarm"]
  C -. "high-stakes decisions" .-> H["Add human-in-the-loop"]

The patterns

Single agent

One model with a set of tools and a detailed system prompt. It plans and picks tools itself: the plain agent loop.

  • Use for: multi-step tasks needing external data, and as the first version of any agent.
  • Watch for: quality drops (wrong tool choice, failures, latency) as tools and responsibilities grow. That’s the signal to move to ReAct or multi-agent. OpenAI’s advice is the same: get the most out of one agent first (OpenAI guide).
flowchart LR
  U["Request"] --> A["Agent + system prompt"]
  A <--> T["Tools"]
  A --> F["Answer"]

ReAct (reason and act)

See ReAct for the full pattern. A single agent that cycles explicitly through Thought (reason about the next step), Action (call a tool or answer) and Observation (record the result), until it reaches an answer, an iteration limit or an error.

  • Use for: dynamic tasks that need continual re-planning.
  • Trade-off: cheaper than multi-agent, and the reasoning trace helps debugging. But the loop adds latency, quality depends on the model’s reasoning, and one bad tool result can mislead the final answer.
flowchart LR
  Q["Request"] --> T["Thought"] --> A["Action: call a tool"] --> O["Observation"] --> T
  T -- "answer ready" --> F["Final answer"]

Fixed workflows (code decides the order)

These subagent arrangements use predefined control flow with no model orchestration. They’re cheaper, faster and predictable, but rigid. At its smallest, a fixed workflow is prompt chaining: split a task into sequential calls when you need to inspect, log or branch on intermediate outputs, such as Anthropic’s draft → critique against criteria → refine chain (Anthropic prompting; see prompt engineering).

Google’s case for fixed workflows is that model-led routing, scheduling and error handling are slow, costly and variable. In its illustrative refund benchmark (Gemini Flash with mock APIs, not a production measurement), routing with a workflow instead of a free agent cut tokens from 5,152 to 2,265 and latency from 7.2 s to 5.7 s. It also argues a graph narrows prompt-injection impact, because no edge exists for unauthorised actions (Why ADK 2.0).

PatternShapeUse forWatch for
SequentialA → B → C; each output feeds the nextStructured, repeatable pipelines (extract → clean → load)Can’t skip unneeded steps
ParallelFan out to agents running at once, then mergeIndependent sub-tasks, or gathering several views quicklyHigher burst cost; merging conflicting results is hard
LoopRepeat a sequence until an exit conditionPolling, self-correctionA bad exit condition means runaway cost

Sequential

flowchart LR
  I["Input"] --> A["Agent A"] --> B["Agent B"] --> C["Agent C"] --> O["Output"]

Parallel

flowchart LR
  I["Input"] --> A["Agent A"] & B["Agent B"] & C["Agent C"]
  A & B & C --> M["Merge"] --> O["Output"]

Loop

flowchart LR
  I["Input"] --> A["Agent A"] --> B["Agent B"] --> X{"Exit condition met?"}
  X -- no --> A
  X -- yes --> O["Output"]

Two loop variants focus on output quality:

  • Review and critique (generator–critic): a generator drafts and a critic checks the draft against criteria (accuracy, format, safety), then approves it or sends it back. An example is a code generator paired with a security-auditor critic that runs tests. This is verification built into the architecture.
  • Iterative refinement: one or more agents improve a shared result over several passes until it meets a quality bar or hits an iteration limit. Good for drafting, planning and debugging code.

Two reasoning techniques from chain-of-thought are really parallel and search patterns run in your own code. Self-consistency samples N reasoning paths and majority-votes the final answer (gains saturate at about 5–10 paths, and it needs comparable answers such as numbers or options) (Wang et al.). Tree of Thoughts has a model propose and score steps, then searches them with BFS or DFS and backtracking. On Game of 24 with GPT-4 it reached 74% against 4% for CoT, at 5–100x CoT tokens (Yao et al.). See test-time compute.

Review and critique

flowchart LR
  R["Request"] --> G["Generator"] --> D["Draft"] --> C{"Critic: meets criteria?"}
  C -- "no, with feedback" --> G
  C -- yes --> O["Output"]

Dynamic orchestration (a model decides)

PatternShapeUse forWatch for
CoordinatorA model routes each request to specialist agentsVaried inputs to a structured process (e.g. support: status, return, or refund)Extra calls for every routing decision
Hierarchical decompositionA root agent splits the task, sub-agents split further, workers executeAmbiguous, open-ended work (research, planning)Hard to design and debug; nested calls make it slow and costly
SwarmAgents talk all-to-all, critique and hand off, with no central supervisorVery complex problems that benefit from debateThe most complex and costly pattern; needs exit conditions to converge

There are two ways to delegate in these patterns (OpenAI Agents SDK handoffs; see agent handoffs):

  • Handoff (decentralised): the model calls a transfer_to_<agent> tool, and the target agent takes over the conversation, by default with the full history. This is how swarm-style and triage coordinators pass control.
  • Agent as tool (manager): a manager agent calls a specialist like a tool and keeps control of the conversation. This fits coordinators and hierarchical decomposition.

Deep research is the reference hierarchical (orchestrator-worker) system. In Anthropic’s research system, a lead agent plans and spawns parallel subagents, each with its own context window. An Opus 4 lead with Sonnet 4 subagents beat a single Opus 4 agent by 90.2% on Anthropic’s internal eval, and token usage alone explained about 80% of performance variance on BrowseComp. The price is about 15x chat-level tokens (Anthropic; see deep research agents).

Coordinator

flowchart TD
  R["Customer request"] --> C["Coordinator: a model picks the route"]
  C --> A["Order status agent"]
  C --> B["Returns agent"]
  C --> D["Refunds agent"]

Hierarchical decomposition

flowchart TD
  R["Root agent: research project"] --> G["Gather"]
  R --> N["Analyse"]
  R --> W["Write report"]
  G --> G1["Worker"]
  G --> G2["Worker"]
  N --> N1["Worker"]
  N --> N2["Worker"]

Swarm

flowchart LR
  U["Request"] --> D["Dispatcher"]
  D <--> M["Market research"]
  D <--> E["Engineering"]
  D <--> F["Finance"]
  M <--> E
  E <--> F
  F <--> M

Special requirements

  • Human-in-the-loop: the agent pauses at set checkpoints for a person to approve, correct or add input. An example is a compliance officer approving a redacted dataset before release. It’s one of the strongest guardrails, at the cost of building the review system. OpenAI’s guide names two triggers for intervention: exceeding failure thresholds (too many retries, repeated misunderstanding) and high-risk actions (cancellations, large refunds, payments). It also recommends planning for intervention especially early in deployment, to find failures and build the eval cycle (OpenAI guide).

Contradiction: this page, following Google Cloud’s pattern guide, calls human-in-the-loop one of the strongest guardrails. Human-in-the-loop cites Meta’s Rule of Two (“blindly approved” confirmations still fail; HITL does not replace sandboxing or least privilege) and Anthropic’s autonomy study, which recommends designing for visibility and easy intervention rather than approving every action. HITL is strong only when gates are few and reviewers actually read them. Newer sources narrow rather than resolve this: OpenAI’s guide places gates at failure thresholds and high-risk actions, not on every step (OpenAI guide), which is consistent with the few-gates reading.

  • Custom logic: your own code branches between other patterns. For example, a refund agent runs parallel checks, then branches to either a refund or a sequential store-credit flow. This is the most flexible option, but also the most work to build and maintain.

Human-in-the-loop

flowchart LR
  A["Agent works"] --> C{"Checkpoint"} --> H["Human reviews"]
  H -- approve --> N["Agent continues"]
  H -- "correct or add input" --> A

Custom logic (the refund example)

flowchart TD
  Q["Refund request"] --> C["Coordinator: custom code"]
  C --> V["In parallel: verify purchaser, check eligibility"]
  V --> E{"Eligible?"}
  E -- yes --> P["Refund processor"]
  E -- no --> S["Store-credit flow, sequential"]
  P --> F["Response agent"]
  S --> F

Cross-cutting concerns

  • Cost and latency grow with every added agent, loop or cycle. Most patterns trade extra model calls for quality.
  • Exit conditions are mandatory for loop, refinement and swarm patterns: an iteration cap, a time limit or a goal.
  • Context per agent. Isolate it, persist what later steps need, and compress large data. See context engineering.
  • Multi-agent systems fail in their own ways. MAST labelled 1,642 multi-agent traces into 14 failure modes across system design, inter-agent misalignment and task verification. The most common were step repetition (15.7%), reasoning-action mismatch (13.2%), unawareness of termination conditions (12.4%) and disobeying the task spec (11.8%) (MAST). Several of these map to the exit-condition and verification points above. Use error analysis on real traces to find which ones hit your system.
  • Crossing boundaries. When a specialist is a separate service, owned by another team, or written in another language or framework, delegate over Agent2Agent (A2A) rather than a local sub-agent. Keep local sub-agents for internal structure, latency-sensitive paths and shared state, because A2A adds network and serialisation overhead (ADK A2A docs).
  • Synthesis: the patterns compose. Real systems often nest them, for example a coordinator whose specialists run sequential or generator–critic loops. The custom-logic pattern is the general case of that.

Patterns in frameworks

The major frameworks implement these patterns directly:

FrameworkFixed workflowsModel-orchestrated
Google ADK1.x SequentialAgent, ParallelAgent, LoopAgent (stops at max_iterations or escalate); 2.0 graph and dynamic workflows2.0 collaborative workflows: a coordinator with sub_agents in chat, task or single_turn mode (workflows)
CrewAIsequential processhierarchical process: a manager agent allocates and validates tasks, at the cost of extra calls and possible mis-delegation (processes)
Microsoft Agent Framework (successor to AutoGen)Sequential and concurrent orchestrationsHandoff, group chat and Magentic (a manager agent coordinates specialists) (workflows)
LangGraphExplicit nodes and edges, subgraphsConditional routing and Command handoffs; graph=Command.PARENT routes from a subgraph into its parent (graph docs)
OpenAI Agents SDKYour own code around RunnerHandoffs and agents-as-tools (handoffs)

Synthesis: ADK 2.0 graphs, LangGraph and Agent Framework workflows all make the same move, a deterministic skeleton with LLM calls at the leaves, which is the custom-logic pattern turned into a framework feature.

Going deeper

  • Deep research agents: the reference orchestrator-worker (hierarchical) system, with token-cost data.
  • Google ADK: workflow agents (sequential, parallel, loop) and graph-based orchestration as code.
  • CrewAI: sequential and hierarchical processes, and when a single agent is better.
  • AutoGen and Microsoft Agent Framework: sequential, concurrent, handoff, group-chat and Magentic orchestrations.
  • OpenAI Agents SDK: handoffs vs agents-as-tools, two ways to implement the coordinator pattern.
  • LangGraph: fixed workflows and dynamic routing as explicit state graphs.
  • Agent2Agent protocol: delegating to agents across team, vendor or organisation boundaries.
  • Durable agent execution: pausing and resuming long workflows and human checkpoints.
  • Agent evaluation: how to measure whether a more complex pattern actually pays off.
  • Model selection: the model-tier side of the cost and latency trade-off.

Sources

Suggest changes →