System prompts

The privileged, operator-level instructions that frame a conversation: how vendors implement them (system, developer, instructions), their precedence over user turns, what belongs in them, and their limits.

Contents

A system prompt is the block of instructions an application developer (the “operator”) sends with every request, separate from the end user’s messages, to set the model’s role, rules, output conventions and context for the whole conversation. Models are trained to give it more authority than user turns, so it is where an application encodes behaviour the user should not be able to override (OpenAI Model Spec, Anthropic mid-conversation). It is the persistent layer of prompt engineering and the first item of an agent’s context (context engineering).

Vendor implementations (as of 2026-11)

Vendor / APIFieldNotes
Anthropic Messages APItop-level system (string or blocks)Plus role: "system" messages inside messages on supported models (Anthropic mid-conversation)
OpenAI Responses APIdeveloper role messages, or the instructions parameterinstructions applies to one request only and is not carried over via previous_response_id (OpenAI guide). Reasoning models since o1-2024-12-17 take developer messages instead of system messages (OpenAI reasoning)
OpenAI “system”Platform-level, written by OpenAIAbove developer; may be invisible to developers (Model Spec)
Google Geminisystem instructionFor persistent behaviour: persona, verbosity, date and knowledge cutoff, grounding rules (Google)

Precedence: the instruction hierarchy

OpenAI’s Model Spec gives the most explicit ordering: root (the spec itself) > system (platform) > developer > user > guideline (soft defaults). Assistant messages, tool outputs, quoted text, files and images have no authority by default; instructions inside them are information unless a higher-level instruction delegates authority. Conflicts resolve by discarding instructions that clash with a higher level or are superseded by a later same-level one; then higher authority and recency win (Model Spec). Anthropic’s API treats system messages as operator-level priority and user messages as end-user input (Anthropic mid-conversation).

flowchart TD
  R["Platform / vendor policy"] --> D["System / developer prompt (operator)"]
  D --> U["User messages"]
  U --> T["Tool results, retrieved docs, files: data, no authority"]
  T -. "embedded instructions are ignored unless delegated" .-> U

This hierarchy is learned, not enforced by the runtime. OpenAI’s Instruction Hierarchy paper argues that injections and jailbreaks succeed because models gave system prompts and untrusted text equal weight. Training GPT-3.5 on synthetic conflicts to ignore lower-privileged instructions substantially improved robustness, including against unseen attacks (Wallace et al.). Anthropic reports that Sonnet 5.5 and Haiku 5.5 are trained to resist injection via tool results, to the point that they may ignore a system message placed directly after a tool result (Anthropic mid-conversation).

What goes where

Put in the system promptPut in the user turn
Role/identity, tone, audienceThe specific task instance and its variable input
Hard rules and policies, refusal and escalation behaviourEnd-user questions and uploaded content
Output conventions (format, length, language)Per-request documents (long ones at the top of the turn, query last)
Tool-use policy, stable few-shot examplesContinuations of interrupted replies
Current date, model identity, knowledge cutoff

Sources for the table: role in system (Anthropic); identity → instructions → examples → context layout (OpenAI guide); date, cutoff and grounding rules (Google); role, constraints and format in the system instruction or at the start for Gemini 3 (Google); continuations in the user turn (Anthropic). Synthesis: the dividing line is stable and privileged (system) vs per-request and untrusted (user).

Role prompting. Anthropic says one sentence of role in system noticeably focuses behaviour and tone (Anthropic). The Notebook lists role/persona as a standard technique for expertise and tone (Notebook).

Changing instructions mid-conversation

Editing the top-level system field changes the start of the prompt and invalidates the prompt cache for everything after it. On many current Claude models it also invalidates earlier thinking blocks (Anthropic best practices). Anthropic’s mid-conversation system messages append a role: "system" message instead (Anthropic mid-conversation):

  • It applies from its position onward. Later system messages override earlier ones and the top-level field.
  • It must follow a user turn and cannot sit between tool_use and tool_result (400 error).
  • Betas: add or remove tools mid-conversation (tool_addition / tool_removal), turn-scoped reminders (clear_at: "next_user_message"), per-message effort.
  • Not available on Claude Sonnet 5. Use the top-level field for whole-conversation rules, system messages for later or frequently changing rules (mode switches, state changes, budget reminders), and user messages for anything from the end user.

OpenAI’s per-request instructions gives a similar “swap the rules this turn” ability, but you must resend it each request (OpenAI guide).

Writing guidance

  • Calm, specific language. Newer Claude models are more responsive to the system prompt; “CRITICAL: you MUST…” written for older models causes over-triggering (Anthropic). Google warns against overly persuasive language with Gemini 3 (Google).
  • Explain the reasons for rules so the model generalises them (Anthropic).
  • Structure with headers or XML sections. OpenAI uses Markdown headers for sections and XML for documents (OpenAI guide); Google says to choose one format and use it consistently within a prompt for Gemini 3 (Google).
  • Format toggles: OpenAI reasoning models avoid markdown unless the developer message starts with “Formatting re-enabled” (OpenAI reasoning).
  • Version it as code, pinned to a model snapshot and covered by evals (OpenAI guide).
  • Keep it stable and first for cache hits (OpenAI guide).

Limitations and failure modes

  • Not a security boundary. Priority is a trained tendency, so a determined prompt injection or jailbreak can still win (Wallace et al.). Synthesis: enforce permissions in code and tooling (agent guardrails), and treat the system prompt as policy guidance, not access control.
  • Not secret. Synthesis: assume users can extract it, and never put credentials or sensitive logic in it.
  • Bloat. Synthesis: long, accreted system prompts cost tokens on every call and can trigger unwanted behaviour. Anthropic notes that large, complex system prompts can make adaptive thinking fire more often (Anthropic).
  • Drift across models. A prompt tuned for one model can over- or under-steer the next. Re-evaluate on upgrades (Anthropic).

Sources

Suggest changes →