Few-shot prompting

Steering an LLM with a handful of input-output demonstrations in the prompt (in-context learning): how it works, how to pick and format examples, and its biases.

Contents

Few-shot prompting (also “multishot” or in-context learning) means putting a small number of worked input → output examples in the prompt so the model infers the task, format and style from them, with no change to its weights. Zero-shot gives only an instruction; one-shot gives a single example. GPT-3 established the paradigm: a 175B model performed translation, QA and cloze tasks from text demonstrations alone, with no gradient updates (Brown et al.). It is one of the main tools of prompt engineering and the cheapest alternative to fine-tuning.

What the examples actually teach

Min et al. found, on classification and multiple-choice tasks across 12 models, that replacing demonstration labels with random ones barely hurt accuracy. The gains came from showing the label space, the distribution of inputs and the format (Min et al.). Synthesis: for modern instruction-tuned models, examples work mainly as a format and scope specification, not as training data. That is why all three vendors frame them as the most reliable way to lock output format and tone (Anthropic, Google, OpenAI). Synthesis: do not over-read this. It was measured on classification and multiple-choice tasks; for generative tasks the content of the example output is what the model tends to copy (compare Google’s overfitting warning below).

flowchart TD
  A["Instruction"] --> P["Prompt"]
  E1["Example 1: input → output"] --> P
  E2["Example 2: input → output"] --> P
  E3["Example 3 (edge case)"] --> P
  X["New input (fenced)"] --> P
  P --> M["LLM infers label space, input type, format"]
  M --> O["Output in the demonstrated format"]

How many examples, and which

VendorGuidance (as of 2026-11)
Anthropic3–5 examples; relevant to the real use case, diverse enough to avoid accidental patterns; wrap in <example> inside <examples>. You can ask the model to critique or extend your set (Anthropic).
OpenAI (GPT models)Cover a diverse range of inputs; place examples in the developer message (system prompts) (OpenAI).
OpenAI (reasoning models)Start zero-shot; add few-shot only if needed and make examples match the instructions closely (OpenAI reasoning).
Google GeminiAlways include few-shot examples; a few usually suffice; too many risk overfitting; clear examples can replace instructions (Google).
Notebook2–5 examples, used when a pattern or format keeps coming out wrong (Notebook).

Contradiction: Google recommends always including examples, while OpenAI’s reasoning-model guide says to try zero-shot first. Synthesis: the difference tracks model type. Reasoning models infer intent well from goals, and examples can over-constrain their approach, while non-reasoning models benefit more from demonstrations.

Formatting rules

  • Identical structure across examples: same tags, separators, whitespace and newlines. Inconsistent examples produce inconsistent outputs (Google).
  • Separate examples from instructions and from the live input with tags (Anthropic); fence the real input so it can’t inject instructions (Notebook).
  • Cover edge cases, including the “none of the above” or refusal case, so the model sees the full label space (Anthropic, Min et al.).
  • Stable prefix: examples rarely change, so place them early to benefit from prompt caching (OpenAI).

A compact pattern (from the Notebook): a system line defining the one-word label set, three input → label lines, then the real input inside <ticket> tags (Notebook).

Variants

  • Few-shot chain-of-thought: examples include the reasoning steps, not just answers. PaLM 540B with eight such exemplars set the GSM8K state of the art in 2022 (Wei et al.). See chain-of-thought.
  • Examples with thinking models: on Claude, worked examples shaped as problem → method → answer influence how the model reasons in its thinking blocks (Anthropic).
  • Dynamic few-shot: Synthesis: retrieve the most similar labelled examples per input with embeddings instead of a fixed set. This trades cache hits for relevance.

Failure modes

  • Instability. Accuracy can range from near chance to near state of the art depending on which examples are chosen, their order and the format (Zhao et al.).
  • Recency bias. Labels appearing near the end of the prompt are favoured; common-token bias favours answers frequent in pre-training. Contextual calibration (probe with a content-free input like “N/A”, then rescale) gave up to +30 points on GPT-2/3 (Zhao et al.). Synthesis: this requires access to label probabilities, so it is rarely possible on chat APIs that do not expose log-probabilities.
  • Spurious patterns. If every example’s answer is “(A)”, the model drifts toward “(A)”, and its chain of thought will not admit it (Turpin et al.). Balance labels and vary surface features.
  • Over-copying. Too many or too similar examples cause the model to imitate length, phrasing or content (Google, Anthropic).
  • Token cost. Examples take up context window space on every call. Synthesis: caching mitigates the price but not the context budget.

When to use / when not

Synthesis: the table below combines the vendor guidance and papers above; the ~10-case threshold is a rule of thumb, not a sourced figure.

Use few-shot whenPrefer something else when
The format or tone keeps coming out wrongYou need a guaranteed schema: use structured outputs
Labels or categories are subtle and easier to show than to describeThe task needs knowledge the model lacks: use RAG
You are prototyping and have under ~10 labelled casesYou have thousands of examples and high volume: consider fine-tuning
A non-reasoning model needs a demonstrated reasoning styleA reasoning model already does well zero-shot (test first)

Sources

Suggest changes →