Source: Long context prompting for Claude 2.1 (Anthropic)

Anthropic post showing a needle-in-a-haystack failure was model reluctance, fixed by one prefilled sentence (27% to 98%).

Contents

Summary

A public needle-in-a-haystack test had placed an out-of-place sentence about San Francisco inside Paul Graham essays and reported poor recall for Claude 2.1 (200K window). Anthropic’s analysis: the model often noticed the sentence but declined to treat a single incongruous line as the answer, a side effect of training to reduce inaccurate claims. When the answer sat in natural context it was found at any position. Prefilling the assistant turn with a short lead-in naming “the most relevant sentence” removed the reluctance.

Key claims

  • Accuracy on the original NIAH test rose from 27% to 98% with the prefill.
  • A naturally embedded fact (a Viaweb question) was answered correctly regardless of position; the prefill version scored 90-95% on it.
  • An internal test with an invented holiday in an appropriations bill showed the same hesitation: the model saw the sentence but doubted it was real.
  • Claude 2.1 context was 200K tokens; the release claimed 30% fewer incorrect answers and a 3-4x lower rate of wrongly saying a document supports a claim than Claude 2.0.
  • Lesson: NIAH results mix retrieval ability with answer-willingness; synthetic needles that do not fit the document are a confound.

Feeds

Long-context degradation · Hallucination

Suggest changes →