Source: Red Teaming Language Models to Reduce Harms
Ganguli et al. (Anthropic, Aug 2022): 38,961 human red-team attacks across 2.7B-52B models; RLHF models get harder to red team as they scale.
Contents
- What: research paper and dataset, arXiv:2209.07858
- Authors: Deep Ganguli, Liane Lovitt and 34 others (Anthropic)
- Date: 2022-08-23 (v2 2022-11-22)
- Original: arXiv:2209.07858
Summary
A large manual red-teaming study that also released its attack data, and documented the instructions, process and statistics of the exercise so others could reproduce it.
Key claims
- Released 38,961 human red-team attacks, ranging from offensive language to subtler non-violent unethical outputs.
- Compared four model types at 2.7B, 13B and 52B parameters: plain LM, LM prompted to be helpful/honest/harmless, rejection sampling, and RLHF.
- RLHF models became harder to red team as they scaled; the other three types showed a roughly flat trend.