Source: Extracting Training Data from Large Language Models

Carlini et al. (2020/2021): querying GPT-2 recovered hundreds of verbatim training sequences, including names, phone numbers and emails; larger models leak more.

Contents
  • What: research paper, arXiv:2012.07805
  • Authors: Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Úlfar Erlingsson, Alina Oprea, Colin Raffel
  • Date: 2020-12-14 (v2 2021-06-15)
  • Original: arXiv:2012.07805

Summary

Shows that an adversary with only query access can recover individual training examples from a language model, establishing memorisation as a privacy risk rather than a theoretical one.

Key claims

  • Hundreds of verbatim sequences extracted from GPT-2: public PII (names, phone numbers, emails), IRC logs, code, 128-bit UUIDs.
  • Extraction worked even for sequences that appeared in a single training document.
  • Larger models are more vulnerable than smaller ones.

Feeds

LLM security risks

Suggest changes →