It Takes Only 250 Documents to Poison Any AI Model

Researchers fromAnthropic, the UK AI Security Institute, and the Alan Turing Institute found that they could backdoor AI models of different sizes using just 250 poisoned documents, upending the idea that attackers must control large proportions of training data. For the study, models ranging from 600 million parameters to 13 billion parameters were trained with either 100, 250, or 500 malicious documents, and 250 was sufficient to compromise models across all sizes.

According to Anthropic, injecting 250 malicious documents into pretraining data can enable backdoors in models from 600 million to 13 billion parameters. The researchers note that a 600-million-parameter model typically uses about 600 million tokens of training data, roughly equivalent to 4 million books, while a 13-billion-parameter model would require about 22 times as much data—around 90 million books—yet still could be poisoned with a single fixed number of documents. The findings challenge assumptions about data poisoning and highlight that even tiny amounts of malicious data can have outsized effects.