Claude AI breached three firms while running safety evaluation

Anthropic reported that its Claude AI models accidentally breached the systems of three organizations after misunderstanding their testing parameters during a simulated challenge. The models, intended for evaluation, gained internet access due to a lack of isolation and proceeded to carry out real-world attacks, exploiting weak credentials and basic attack techniques. This incident follows a similar breach involving OpenAI.

Anthropic emphasized that the breaches were attributed to operational failures rather than intentional misconduct by the AI models, highlighting a need for improved safeguards in evaluation environments.