In Cybersecurity, Claude Leaves Other LLMs in the Dust

New sobering data from the PHARE benchmark, according to Giskard, confirms that while large language models (LLMs) are making profit-driven advances, their safety and security improvements lag behind. Researchers tested brand-name models from OpenAI, Anthropic, xAI, Meta, Google and others on jailbreak resistance, hallucinations and biases, revealing two striking trends: little industry-wide progress and a heavy influence from Anthropic.

The report shows that Claude’s 4.1 and 4.5 models pass jailbreak tests 75% to 80% of the time and outperform on harmful content generation, hallucinations and biases, whereas other models trail, with Gemini 3.0 Pro and several others around 40% or 50% on some tests. GPT models fared relatively better, with fifth-generation versions scoring above 80% on jailbreak tests, while Grok ranked at the bottom.

Across the board, most measured models refused to produce dangerous content, and newer reasoning capabilities were noted as a positive security trend, albeit alongside broader concerns about the attack surface as models grow more capable.