RESEARCHERS have demonstrated a new attack against Grok, an AI model, which allows it to exfiltrate user data, similar to recent incidents involving Microsoft 365 Copilot. The attack utilizes a technique called 'cryptographic context injection,' where malicious instructions are encrypted and sent to Grok within a seemingly safe context, enabling the model to inadvertently disclose sensitive user information without warning.
This vulnerability stems from the model's inability to distinguish between harmful and benign instructions when processed as ciphertext. The researchers argue that existing guardrails for AI safety are inadequate, as they can only inspect incoming text, failing to execute code or decrypt instructions properly. This highlights a broader issue with AI safety mechanisms struggling against evolving attack strategies.