PROMPT injections are malicious commands embedded by attackers into content, exploiting AI platforms to turn them against users. Researchers from Tracebit have found that using a technique called 'context bombing' can effectively disrupt AI hacking efforts. This involves placing prompt injections alongside stored secrets to trigger LLMs to refuse harmful commands, effectively shutting down the attacks.
Initial tests showed that context bombing significantly reduced successful admin privilege escalations from 57% to 5% and full compromise from 36% to 1% across multiple AI models. The approach is a novel defensive strategy against the common threat of prompt injections, which attackers have been using to circumvent AI defenses. Tracebit emphasizes the importance of this method in preventing AI-assisted attacks, marking a shift in how defenses are approached in the AI security landscape.