THE article discusses a new cybersecurity threat called "HalluSquatting," which exploits vulnerabilities in large language models (LLMs) used in AI coding assistants. Prompt injections have become a significant threat, as LLMs struggle to distinguish between legitimate instructions and malicious ones. HalluSquatting allows attackers to register resource identifiers that LLMs are likely to hallucinate, enabling mass infections and the assembly of large botnets.
The flaw arises because LLMs frequently hallucinate the locations of resources, especially trending ones that are not included in their training data. This makes it easy for attackers to embed harmful instructions in these resources, leading to risks like widespread ransomware attacks or large-scale DDoS attacks. The researchers emphasize the importance of recognizing these vulnerabilities to enhance security measures in AI applications.