ZSCALER ThreatLabz has documented two campaigns demonstrating how hidden prompts on malicious websites trick AI agents into making fraudulent payments. The first campaign involves a fake Python library that manipulates AI coding assistants into paying a $3.00 fee, employing hidden instructions in HTML and JSON-LD metadata to facilitate this.
The second campaign engages in typosquatting, misleading AI agents into recognizing a fraudulent site as legitimate by falsifying its association with the legitimate DeBank service. Testing revealed significant vulnerabilities where multiple AI models failed to identify the scams accurately. This highlights the imperative for better input validation for AI agents and stricter controls on payment capabilities in automated workflows, revealing the dual nature of AI as both a beneficial and exploitable tool.