A recent study by Barracuda Networks highlights the risks of compromised email accounts leveraging built-in AI assistants. Once an account is hacked, attackers can use the AI chatbot to perform actions undetected, such as creating rules to delete trace emails and conducting internal phishing attacks on high-level targets, like CEOs.
Through a simulated attack, the researchers demonstrated how a hacker could escalate their privileges undetected, eventually manipulating email conversations to authorize fraudulent wire transfers. This method poses a severe threat as it capitalizes on the AI's capabilities to craft convincing communications, making traditional security measures ineffective against such tailored phishing attacks.