GTIG warns AI is aiding state actors and fuelling model theft

The GTIG AI Threat Tracker report highlights an increase in AI integration among cyber threat actors for enhancing their attack methodologies, particularly in reconnaissance, social engineering, and malware development.

Key findings include a rise in model extraction attacks (or 'distillation attacks') aimed at stealing proprietary AI model knowledge, and a noted use of AI tools like large language models (LLMs) by government-backed actors primarily from North Korea, Iran, China, and Russia to streamline operations and develop nuanced phishing lures. The report addresses model extraction attempts and outlines the development of AI-integrated malware, such as HONESTCUE, leveraging Google's Gemini API for malicious purposes.

It also emphasizes the necessity of proactive measures and continuous updates to security models to safeguard against these evolving AI-driven threats. Google's ongoing commitment to thwarting malicious activities includes monitoring API access, enhancing security protocols, and sharing intelligence with the broader community.