Prompt injection is a problem that may never be fixed, warns NCSC

Prompt injection is shaping up to be one of the most stubborn problems in AI security, according to the UK’s National Cyber Security Centre (NCSC), which warns it may never be “fixed” in the same way SQL injection was. The article notes that prompt injection occurs because AI models can’t reliably distinguish between a programme’s instructions and an attacker’s, so they may follow the wrong prompt.

Guardrails are used to stop models from harmful actions, and jailbreaking describes crafting prompts to bypass those safety systems; prompt injection hides instructions inside user input so the model follows them instead of the original guardrails. The danger grows when large language models become autonomous agents able to perform tasks such as moving money, reading email or changing settings, potentially leading to data breaches or fraud.

The NCSC argues that one‑shot fixes are unlikely and urges caution as more organisations bolt generative AI onto sensitive back‑ends. It also offers practical guidance for users and developers, emphasising careful validation of AI outputs and limiting the data and systems that agents can access.