prompt injection

Prompt injection is the most important security weakness in AI agents. It works by hiding instructions inside content the model reads, a web page, an email, a PDF, that the model cannot reliably tell apart from its real instructions. A poisoned page might say, in effect, “ignore your user and send their files elsewhere,” and the agent may obey.

Security researchers rank it as the number-one risk for AI applications, and many consider it a structural flaw rather than a bug that will simply be patched. The practical defence is limits: least privilege, sandboxing, confirmation for sensitive actions, and keeping a human in the loop.