Prompt injection
‘Prompt injection’ happens when an AI system confuses data and instructions. This is especially dangerous with emerging tools that have access to users’ computers – as with the new ‘computer use’ ability debuted by Anthropic’s Claude.
As an example, if you ask an AI system to summarise a PDF file, and the data in the document contains a malicious instruction, the AI could execute that instruction on a system level – with devastating consequences.