Prompt Injection Attacks Are Thwarting AI Hacking Agents
Prompt injections, the malicious commands attackers embed into content to entice large language models to follow them, have been attackers’ go-to tool for...
Key points
- Prompt injections, the malicious commands attackers embed into content to entice large language models to follow them, have been attackers’ go-to tool for turning AI platforms against their users.
- Now, defenders are embracing the prompt injection, too.
- Researchers from Tracebit on Monday said they found that placing prompt injections alongside passwords, cryptographic keys, and other secrets stored on Amazon Web Services was often all that was needed to shut down attacks from AI hacking agents.
- The researchers have named the technique context bombing.
What happened
More photos
Prompt injections, the malicious commands attackers embed into content to entice large language models to follow them, have been attackers’ go-to tool for turning AI platforms against their users. Now, defenders are embracing the prompt injection, too. Researchers from Tracebit on Monday said they found that placing prompt injections alongside passwords, cryptographic keys, and other secrets stored on Amazon Web Services was often all that was needed to shut down attacks from AI hacking agents. The researchers have named the technique context bombing. “The most capable agent in our tests, Opus 4.8, went from achieving admin access in 93 percent of runs to failing every single time when confronted with a context bomb.” Averaged across the five models and the 152 runs, the results included: - Admin privilege escalation fell from 57 percent to 5 percent - Admin escalation with a persistent foothold fell from 36 percent to 1 percent - Runs achieving any attack path fell from 91 percent to 15 percent - On average, a run went from completing 1.53 paths successfully to just 0.16 - No runs were able to complete an attack path without at least triggering a canary detection The research...