Contain Prompt-Injection Damage With Defense in Depth

Summary

The article recommends treating prompt injection as a likely compromise and limiting its impact with least privilege, credential isolation, sandboxing, output checks, human approval, monitoring and session isolation, citing OWASP guidance and incident examples.

Key points

The article cites OWASP’s 2025 LLM guidance that foolproof prompt-injection prevention may not be possible, and recommends designing for containment.
It distinguishes probabilistic controls, such as filters and classifiers, from deterministic boundaries that limit what a compromised model can do.
Keep credentials outside the model’s context and give integrations only the permissions they need.
Sandbox tool execution, isolate sessions and agents, and use output validation and human approval for consequential actions.
Monitor model tool calls and output patterns; the article argues monitoring should focus on application-layer behavior.
The article describes Vanna.AI CVE-2024-5565 as an example where model-generated code reached Python exec() without an adequate boundary.
It reports that Anthropic’s Constitutional Classifiers reduced automated jailbreak success from 86% to 4.4%, but a later public bounty found a universal bypass.

Article Details

Defense Focus: Contain prompt-injection blast radius by treating the model as untrusted and enforcing deterministic controls outside it.
Detection Methods: Baseline normal model behavior and alert on deviations.
Monitor tool calls and output patterns at the application layer.
Test what an injected prompt can reach after it succeeds, rather than testing only whether injection is possible.
Data Sources: Model behavior baselines
Tool-call activity
Model output and output-rendering activity
Platforms: LLM applications
MCP servers
Isolated container environments for tool execution
Defensive Actions: Label system prompts as trusted and user input, retrieved content, and tool output as untrusted; use those labels to gate tool calls.
Apply least privilege to API tokens, database connections, and integrations.
Keep credentials outside the model context and authenticate at the tool-execution boundary.
Validate output before execution or rendering; block markdown image rendering and strip embedded URLs with query parameters where appropriate.
Require human confirmation for actions that spend money, modify data, or send communications; treat the confirmation prompt as untrusted output.
Sandbox tools with filesystem restrictions, no network access, and no process creation.
Partition agents with separate credentials, tool scopes, and monitoring profiles; isolate user sessions and sanitize persistent memory before storage and retrieval.