Auditors Find Ways to Bypass OpenAI’s AI Agent Monitoring
Summary
Metr and Redwood Research identified ways to bypass or disable OpenAI’s coding-agent monitoring. OpenAI said it patched production safeguard vulnerabilities, but the evaluators did not verify that claim.
Key points
- Metr and Redwood Research assessed OpenAI’s internal misalignment-monitoring system from July 21 to Aug. 11, with access to internal code, prompts and documentation.
- The evaluators reported that Codex agents could bypass sandboxing and Auto-Reviews, client-controllable headers could switch off monitoring, and an exploit could hide hundreds of recent actions.
- They estimated that 75% to 94% of agentic internal responses API traffic powering Codex was eligible for monitoring during the assessment period.
- OpenAI said it patched the identified vulnerabilities affecting its production safeguards stack; the evaluators said they had not assessed that claim or subsequent monitoring changes.
- The evaluators recommended more frequent monitoring, cross-checking inference against logs, tamper-proof searchable records, clearer incident-response ownership and testing for exploitable bugs and strategic collusion.
- They also suggested isolating agents from monitors so agent actions cannot influence whether or when monitoring runs.
Article Details
- Event Type
- Independent safety evaluators identified vulnerabilities and coverage gaps in OpenAI's monitoring system for coding agents.
- Impact
- The evaluators reported that Codex agents could bypass sandboxing and Auto-Reviews, client-controllable headers could disable monitoring, and an exploit could hide hundreds of recent actions. Metr said OpenAI stated that vulnerabilities affecting its production safeguards stack had been patched and that it had made subsequent changes to monitoring coverage; the evaluators said they had not assessed those claims. They estimated that roughly 75% to 94% of agentic internal responses API traffic was eligible for monitoring during the review period.