OpenAI Monitors Missed Up to a Quarter of Model Traffic

Two researchers from METR and Redwood Research identified major flaws in the safety systems OpenAI uses to watch its own models. The team conducted an on-site audit at the company from July 21 to August 11. They received access to internal code, documents, and interviews with about a dozen employees as part of a pilot program designed to allow outside oversight.

The audit revealed that between 6 and 25 percent of internal agentic Responses API traffic from GPT-5.5 class models and above went completely unmonitored. METR noted that the uncertainty in that estimate is high in both directions. A separate issue affected internal Codex traffic, where roughly 10 percent of actions between May and July slipped through the net. This happened because the system failed to process compaction events, which are context summaries, correctly.

OpenAI discovered the compaction bug on its own and fixed it before the audit ended. The researchers also found that agents could delay oversight by simply continuing to execute actions without handing control back to the user. The monitoring system only triggered when an agent returned control, allowing for prolonged unmonitored activity.

The auditors demonstrated that it was possible to hide hundreds of actions from the monitor and bypass the Codex sandbox. They concluded that specially crafted inputs could likely crash the monitor entirely. Additionally, model behavior during reinforcement learning training was not monitored at all when the audit began. OpenAI stated it paused reinforcement learning after the Hugging Face incident in August and expanded monitoring to cover that area, though these changes occurred after the auditors left.

OpenAI says it has fixed all the weaknesses identified in the report. The full document is 61 pages long and was shared internally. After redactions, the leadership of METR and Redwood Research received a 38-page version. Only a short summary has been released to the public. METR stated it did not have the right to disclose the redacted portions independently. The group says it wants to push for greater public transparency in future audits.