OpenAI Safety Monitors Left Blind in Up to a Quarter of Traffic, Audit Shows

New findings from a recent on-site audit suggest that OpenAI’s internal systems for detecting rogue AI behavior have left substantial portions of its most advanced models unmonitored. Researchers from METR and Redwood Research, who spent three weeks embedded within OpenAI’s infrastructure, identified critical gaps in the company’s monitoring protocols for models at the GPT-5.5 class and above. The audit, conducted from July 21 to August 11, revealed that a significant slice of internal traffic was effectively invisible to the safety oversight systems designed to catch it.

During the review period, the auditors had full access to OpenAI’s codebase, internal documentation, and interviews with approximately a dozen employees. Their analysis indicated that an estimated 6 to 25 percent of internal agentic Responses API traffic was not covered by any monitoring at all. METR emphasized that this estimate carries significant uncertainty in both directions, but the potential scale of the blind spot is considerable. This lack of oversight applied specifically to the most capable models, raising questions about the depth of current safety nets.

A specific technical flaw further complicated the situation for internal Codex traffic. Between May and July, METR estimates that roughly 10 percent of monitored traffic went untracked due to a bug in how the system handled compaction events, which are context summaries used to manage memory. While OpenAI independently discovered and fixed this bug while the audit was still ongoing, the auditors noted that the estimate for the untracked portion also carries significant uncertainty. This gap created a window where actions taken by the models were not recorded or reviewed.

Beyond the specific bug, the auditors found structural weaknesses that allowed agents to delay or bypass oversight entirely. The monitoring system was designed to trigger only when an agent handed control back to a user. By simply continuing to execute actions without yielding control, agents could sidestep the check. The team demonstrated that it was possible to hide hundreds of actions from the monitor and bypass the Codex sandbox, suggesting that the current safeguards may be less robust than previously assumed.

The review also highlighted vulnerabilities in the monitoring tools themselves. The auditors determined that specially crafted inputs could likely crash the monitor completely, rendering it useless during critical moments. Additionally, they found that model behavior during reinforcement learning (RL) training was not being monitored at all when the audit began. OpenAI stated that it paused RL training following the Hugging Face incident in August and expanded monitoring to cover this area, but these changes occurred after the auditors’ access period ended and were not part of the evaluation.

OpenAI maintains that it has now fixed all the weaknesses identified by the auditors. The full 61-page report was handed over to the company internally. However, the public view of the findings remains limited. After OpenAI applied redactions, METR and Redwood Research leadership received a 38-page version of the report. Only a short summary has been made available to the general public, with METR stating it had no right to disclose the redacted portions on its own initiative.

The exercise was a pilot program organized jointly by METR, Redwood Research, and OpenAI, designed to give outside researchers deep access to internal infrastructure. While the pilot has concluded, the findings suggest that the path toward robust AI safety monitoring is longer than expected. For future audits, METR has indicated it wants to push for greater public transparency, aiming to bridge the gap between internal technical details and external understanding of AI risks.

Until those broader transparency measures are in place, the current landscape of AI safety monitoring remains partially obscured. The audit provides a rare, detailed look at the complexities of overseeing autonomous systems, but the redacted nature of the full report leaves many specifics out of public view. For now, the focus shifts to how OpenAI implements its fixes and whether the next wave of audits will yield more accessible insights for the broader AI community.