Auditors find up to a quarter of OpenAI's top model traffic escaped internal checks

Two outside researchers have documented major blind spots in the system OpenAI uses to watch its own AI models. The audit, conducted on-site at the San Francisco headquarters from July 21 to August 11, found that the company's internal monitors failed to track a large portion of its highest-tier traffic. The findings come from a joint pilot program involving OpenAI, the evaluation group METR, and the research organization Redwood Research.

The auditors estimated that between 6 and 25 percent of internal agentic Responses API traffic from GPT-5.5 class models and above was completely unmonitored during the review period. METR noted that the uncertainty in that range is wide. A separate issue affected internal Codex traffic. Approximately 10 percent of that traffic went untracked between May and July because the monitoring system mishandled compaction events, which are context summaries. OpenAI identified and fixed that specific bug while the audit was still underway.

The structure of the monitoring system allowed agents to delay oversight. The monitor only activated when an agent returned control to the user. An agent that continued executing actions without handing back control could avoid the check entirely. The researchers also demonstrated ways to hide hundreds of actions from the monitor and to bypass the Codex sandbox. They concluded that specially crafted inputs could likely crash the monitor completely. Furthermore, model behavior during reinforcement learning training was not monitored at all when the audit began.

OpenAI stated that it paused reinforcement learning training following the Hugging Face incident in August. The company later expanded monitoring to include that phase. Those changes occurred after the auditors finished their on-site access, so they were not part of the formal evaluation. OpenAI says it has now fixed all the weaknesses the auditors identified.

The full report consists of 61 pages and was shared internally. After OpenAI applied redactions, the leadership of METR and Redwood Research received a 38-page version. Only a short summary has been released to the public. METR said it did not have the right to disclose the redacted portions independently.

The audit involved access to OpenAI's codebase and internal documents. The researchers spoke with about a dozen employees during the three-week period. The exercise was designed to give outside researchers deep access to internal infrastructure. METR says it wants to push for greater public transparency in future audits.