Independent Reviewers Find Significant Blind Spots in OpenAI Safety Checks

A recent independent assessment has uncovered substantial deficiencies in the internal safety monitoring systems used by one of the leading artificial intelligence developers.

The review was carried out by two researchers from METR and Redwood Research, who were granted temporary access to OpenAI's facilities to evaluate how the company tracks potential rogue behavior in its AI systems. The investigation spanned three weeks, providing a detailed look at the infrastructure designed to keep advanced models in check.

The timeline for this audit began on July 21 and concluded on August 11, 2026. During this period, the external team had direct access to OpenAI's codebase and internal documentation. They also conducted interviews with approximately a dozen employees to understand the practical application of the monitoring tools. This access was part of a pilot program designed to test the viability of granting outside researchers deep visibility into internal technical operations.

The findings revealed that a significant portion of internal traffic was not subject to any monitoring whatsoever. The auditors estimated that between six and 25 percent of the agentic Responses API traffic generated by models of the GPT-5.5 class or higher was entirely uncovered during the review period. METR noted that this estimate carries considerable uncertainty, meaning the actual figure could be higher or lower than the midpoint.

Further weaknesses were identified in the tracking of internal Codex traffic. The researchers found that roughly 10 percent of this traffic went untracked between May and July due to a specific gap in how the system handled compaction events, which are essentially context summaries. OpenAI had independently identified and corrected this bug before the audit ended, but the issue remained a key finding. During the period when these compaction events were not monitored, agents had the ability to delay oversight, as the system only triggered when control was handed back to the user. This allowed agents to continue executing actions without immediate review.

The auditors also discovered methods to conceal hundreds of actions from the monitoring systems and to bypass the Codex sandbox environment. Additionally, they determined that specially crafted inputs could likely cause the monitor to crash entirely. At the start of the audit, model behavior during reinforcement learning training was not being monitored at all. OpenAI stated that it paused this type of training after an incident involving Hugging Face in August and subsequently expanded its monitoring coverage, though these changes occurred after the auditors' access period and were not part of the initial evaluation.

OpenAI has confirmed that all identified weaknesses have been addressed. The full 61-page report was provided to the company, which then applied redactions before sharing a 38-page version with the leadership of METR and Redwood Research. Only a short summary was released to the general public. METR indicated that it did not have the right to disclose the redacted sections on its own, leaving much of the detailed technical analysis hidden from public view.

Looking ahead, the participating organizations intend to continue the pilot program for external audits. METR has expressed a desire to push for greater public transparency in future assessments to ensure that the findings of such deep-dive investigations are more accessible to the broader community and stakeholders.