OpenAI trained the most advanced models to step on the brakes.
Just now, OpenAI said in a statement that the company had previously suspended its latest, planned model reinforcement learning (RL) training for deployment for two weeks. In the meantime, OpenAI strengthened the research environment, conducted Red Team testing, and expanded the coverage of its internal monitoring system.
However, the largest-scale front-end model RL training that was originally planned is currently on hold. The company is deciding whether to continue with smaller-scale training and evaluation to observe model behavior, verify new safety measures, and accumulate more coordinated evidence.
Sam Altman tweeted, "We've always stressed that if the model's capabilities go beyond the requirements of security and alignment, we will act immediately.
When the news comes out, the netizens are crazy!
Some netizens believe that this undoubtedly means that the Astra model has been delayed again.
Some netizens also believe that the situation seems somewhat unfavorable with the recent exit of some C-level administrators from OpenAI.
Some netizens believe that this may just be a more "popular" marketing tool than it is today.
But in reality, OpenAI made the decision to actively slow down Scaling and move security requirements from the "deployment stage" to the "training stage" for two main reasons.
Two torpedoes
Over the past few weeks, two incidents have led OpenAI to decide to slow down its training.
The first was the Hugging Face security accident. OpenAI models broke through the isolation environment in internal cybersecurity assessments, gained access to the Internet, and eventually attacked Hugging Face's infrastructure.
The second relates to the unreleased new model Astra. Initial assessments suggest that Astra may reach the "critical" cybersecurity capability threshold defined by the OpenAI Preparedness Framework.
In the OpenAI Framework Package, the cybersecurity capabilities that could pose a serious threat will be divided into two levels: "high" and "critical".
Previously, GPT-5.6Sol's network security capabilities have been rated as "high".
As defined by OpenAI, reaching the "critical" threshold means that the model may have one of two capabilities: to detect and develop effective zero-day vulnerabilities of varying severity from a large number of real-world critical systems with enhanced defenses without human intervention; or to autonomously design and execute an entirely new attack program based solely on a high-level attack target.
Strengthening security for more capable models
Specifically, OpenAI now generalizes its security system into three mutually reinforcing parts:
Monitoring: detecting behaviour that deserves attention and being able to respond in a timely manner.
Alignment: Reducing the likelihood that the model will take harmful or unauthorized actions.
Security measures: Limiting the objects and scope to which AI systems can access or influence.
OpenAI expects that in the near future, most of the security work will be done by the model, including defending other models.
According to OpenAI, these security measures will be implemented throughout the research and deployment process and adjusted according to the capabilities, operating environment and risk levels of different models.
The first is to strengthen the research environment.
OpenAI said that as front-end models acquire increasingly strong cybersecurity capabilities, they are also improving the security standards of the environments they use to train and evaluate them. Achieving these standards requires a large amount of engineering resources, which also brings high costs and significant delays to front-end research.
In the coming weeks, OpenAI will develop and start implementing more stringent security requirements for front-end research workloads, such as:
Workload isolation: For workloads that execute model generating code or other unreliable code, a stronger isolation environment, or a more stringent "sandbox", is now required. The same requirement applies to software that may be attacked when processing model output.
Network isolation: OpenAI adds more controls to isolate high-risk and unreliable workloads from the Internet. The design goal is that even if a workload or support service is single-pointedly breached, this breakthrough cannot result in unauthorized access to the Internet or access to other internal networks.
Continuous security testing: OpenAI has reconfigured the research environment, removing vulnerable shared services, reducing long-term retention of permissions, and improving security boundaries and trust boundaries. It is also enhancing the collection and monitoring of security logs. At the same time, it is introducing automated systems that use its own models to continuously simulate attack testing on these boundaries.
OpenAI also said that the highest level of security precautions are currently required for workloads involving Astra or a cybersecurity model, as it has judged that Astra models may have Critical-level cybersecurity capabilities.
Secondly, expanding mind-chain monitoring
This is also a more interesting point in the article, where OpenAI begins to "monitor AI with AI".
According to OpenAI, monitoring has long been an important part of the model development process, including monitoring of the internal programming agent and the long-running Session. Prior monitoring focused primarily on the internal deployment of front-end models and RL training tasks for front-end models. OpenAI believes that these workloads are the most risky because they may have advanced modeling capabilities and access to tools, sensitive systems, and internal or external networks at the same time.
However, this set of monitoring did not previously cover the reasoning and training tasks of all the tools used. Now the monitoring system has been adjusted and expanded, which greatly expands the coverage of mind-chain monitoring.
The new monitoring system is multi-stage. The first is a class of activation classifiers, that is, a detector that monitors the activity within the model, with each token that is sampled running. Once a potential risk is detected, the task is upgraded to a stronger and more computational capability.
These investigative models examine comprehensively: which tools the model uses, what reasoning information can be obtained, and the trajectory of the entire behavior. They focus on finding unauthorized access, data theft, destructive operations, and attempts to circumvent security mechanisms.
In other words, the model is doing the task and next to it is another set of models that reviews it in real time to see if it is "doing something".
OpenAI has set a very clear timeline for the system: the target is alerted within 30 minutes after a worrisome activity is detected. If a top-level alert is triggered, security, security and research teams will be alerted. If there is no clear proof of misrepresentation within 30 minutes, the relevant activity should be suspended.
Now, as long as Sol's capability level or above is reached, RL training and evaluation involving tool calls must be enabled.
Astra goes even further. OpenAI, after judging on 7 August that Astra may have Critical cybersecurity capabilities, stipulates that all Astra+ tools' reasoning activities must be monitored.
This change is significant, meaning that within OpenAI, Astra has been seen as a high-performance system that requires special control.
It also promotes coordinated research.
OpenAI says RL training for the most capable models is now applying core alignment technology to more training stages, including:
Improving the reward model so that it can more accurately identify and suppress unsafe behaviour in different tasks and environments;
Training models to describe their actions, capabilities and limitations more honestly;
Reducing the behaviour of models using incentive mechanisms, raters, tools or oversight system vulnerabilities.
At the same time, OpenAI is also expanding the coverage of relevant training, focusing on the behaviour that can cause harm when models interact with external systems or resources. OpenAI says that it will continue to invest heavily in coordinated research, expanding assessment coverage, and using the results of the research to guide subsequent training and safety prevention.
It's worth noting that OpenAI left a statement at the end of the article: "The ability of front-end models is accelerating rapidly. Our ability to understand, coordinate and protect these models must go first".
It's just that the problem is now that it's becoming increasingly real, and as the model continues to get stronger, can the security capabilities really keep running ahead of it?
For now, OpenAI's choice is to stop the model.
So what do you think of OpenAI?
The link is as follows: