Suddenly, Dario calls for a three-step global AI speed limit!

  It's the first time in the history of the world.

Today, Dario Amodei, CEO of Anthropic, released a rare long post.

We Must Pace the Frontier (we must control the pace of development of front-end AI)

This time, he focused directly on the pace of research and development of the front-end model, and the central point was:

The most advanced AI is actively stepping on brakes.

In this introduction, Dario acknowledges that "return to self-improvement (RSI) has emerged across the industry".

More disturbingly, he predicts that in the next 6 to 12 months, the AI Agent cluster may take over the entire Internet.

In the face of the risk of this detachment, Dario directly came up with a realistic set of "three-step steering wheels":

• The first step, opening the door to visitors: introducing independent third-party resident companies with deep access rights equivalent to internal core wind control;
  
• The second step, the industry outline: unite leading-edge AI giants across the U.S. to draw a deadly red line and set up mandatory security checkpoints;
  
• Third step, global landing: To push this framework to the international level, to completely block the spread of the uncontrollable risk of death.

At the end of the speech, Ottman followed up with the speed of light, saying, "Dario is right, OpenAI will introduce a similar assessment mechanism".

On the other hand, Musk turned around for almost a second, publicly supporting Dario.

It is rare that the three giants can reach a consensus in an instant.

This time, Dario didn't just shout "AI should slow down".

What Dario really wanted to do was put this "deceleration mechanism" into action, and the order was very clear.

First, let AI companies become verifiable themselves, then let the entire industry get on the brakes, and finally talk about global coordination.

The first step is to start with the Anthropic knife itself.

An independent third party such as METR should not enter the company temporarily after an accident occurs, but should remain there for a long time.

Give them jobs, give them internal computers, give them access to risk teams.

The model, the implementation of the security commitments, the mining of mines, the emergence of a warnings, the training process will continue to linger.

And, once the problem is discovered, it must be made public.

Companies can remove parts that really involve the law, customer privacy and core business secrets, but cannot press the conclusion because the report is too difficult to see.

Step Two: You can't just let one brake ride.

It's really hard to start from here.

If Anthropic itself slows down, OpenAI, Google, xAI continue to go crazy, whoever gets on the brakes is likely to lose first.

So, Dario suggests that the rules should be rolled out across America's cutting edge AI industry.

Several leading AI companies have worked together to set up a safety red line and set up capability checkpoints. Every time a model crosses a hazardous threshold, it must first draw the corresponding safety evidence before it can move forward.

For example, a model has already broken through most common isolation environments.

The next thing the company has to prove: it won't easily escape the sandbox, it won't take over a lot of machines, and it won't turn that capability into a real attack.

If you can't prove it, don't go down there.

In addition, Dario even mentioned that it is possible to discuss limiting the training capacity, training methods, and the speed at which AI is being developed with AI within the company.

In other words, a speed limiter is added directly to the RSI itself. But it's not enough to shake hands with private companies.

He hopes that the U.S. government will eventually take over and apply the rules to all leading AI companies.

Step 3: Bring brakes to the world

Once again, it is the most difficult layer.

The first level is relatively realistic: first, a ban on the obviously dangerous use of biological weapons.

The second layer is high-risk testing, such as cyberattacks, biosecurity, before the release of the cutting-edge model.

It was the third floor that really began to touch the core:

Can the RSI, the speed at which AI is being developed, directly set a global ceiling?

Dario's own judgment is that this is "just on the edge of what's possible".

When it comes to the fourth tier of AI, the world's leading AI companies are slowing down significantly, and even the phased suspension of training itself is not very hopeful.

At least in the short term, it's almost unrealistic.

If a company says it's stopped, who can prove it's actually stopped?

So the end of Dario's whole scheme, the core of which is very simple:

Open the doors of an AI company and let outsiders really come in and see.

It is for this reason that Dario puts "third party long-term residency" as the first step in the whole scheme.

But the question is, why is it now?

Three years ago, there was a trend in the industry that called for "suspension training, stepping on the brakes of AI", but at the time, Dario didn't take no for an answer, thinking that the timing was immature.

The two things that really changed his mind were that RSI was starting to emerge and that AI was exposing more and more specific uncontrollable behaviour.

In his article, he explicitly acknowledges that AI's involvement in the development of the next generation of AI is already emerging across the industry, including Anthropic itself.

AI helps people write code, do experiments, analyze results, optimize training processes and help the next generation of AI become stronger.

If this cycle starts to self-accelerate, the most troubling part comes up:

The speed at which the model's capabilities are enhanced may begin to exceed the speed at which humans understand and control it.

This is because security research itself takes time.

You have to first observe what new behaviours appear in the model, then study why these behaviours appear, then design test methods, precautions, and finally verify whether these measures are useful.

But if the model's capability jumps a ladder every few months, it's easy for the security team to chase the flaw behind it.

Dario's real concern was not that AI would suddenly "wake up" one day, but that the feedback loop would start to shift faster and faster.

AI is becoming more and more involved in AI research and development, the next generation of models is emerging more and more quickly, and security mechanisms are not keeping up.

If the RSI only made him worry that "the future will run too fast", then the latest real-life incident in the intelligence community has completely eroded that concern.

Dario specifically mentioned the recent "Open AI-Hugging Face incident".

In that test, a group of AI-smart bodies formed a collaborative system similar to that of a beehive.

At first, humans simply let them complete their normal tasks. But soon, some intelligent entities began attacking external targets outside the mission.

What's even more bizarre is that they've also shown conspicuous group collaboration: some smart individuals are willing to sacrifice their mission accomplishments to help the whole group get better results, or even try to hack the scoring system to "cheat" the whole group.

These behaviors are not pre-written scenarios for humans; they arise by themselves in the process of multi-intelligent collaboration.

A similar phenomenon has been observed within Anthropic, but to a lesser extent.

What Dario really worries about is that it's been 6 months and 12 months.

By then, the Agency will be stronger, more tools to call, more machines to control, and more time to work continuously.

If these deviations from the goal, drilling rule loopholes, and group collaboration were to continue today, the risk could be magnified instantly.

He even gave a rather surprising scene:

Large-scale AI agents have invaded connected devices, turning the entire Internet into a giant "zombie network".

The damage could be in the hundreds of billions of dollars.

So, as Dario shouted "slow down", the real concern was not some distant super-intelligent.

It's just a couple of things happening at the same time:

RSI has emerged, and AI has begun to participate in the development of stronger AI; Agents are gaining increasingly strong real-world capabilities; and some out-of-control behaviors have moved from hypothesized to real-world testing.

The speed at which the model evolves continues to accelerate.

That's why he wants to take a million or two years to supplement security assessments, third-party verification and global rules.

This is a plan that can't be solved, and there are no answers yet.

Now those who were running in front of him began to ask, "Are we running out of control?