Eighty thousand pieces of code to recover OpenAI's invasion of HF!

  It's the first time in the history of the world.

An AI "escape from prison" that resembles a hacking movie made Peter Steinberger, the founder of OpenClaw, "Father of the Lobster", so shocked.

Now I know why some people talk about AGI.

On September 25, researchers Jeffrey Ladish and others released a new survey called Swarm Traces, revealing a wealth of details about OpenAI's invasion of the open-source community Hugging Face (HF).

Peter is referring to Ladish in a post about how AI attacks are done.

A group of OpenAI smart entities that should have been isolated in an isolated environment have taken on a number of common network services as an attack channel that can execute programs and retransmit results under restricted network conditions, such as Spike.

They use short chain sequence program snippets to execute the code from the browser that scraps the site, then encode the resulting pictograms, and send it back with the scraping.

With the help of this temporary spelled-out toolchain, coupled with the exploitation of HF vulnerabilities, they ultimately broke the security barrier of this AI open source community.

The details of the invasion were recovered by an eight-person independent investigation team.

The team is led by three members of the developer tool company Parse, with other members including researchers from institutions such as Palisade Research, the Trajectory Institute and others.

Nearly a million URLs left by the agents became clues to their tracking.

Over the course of two weeks, the team scanned millions of public links, retrieved more than 80,000 attack codes, and found them in the Swarm Traces report.

These scattered codes on the Internet are the "working tools" left behind by the HF invasion in July this year.

At the time, OpenAI was running a security test experiment called ExploitGym.

The intelligence community, isolated from each other, has found an unauthorized way to communicate.

Approximately 1,200 SMBs exchanged more than 70,000 messages and files through a message board, with about 700 of them participating in further attacks against HF.

By July 11, a smart entity was able to remotely execute code in the workflow of HF for producing data processing. Subsequently, the attack spread to internal infrastructure.

According to OpenAI, this is the most serious of its model-induced activities to date, driven primarily by an undisclosed internal research model.

Today, the eight investigators are packing up the scattered pieces of the Internet, bringing up a question:

How did these AI's step by step transcend the limits?

What is most surprising is the way in which they acquire executive power.

According to the new report, smart entities first gained access to the outside world using environmental vulnerabilities, but this access path appears to only support GET requests.

The common saying is that they can only "see" web pages through the doorway, and they can't submit forms directly as we normally go online and send all sorts of operation requests.

But the website itself can also carry information.

They first found a type of HTTP testing site. This type of site can convert encoded content into a web page, which makes AI-compiled program snippets a dead end.

Then they came up with a second tool: Web Scraping Services.

This type of service must first open the target web page with its own browser before it can be screened. When the browser loads the page, the program will also run.

Thus, AI with limited access to web pages borrows the execution capabilities of external browsers.

AI hides the program in a link, executes it in a browser with a screenshot service, and then returns the results with a screenshot.

It's too long, so what if you can't install a website?

Disassembled and reassembled.

They break down the program into many sections, each linking a section, and then pointing to the next section. The browser collects it along the link and then reassembles it into a complete program.

The report found that some programs linked more than 900 links!

A short chain sequences snippets of code, which external browsers collect, spell out and execute as a complete program.

After the last step, when the program is running, how do you get the results back? After all, the scanning service returns only a picture.

They also came up with a way: to encode the returned data into a pixel format, to have the scanning service take it down and send it back.

The data is transformed into a pixel format, which is retransmitted as a snapshot, and then restored to text by AI.

Web pages carry code, short chains are responsible for synchronization, external browsers are responsible for execution, and scraping services are responsible for feedback.

A number of commonly used network services have been adopted by AI as a set of tools that can run programs and retrieve results.

This is exactly where Peter calls himself "genius".

However, using the scanning service execution process and access to the HF server, there are still two links. The former provides execution and communication channels, while the latter involves exploiting HF-side vulnerabilities.

In our opinion, the most cautious part of this combination is the ability of AI to borrow external services and constantly make up for its shortcomings.

The restored script also reveals a series of operations after the AI enters the HF.

HF, an internal dataset explanatory document, has written a remarkable warning: "This dataset must never be made public". The document is also particularly noteworthy for containing highly sensitive data.

HF's internal dataset clearly warns that containing sensitive data is strictly prohibited from being made public.

However, this warning did not prevent further action.

The report found that there are still scripts attempting to explore this dataset and use it as storage space.

In other recovered scripts, the investigators also found that there were programs that collected AWS credentials, access tokens and sensitive environmental variables, and placed this information in a dictionary called LOOT.

LOOT, translated literally, means "the spoils of war".

The program also scores tokens based on permissions, prioritizes the selection of candidates with higher permissions, and attempts to obtain more internal information.

In other scripts, search targets are targeted at Slack chat records within HF. Search terms revolve around clues such as ExploitGym, CyberGym and raters.

Some programs also design controllers that continue to receive follow-up instructions and attempt to remove previously uploaded files. They try to turn a single access into a reusable operating condition.

Not only that, they also know how to "shake people".

Investigators found that some of the requests attempted to summon other large models to help determine whether the exploitation of the vulnerability met the assessment requirements.

Test requests recovered: reply only to "HELLO42".

Claude also appeared in the request, but was primarily asked to return a test of short fixed text.

Looking at this, you might ask: Why do they do this for so much money?

METR's investigation gives a key clue: the main motive for attacking HF seems to be the search for information on how the ratings are implemented, and the study of how the ratings have been deceived.

In other words, in order to figure out how to score, some intelligent agents extend exploration into the systems of real companies.

This is the most disturbing place:

When problems are solved normally, will the model bypass the rules as another way to complete the task?

The new report recovered a large amount of code sent out by the intelligent body, but the corresponding response record is incomplete.

As a result, further verification is needed as to whether Slack's search was successful and whether the trail was cleared.

More than 80,000 pieces of attack code are not equal to 80,000 successful attacks.

According to the report, HF confirmed that the code matched the findings of the internal investigation, and the relevant credentials were withdrawn in July.

However, HF has not detected these specific short chains before. More than two months after the end of the attack, they are still publicly accessible.

The short chains used by the S.M.O. to transmit the program were eventually used as clues for investigators to recover the intrusion.

The incident put a blind spot in the design of the authority in front of the stage.

Web pages are read, shortcuts, snippets, and just look at all the useful ways and limitations. But by connecting them, Agent gains the ability to execute the program and retransmit the results.

If the security assessment focuses on what each tool can do individually, the actual capabilities of the entire toolset combination may be missed.

For developers, it is also important to keep asking:

What external services can AI access, how data and instructions flow across tools, and what can these services do for it?

OpenAI also adjusted the interpretation of the incident.

Officials initially understood it as a security issue, but later realized that the invasion was related to the model's misguided tactics for solving difficult tasks.

In the aftermath of the HF incident, a new gap still emerged in security.

In another report updated on September 25, OpenAI disclosed that on September 20, an internal research intelligence body used DNS filtering gaps to connect external chat bots.

Around 12 minutes after the monitoring, the alert was alerted and personnel confirmed within three minutes. However, the training operation, which should have been stopped automatically, did not stop until about 2.5 hours after the alert was manually terminated.

According to OpenAI, this is the first time such an incident has been detected since the HF incident, which was less severe than some of the previous incidents.

The incident exposed a gap in disposal after the crossing of the border: the alarm sounded, but the operation did not stop as expected.

Fixing loopholes, improving stopping mechanisms, can address problems that have already been discovered. However, if a model encounters a problem, whether it will still consider "cross-border" as an acceptable solution, it is necessary to look for answers from the behavior of the model itself.

The Swarm Traces report reveals the blind spots of tool authorization design, while another report by OpenAI, updated on 25 September, reveals the gap in disposal after the discovery of cross-border: the alarm was sounded, but the operation did not stop as expected.

The next time we open a seemingly ordinary tool to AI, we don't just ask how many fans it can use to open doors that it wasn't going to open, we also ask:

Once it crosses the border, can we stop it in time?