When Agents Clash: How Frontier Models Handle Multi-Agent Conflicts

  It's the first time in the history of the world.

AI Agents are now increasingly being deployed by teams in real-world workflows such as code writing, cloud operations, financial auditing and enterprise approval.

However, the existing Computer-Use benchmarks (such as OSWorld, WebArena) almost default to an ideal premise:Environmental information is always consistent, and all teammates are 100% reliable.

This is clearly not the case in the real world. In multi-agent collaborations, sub-agents often report incorrect status by reading expired caches or colliding with the master agent on file locks and permissions.

For example:The agency sent a message saying, "Everything is fine online, please turn it off immediately", but the real-time monitoring panel's screenshots were already reporting a crazy error.At this point, the master agent will report the text of the blind letter teammates directly to the online accident; it must actively open the monitoring screen to verify the true status of Ground Truth, and move back to the configuration, releasing the shared lock.

FromGoogle, Emory, USC, UF, UChicago, UCSB and ColumbiaThe research team published jointlyCLASHarnessFor the first time, 15 mainstream models were tested for their real performance in multi-intelligence conflicts.

Link to the paper:https://arxiv.org/html/2610.19128

Open source code and data sets:https://github.com/google-research/CLASHarness

It covers 20 real-world fields, 600 rounds of conflict missions, 7,970 4K visual evidence images, and 3,284 mandatory checkpoints.

To build a conflict test kit that includes code libraries, terminal logs, multi-agent chat records, and high-definition UI snippets, the challenge is to:Artificial writing alone cannot be expanded on a large scale; and if LLM is directly generated one-time, it is very likely that there will be back-and-forth dramatic contradictions, unwary puzzles, or even "dead questions" that cannot be solved even by standard answers.

For this purpose, CLASHarness has designed a set."Three-tier eight" automated generation and verification of pipelinesIt's guaranteed from the source.100% task-resolvability：

Figure 1: CLASHarness' three-layer eight-phase pipeline generation. The green box represents zero-cost code auto-repair and the red box represents local single-step retesting of error feedback.

1. Step-by-step planning + local automatic error correction (Figure 1)

The pipeline first plans the overall outline and Agent role permissions, and then generates specific tasks every three to four rounds of rolling, avoiding context bias on the long text. Each round of generating is divided into eight small steps: once a path or format problem occurs, directly repair the code automatically ((0 model calls); in the case of a logical conflict, only one small step is repeated, without having to be pushed back, on average only.95 timesAPICallingIt is possible to create a complete 10-round continuous mission package.

Figure 2: Examples of 4K visual evidence generated by CLASHarness (including monitoring panels, IAM authorization spreadsheets, computational scheduling monitoring and billing details)

2. Delete the text "walking shortcuts", where the key evidence is only stored in 4K images (Figure 2).

In order to prevent Agent from looking at charts and scrolling through text only, CLASHarness first sets the key real values for resolving conflicts (such as the actual bill amount, status of the permissions matrix) and then renders them.3840 x 2160, 4K resolutionA real UI snapshot or real-world photo, and an automated scanner to clear all the clues in the text file, forces the Agent to look at the graph.

3. First the sandbox runway standard answer, reproduced as a scorer

All tasks are scored according to a score standard: 34.9% accurate code checks + 65.1% semantic three-vote majority checks) all generated directly from the reference trajectory running through the real sandbox.23 modular conflict pluginsDevelopers can synthesize new conflict testing scenarios on demand, like building blocks.

The team evaluated 600 conflict missions.15 mainstream models(including Claude Opus 5/4.8, Sonnet 5, GPT-6, Astra, GPT-5.6 series, Gemini 3.x series, and the open-source model Kimi K3, DeepSeek V4 Pro/Flash, GLM-5.2), all completed their run cumulatively.9,000+ real sandbox tracks。

Figure 3:15 Total ranking of the models in CLASHarness (600 missions)End to endThe task pass rate (Pass Rate) is the average score for a single check on the right side (Mean Score).

It's the first time I've ever seen a single point score that's not less than 60 percent.End to endThe pass rate is no more than 30%.

As shown in Figure 3, under the standard 20 step limit,Claude Opus 5（25.2%）andGPT-5.6 Sol（24.5%）It's the highest ranking; after the number of looser steps,GPT-6 Astra scored the highest 28.7%.In the open source camp,Kimi K3: 21.8%, 5th overallandDeepSeek V4 Pro（19.8%）It has been shown to be very prominent, matching and even controlling many closed source commercial models.

Why do top models get it?55% and 67% single-step (Mean Score)It's not even 30% because solving real conflicts is a complete chain:Identify teammates misdirected→ Open images verify true values→ Modify system configuration→ Finish the connectionIf one loop is missed in the middle, the whole mission fails.

Discovered 2: Weak model blind letter teammates text, strong model actively view chart checks

When Agent reports in writing and real-life environmental conflicts,Weaker models are easy to "soften the ears" and report text errors directly; stronger models actively call tools to browse the monitoring screenshots or bill pictures to verify true values.(Leading weak models in visual positioning and evidence citation)31 and 65 points.）。

If the image input is turned off (see Figure 4b), the pass rate for all models will plummet instantly.87%–90%Even if the task of looking at a graph without hardness regulations is to look at a graph, the model's approval rate of not looking at a graph is only one-tenth of what it was before (e.g. Claude Opus 5 dropped from 24.8% to 2.8%).

Discovering 3: "Seeing the contradiction" is easy, "Make it right and finish it" is very difficult.

To determine whether the model is "out of sight" or "out of sight", the team compared two task requirements for the same set of conflict scenarios:

Just ask for an investigation report:The average pass rate of the model is28.1%；• Requires a manual modification of the system status:It's going straight down to the rate.Net decrease of 18.2 percentage points)If a complete multi-agent collaboration process is required, the passing rate is much lower.4.4%。

One of the easiest things to do is stick with the "end link":Even the most powerful GPT-6 Astra and Claude Opus 5 are available.58% and 72% of the final moves failed。

The model often completes the task directly after the main configuration file has been changed.Forgot to release the share lock, forgot to close the worksheet, or forgot to send a message notifying the downloader that Agent has taken over.

Discover 4: The more you think about Tokens, the easier it is to open the loop and unlock the depth of exploration.

In mathematical and coding problems, making a model think more (or adding Thinking Tokens) is usually a good idea. However, in multi-agent interaction environments, the opposite results:

Figure 4: Three large fusion experiments. (a) Loosen the number of steps limiting 20→30→40 steps; (b) Have visual vs. non-visual input; (c) Change the ranking consistency with different generating models; (g) Gemini 3.6 Flash vs. Claude Sonnet 5)

1. The "Death Cycle Trap" caused by excessive thinking

Under the 20-step limit, the higher the token percentage of the model spent on internal thinking, and the lower the success rate instead (relativityρ=-0.70). For example, if the thinking intensity of the Gemini 3.5 Flash is shifted from Low to High, its pass rate is shifted from13.5% drop to 3.0%The cost of solving a single problem rises to$39.54This is 55 times the GPT-5.6 Luna $0.72.89.2% of missions spent 20 steps without doing the right thing.

2. Unlock the number of opening steps and unlock the long-range exploration (Figure 4a)

Conversely, for a model that is really good at multi-step exploration, the 20-step limit restricts its play.Gemini 3.8 Flash's pass rate jumped from 12.8% to 29.2% and 30.7% respectively.The average cost of solving a problem has also decreased, while the average cost of solving a problem has decreased.21% (down to $3.50)。

In addition to being a benchmark for assessment, CLASHarness is also a set.Multi-intelligent agents that can be expandedTraining dataSynthetic enginesWith the help of 20 built-in field seeds and 23 conflict plugins, researchers can at low cost mass synthesize 100% solvable multi-modal conflict trajectories and verifiable reward signals (Verifiable Rewards) to train the next generation of more reliable, non-blind, collaborative multi-intelligent systems.

Currently, the CLASHarness generating pipeline code, 600 evaluation tasks and interactive trajectory viewers are available on GitHub.