In 2008, Invisalign used a mad experiment to demonstrate the power of parallel computing.
On the scene, a painting robot works by piece, pixel after pixel, drawing a "smiling face" sketch.
But then the painting mutated. The staff started a huge installation full of 1,100 colored projectile tubes. Accompanied by a huge sound, thousands of colored projectiles were fired almost simultaneously, leaving a trace of thick linen on the canvas. A Mona Lisa painting appeared instantly, taking 80 milliseconds.
The two styles of painting show huge differences between sequences and parallels.
18 years later, parallel computing has long been an important foundation for the development of artificial intelligence, with the GPUs of Great Britain supporting today's large-scale model training and reasoning. Interestingly, however, the vast majority of mainstream large-language models still follow a rather "serious" method of text generation.
They are similar to the drawing robots of the same year, from left to right, where one Token after another generates a response. Even with more powerful computing hardware, the generation of the next Token usually has to wait for the first Token to complete.
So is it possible to make a language model like that color bomb device and generate multiple tokens at the same time?
The spread-language model (DLLM) is exploring such a possibility, and the direction has gained a lot of attention in the past two years.
Recently, a key player in this direction, DiffuSpace, the company that created the world's largest distributed language model financing record, announced a new development, releasing the open-source distributed language model Clover-35B-A3B-Preview. Official data shows that the new model reaches or exceeds the same size self-return model in multiple capability assessments, while achieving faster generation speeds and using smart body programming and tools in such difficult tasks.
In the past, people have always tuned models such as "don't be sorry, you can't go fast", meaning that speed and quality are always missing. But this time, the Clover 35B-A3B proved on its scale: high-speed generation and high performance can coexist in the same model.
It seems that the spread of language models is at the forefront of this pathway and is being pushed a step further.
This time, the dLLM and the self-return model
They're on the same runway.
Before we get into the details, let's get a feel for how Clover 35B-A3B works:
In this demo, the model is asked to simulate a desktop environment with a single HTML file, support window drag, and open multiple small applications. According to this requirement, Clover generates a simulated desktop that contains program tabs, window systems, and applications such as notebooks, calculators, clocks, weather, and settings. In the process, Clover can quickly generate and slowly release at multiple locations simultaneously.
Test data showed that Clover generated an overall speed of 680 tokens/s on a single GB300 and 515 tokens/s on an H200, higher than 365 tokens/s on the same-sized Qwen model plus the DFlash2 accelerator.
This speed advantage has been verified in several scenarios:
The model was asked to query "How long does it take to get from CMU to the nearest Wendy's?" A map search without a location signal would jump directly to Maine, Clover would identify the results running out of order, gradually adding city and mail rewrite searches, repositioning the page twice after sticking, and finally finding the correct store in 28 steps, 169.9 seconds; the Qwen3.6 Flash counterparts took 578.3 seconds to complete the 30 steps, still staying in a store 347 kilometers away, without passing the task judgement.
A composite air traffic control report required parallel analysis of five missions, such as lifting a no-fly zone, modifying a flight point, and assigning tasks. Clover gave five correct results in 0.97 seconds, and Qwen in 3.78 seconds, and miscalculated one of the mapping missions as a medical task.
Moreover, it is not only fast, but it is also effective in tasks such as programming and using smart devices.
In six Agent ratings, Clover received the highest score of the following models, including the SWE-bench Verified 73.6, Terminal-Bench 2.1 51.7, et3 Overall 65.3, while outperforming the front-end model LLaDA 2.2 flash with the same spread route, the closed-source Mercury 2.5, and the self-return model Qwen 3.6-35B-A3B of the same size.
The significance of this set of numbers is that previously, dLLM had been chasing after the returns, and some of the more well-known models had either failed to run the Agent mission or were not open source at all. Clover was the first distributed model in the open source camp to successfully defeat the "home field" of the Agent.
It should be noted that such a capability directly corresponds to real-world scenarios such as customer robots searching databases, on-board systems controlling vehicle windows, behind which are models that issue tool calling instructions; and intelligent programming with the largest and most willing to pay user groups, is already a real productivity tool.
This has already been confirmed in real-life product scenes.
DiffuSpace's ecological partner SigmaZ is doing real-time interactive cop: users ask "how does the particle work" and "why does the eclipse not happen every month", and the model generates a clickable, interactive 3D web page to explain it in real time.
This scenario maximizes Clover's "quick" value: a round of interactions would generate tens of thousands of Token web codes, wait a while with a self-returning model API (tens of TPS), and Clover's hundreds of TPS could be completed in a matter of seconds.
It's worth noting that this is not a one-time demo collaboration. According to DiffuSpace co-founder Suzanne III, SigmaZ has been testing Clover's API for a while before the demo went live, and has given a lot of positive feedback on speed and interaction experience. In other words, the speed advantage of DLLM is that people are willing to use it as a product.
How do you do that?
Clover also makes faster speeds and higher quality, and in a relationship where the two are usually only interchangeable, this is a Pareto improvement. But at the same time, everyone is curious: since dLLM can generate multiple tokens at the same time, why didn't it take an advantage earlier in terms of speed and capacity?
Because it can be generated in parallel, it must be faster than it actually is running. From the mode of generation to the efficiency of execution, there are also issues such as caching reuse, attention computation and generating quality in the middle.
Although the self-return model generates tokens one by one, there is a mature acceleration mechanism: the KV Cache. The historical content that has been processed can be cached and directly replicated when subsequent tokens are generated.
To solve this problem, the team introduced a block-level de-noise technique called Block Diffusion. It provides a folding path: the block moves back and forth between the blocks, and the current block simultaneously makes noise through bi-directional attention. The completed historical block can duplicate the computation results, and multiple tokens within the current block can also refer to each other and co-generate.
Sources: NLP
Therefore, Clover's parallels do not generate an entire answer at once, but combine parallels within the block with historical computational duplication.
But after solving the problem of repeated computation, there is a question: how can historical information be handled efficiently in an increasingly long context?
This is also the path taken by the regression model. In traditional full-attention long-series processing, the cost of calculating the sequence length increases by a quadratic scale. To reduce costs, the industry has developed routes such as sliding windows, sparse attention, linear attention. Gate control Delta Network (Gated DeltaNet, GDN) is an efficient structure that collects historical information through the regression state, reducing the cost of processing the long-term context.
Clover also needs to support long-term context, but can't directly reflect the design of the reverse. The reverse predicts the next Token, only reading what's already fixed on the left side; the spread model needs to allow different locations within the current block to refer to each other, and how a location is generated may also be restricted by the right side content.
In an interview, Suzanne mentioned that some schemes would retain the one-way structure of the GDN, changing only the Softmax Attention layer in the mixed structure to two-way. The team initially tried a similar modification, but the experiment found that further introduction of the information and parameters of the two-way GDN could improve model capabilities.
So the core problem with Clover is that it retains both the ability to process history efficiently and allows the current blocks to have full bi-directional interaction.
According to the technical explanation published by the team, Clover first had two pieces of information processed in different ways: the generated history, which was efficiently compressed through the regression state of GDN; and the Token within the current block, which was finite-particle interaction through the two-way Softmax Attention.
Subsequently, the model reads the historical memory as a separate unit of information, placing the token in the block at the same time as the Softmax to participate in information selection, learning when to rely on history and when to refer to the current block.
Here's another detail: the bigger the block, the more tokens involved in information selection, the weight of historical information may be diluted. For this reason, the team introduced a learning block size calibration to compensate for this change.
Local characteristic extraction was also adjusted accordingly. The original causal curve can only read left-handed content, and Clover added an independent, zero-initialized curve branch to the context on the right-hand side, allowing the model to proceed from pre-trained causal curve behavior and learn to use the characteristics on both sides of the left and right-hand side.
These transformations all point to one goal: to keep historical memories tight, to make blocks interact sufficiently well, while making information selection and local characteristics extract adaptation alongside noise.
This also shows the similarities and differences between the two routes of regression and scattering: moving towards scaling, it is necessary to address the costs of context, caching and reasoning efficiency; the scattering model also requires additional processing of the two-way interaction, and the relationship between parallelism and generating quality.
Of course, the two-way interaction within blocks is not the same as the overall planning of the entire long sequence. The question of extending this capability to a longer sequence is still to be explored. Clover demonstrated how to transform the backward accumulated architecture and engineering foundations around the distributed generation method to make parallel potentials more fully transformed into real capabilities and speeds.
Two-way attention has shifted not only to scores. In the past, the decision-making model Jev of TypeSafe accidentally turned red. It does not generate, specifically, quick, structured judgments, which reawakened the industry's awareness of the value of System 1 fast decision-making.
Unlike Jev's one-on-one, unrelated judgments, using two-way attention, the model can put up to eight associated judgments into the same noise and respond simultaneously, referring to each other and binding each other between the judgments, which teams call "joint decision-making".
In the internal verification of the 500-group, 2420-way correlation problems, the combined model set has a total parity of 80.4%, with only 62.4% of individual questions answered independently, and only 15.8% of Jev. It is also the basis of DiffuSpace's intelligent-ready System 1.
The two conditions in the same contract are dependent on the rules. The combined model allows the two judgments to be generated together, helping to reduce back-and-forth contradictions, as well as reducing the cost of re-reading the material and summoning the subjects.
2022～2026，
A team on a deadly "non-mainstream" route
Although the company was founded in 2026, DiffuSpace's exploration of this path in dLLM began as early as 2022.
At the time, ChatGPT was not yet in its infancy, and the company's founders, University of Hong Kong professor Kunming Liu, led by Dr. Hsien Loong and others, began to study the creation of text using a distributed model. The starting point of the moment was simple: first, to investigate curiosity, and second, to conceal the idea that the new architecture might bring new capabilities beyond the regression paradigm.
Looking back at the four-year trajectory, we can see that the team has been very steady every step of the way, solving one problem after another.
In 2022, they answered it with DiffuSeq, which was one of the first works to use the spread model to generate sequence to sequence text, published in ICLR 2023.
From 2023 to 2024, the team published DoT Diffusion-of-Thought-related studies, introducing thought chains into distributed generation and validating this pathway's reasoning ability in mathematical and logical tasks. It differs most significantly from self-reported CoT in that the intermediate reasoning process can be "altered" and reversed.
In 2024, the team found a key breakthrough in the development of scaling modules from continuous space to discrete space. The text is naturally discrete, and the learning and optimization difficulty of this path is much lower. After the obstacle clearance in the first half of 2025, the Dream 7B was released, and for the first time at the same parametric scale, the scaling and planning capabilities of the DeepSeek V3 with the parameter 671B were completely surpassed. This model has been downloaded over 270 times to date, which makes it possible to actually use the "Language scaling model".
The fourth question, and the most recent off-the-shelf question, is: Can you do Agent? This is Agent's big year, and the self-return model has skyrocketed in this class of tasks, and the distributed model doesn't work? Clover answers this question: terminal operations, tool calls, intelligent programming, these self-return "home" tasks, which it not only does, but does the best of all the distributed language models.
For four years, this path has not been dissuaded by anyone. In an interview, Ms. Chen said that after 2023, few people are looking in this direction, "many people don't look good", and there are voices outside urging them to do something else.
Her answer is based on the principle of primacy: the order in which regressive models are generated gradually, essentially by humans in order to simplify the training complexity of their predecessors, which is consistent with human writing habits, but it does not mean that this is the best optimization of intelligence. And according to the logic of "bitter lessons", the less precedence the model has, the higher the freedom the model learns from the data, the higher the ceiling. In other words, everything that has been done since regression today can be replicated in principle and theoretically higher.
Over the past two years, with more teams entering the field, Suzanne said "it's much better now than it was then", and Google, Inception Labs, Silicon Group, and many leading academics from around the country and abroad have been their partners. People have been filling up on dLLM shortcuts in all directions, such as LLaDA in Silicon, which pioneered the 100B scale of distributed language models; Great Britain has published extensive research on logic efficiency in open-source models; Google has pushed production speeds to a new level; Mercury in Inception Labs, although closed-source, has demonstrated for the first time that dLLM can be made into a commercial product based on API sales.
According to Hasan III, the appearance of these comrades is in itself a form of mutual recognition.
It's the beginning of the fluorescence.
What's next?
With more peers, the next question becomes more specific: where are these models used first?
One answer given by DiffuSpace is the end-side. This usually requires serving only one user, which makes it easier to play the spreadsheet model with the speed advantage under low-volume conditions. Currently, Clover has implemented quantum INT4 deployment on the 16GB RTX 5070 Ti, reaching 101.8 tokens/s in MATH-500 tests, up from 48.5 tokens/s in the control model.
The machine is also the focus of the team. Navigation, chat, and equipment control all require timely response. According to public information, Huawei and Horizon are industry strategic investors in DiffuSpace, specific locations are being explored.
Of course, the development of the entire dLLM direction is currently faced with many obstacles. One of the biggest challenges in the industry today is to truly translate theoretical parallel advantages into real efficiency. This involves not only reasoning frameworks, caching and algorithm optimization, but also chip design: the existing software hardware ecosystem revolves around the development of regression models, whereas the needs of distributed models for algorithms, storage and proportions vary.
There are also limitations on capabilities. How to further improve parallelism while ensuring the consistency of multiple tokens? How to extend the bi-directional interaction within the block to longer sequence planning capabilities?
In the future, DiffuSpace is preparing to move forward with several things simultaneously. On the model side, the next step is to scale 200 to 300B, Doka communications and infra can be leveraged from the back-to-back camp, and the real hard bone is massive reinforcement learning. Later, adding multi-models, native training data. On the product side, they work with SigmaZ to target the absorption target of 2000 tokens/s, and the self-developed Agent client frameworks and desktop clients will also come online in the near future.
Compared to four years ago, there are more and more people who believe that the spreading language model can run away. So much so that it can't be an alternative to self-return, the answer may not have to wait too long.
© THE