Do you remember The Bitter Lesson?
In the history of the development of AI, the long-term decisive victories have often been achieved not by systems of extensive expertise that have been elaborated by humans, but by general learning methods that can be expanded with computing, data and search scope.
The bitter lesson reminds us not to think of the human way of solving problems as the only correct way to compute intelligent systems.
Language models do not necessarily have to be self-regulatory. Distributed language models have long been a research hotspot. They provide another possibility through parallel noise, global revision and iterative generation.
The team officially released and open-source LLaDA 2.2, the world's first large-scale Agentic deployment model, to take the technological path of deployed language models further and officially enter the intelligent age.
Why can't the spreading language model be the mainstream route before then?
We previously reported on a study entitled "The Bitter Lesson of Diffusion Language Models for Agentic Workflows".
The research team system evaluated a set of distributed language models such as LLaDA, Dream, and concluded that the results were not optimistic: in body-planning tasks, dLLMs would repeatedly attempt the same failed action, unable to switch branches in a timely manner; in tool calling tasks, they would be difficult to maintain rigorous JSON schemes when disturbed by noise in parallel.
The best paper on ICML 2026 in The Flexibility Trap notes that one of the most acclaimed advantages of the scattering model, which is the greater clearance of spaces resulting from arbitrary sequential generation, constitutes a trap in reasoning tasks such as mathematics and code.
Models tend to bypass high uncertainty tokens using frequency order, prioritizing easy-to-fill positions, and waiting for critical decision-making points when the surrounding context has limited the answers, shrinking the space prematurely.
Both of these works address the same kind of problem as the proliferation language model and ruthlessly reveal the fact that it has been unable to cope with its real-world tasks in the past.
Distributed language models with block-parallel decoding get the speed limit that the regression model cannot reach, LLaDA 2.0 pushes this path to 100B parameters, LLaDA2.1 also runs 892 TPS on the token editing mechanism in HumanEval+, but these are all shown in static generation data.
But once you get into the Agent, the stability of the entire paradigm that requires tracking long interaction histories, calling external tools, handling environmental feedback and correcting its own behavior, starts to falter.
The end result is "one-time generation". The editing capability of traditional spreadsheet models is limited to equal-length substitutions: a token can only be exchanged for another token, a redundancy cannot be removed, and new content cannot be inserted at a specified location. Sequence length is fixed, structural rigidity. Small deviations in early blocks will solidify the context as a hard constraint on the conditions of subsequent blocks.
This is the breakthrough in LLaDA 2.2.
LLaDA2.2 Overview of the training and reasoning framework
Liu has released a technical report on LLaDA2.2 for interested readers.
Model address:https://huggingface.co/inclusionAI/LLaDA2.2-flash
The code repository:https://github.com/inclusionAI/LLaDA2.X
Technical report:
Four atomic operations to write down the noise process
First, let's look at the biggest highlights of LLaDA 2.2.
In the previous generation of LLaDA2.1 work, an editing mechanism known as Token-to-Token (T2T) was adopted. At the time, T2T editors only supported two movements: keeping (keep) and substituting (substitute).
This mechanism can correct local error tokens, but cannot address structural defects. N-gram repeatedly needs to remove redundant segments, tools to call missing parameters need to insert new content, code patches are incomplete need to extend the length, and T2T is unable to handle all three types of problems.
Levenshtein distance is a classical measure in computer science, proposed by Soviet mathematician Vladimir Levenshtein in 1965 to measure the difference between two strings: converting a sequence into another sequence requires at least four single-character operations. There are four permitted operations: substitution, deletion, insertion, and retention, which constitute a complete set of edited originals. Conversions between any two sequences can be broken down into a combination of these four operations.
The key is that there are two operations in this native language that change the length of the sequence. Deleting makes the sequence shorter and inserting makes the sequence longer.
LLaDA2.2 extends the action space to four atomic operations: KEEP (reserve), SUBSTITUTE (substitute), DELETE (remove), INSERT (insert). The latter two exist in the form of an editing control token. At position i, DELETE removes the current token and moves the subsequent token to the left, while INSERT extends the x_i to [MASK], x_i, creating a new filling position for the subsequent de-noise rounds.
The difficulty lies in overseeing the source of the signal. Between the middle draft generated by the model and ground truth, an editor tag is required for each position. LLaDA2.2 is used to calculate the longest public sub-sequence of the two (LCS) in order to derive the matching position as a point of reference: the matching point of reference is keep; in the space before the point of reference, the aligned position is substitute, the excess position of the draft is delete, and the excess position of the target sequence is assigned an insert tag to the next point of reference.
From the merger experiment: the same LLaDA2.2 baseline, the same set of SWE trajectory data, the only variable is whether to enable Levenshtein editing. On the SWE-bench Verified, 35.8 is 44.4, an absolute 8.6 percentage point improvement. For tasks such as warehouse-level software engineering that rely heavily on structural integrity, the benefits of inserting and deleting are quite significant.
This was the first time in the industry that Levenshtein edited a large-scale integration into a scattering model. The scattering model was transformed from a static generator to a dynamic system with the ability to generate intermediate self-correction. This marked the first time that a scattering model truly had the ability to participate in sequential tasks.
L-EBPO: Let environmental feedback drive editorial decisions
After being able to edit, the next question is when to edit, where to edit.
This is a decision-making question that is beyond the scope of grammatical judgement, and only in the real world can the right answers be found: whether tools call for errors, whether formatted tests pass, whether tasks stagnate.
The fundamental difficulty of doing RL in dLLM is to calculate the likelihood that a generated trajectory contains multiple blocks of noise, that the model is not decided on a token-by-token basis, that precise sequence levels are difficult to solve efficiently, and that the tactical gradient method requires the likelihood ratio of the new and old tactics.
The name of the LLaDA2.2 program is long: a sub-block strategy optimization method based on Levenshtein's editing evidence bottom line, abbreviated as L-EBPO.
The structure of L-EBPO is a two-tiered control system. The outer layer is governed by EBPO, which optimizes trajectory-level decision-making across Agent interaction rounds, managing the strategic direction of the entire interaction link; and the collage editing within the internal layer management block, i.e. when and where DELETE and INSERT are applied in a single generation step.
The time scale of the two layers varies greatly, one in series and one in noise steps. L-EBPO's practice is to unify the two layers into the same target function by expanding the action space:
Token prediction and editing decisions are thus incorporated into the same target function optimization.
The multi-cycle Agent trajectory differs from the single-cycle response, and if a spreadsheet contains multiple-cycle model output and tool returns, all rounds other than the first-cycle must be disguised, otherwise all attention within the block will result in information leakage.
The reward signals come entirely from environmental feedback, which consists of a combination of three elements: the correctness of execution, the legality of the output format, and the overall completeness of the task.
The direct consequence of reward design is that the model's editing strategies are environmentally calibrated: what types of deletions and insertions can improve the success rate of tool calls, and what types of modifications can allow the format to be validated, all of which are given feedback by the execution results, without relying on preset rules.
Traditional error-correcting methods can only do coverage corrections on the wrong surface, and cannot address structural defects that need to change the sequence topology. L-EBPO offers another path: a set of environmental feedback-driven self-correcting strategies that can actually eliminate and replenish errors from the trajectory to block the transmission chain in long-range interactions. It really solves the structural problem posed by The Flexibility Trap.
128K in the original context, with a fixed capacity pool of experts
The Agent mission has a natural requirement for the length of the above context, and the above context is the "buffer of ammunition" for the intelligent body to execute the task, which is the weak point of the dLLM in the past.
The starting point for LLaDA2.2 is LLaDA2.1's 8K, targeted at 128K, 16 times the bandwidth. Contextual expansion does not just enlarge the parameters of the window.
To this end, the research team took a two-step approach: first, 300B tokens on 64K lengths and then extending to 200B tokens on 128K lengths.
The organization that trains the data is also concerned. This phase is mainly document-length and warehouse-level code, focusing on effective length samples that actually fall within the range of 64K to 128K. When spelling, document-perceived packing is used as a boundary between attention and attention, meaning that each document focuses only on its own content.
This is particularly important for the spread model: the focus of the spread model is two-way, and if there are no boundaries, unrelated documents in the same batch will "see" each other, and what the model learns will be false two-way dependencies.
Agent data is arranged to be added in the final 128K phase, including long software engineering context, tool use trajectory and browsing trajectory. This sequence is intentional: first the model exercises the basic capabilities of long context and then teaches it to do Agent in long context.
More tricky problems arise in the MoE architecture, and this problem is specific to block spreads.
MoE works by activating only a small number of specialists per token, such as eight from 256 specialists, so that the single calculation can be controlled while maintaining a large number of parameters. Since the regression model only generates one new token at a time, the activated specialists naturally generate around that eight, and the load is stable. However, block spreads have to process the entire block in parallel, usually 32 or 64 tokens. If each token in the block is independently selected by an expert, then this block actually needs to be loaded by an expert, which is the sum of all the tokens selected.
If each token is routed separately, the set of experts touched by a block is a combination of all the token options, resulting in an increase in HBM traffic, a decrease in the rate of expert authorization duplication, an increase in the cost of dispatch/combine communication with experts in parallel. In order to ensure the model's throughput, delay with the first token time, the team adopted a design called Block Routing.
Block-diffusion MoE shows the block routing process.
Block Routing's design was based on an observation by the team on the LLaDA2 series: Although the token-level routing as a whole is decentralized, the preference of experts within the same block is highly concentrated, with a few experts taking over most of the routing traffic. Accordingly, a fixed-capacity pool of block-level experts is a reasonable choice for block spreading reasoning.
The specific approach is to align routing units with distributed generation units. Block-level access scores are calculated using token racing strategies:
This means that each token has the opportunity to nominate its own preferred expert. The block then accesses top-C experts according to g, forming a pool of fixed-capacity experts P.
The design provided a predictable expert workflow for each block over the O (C) boundary, improved cache planning and expert parallel execution; while the token-level selection remained in the access pool, the model did not degrade the route into a single block-level decision, and the token-level expertise was maintained. The technical report said that switching to block routing after post-training did little damage to model quality.
The context of development really determines the value of distributed model engineering deployments in the field of smart bodies. Once both conditions and costs are met in the context above, model capabilities can be transformed into engineering capabilities that can be deployed in cost-sensitive scenarios.
Competing positively with the self-return model
The evaluation covers 17 benchmarks, including 7 Agent classes and 10 General classes, and compares with the Ling-2.6-flash self-return model of the same parameter.
On the Agent 7 scale, LLaDA 2.2-flash averaged 53.83, and Ling-2.6-flash was 55.74, a score very close.
• τ2-Bench ((Multicycle tool - Agent - user interaction): 80.33 vs 76.36, lead
• PinchBench ((real-life AI coding benchmark): 81.66 vs. 81.30, leading
MCP-Atlas (Real MCP server tooling capability):46.21 vs 41.12, leading by 5.09 points
Claw-Eval (autonomy): 64.22 vs 64.56, which is essentially level playing field.
LLaDA 2.2-flash is very competitive in the use of interactive tools.
Contextual capability was also verified. Among the 10 general benchmarks, the LLaDA2.2 flash on LongBench v2 was 45.13 versus the 42.94 over-Ling-2.6-flash, matching the goal of context expansion on 128K.
Efficiency is a traditional strength of the spread language model, and LLaDA 2.2-flash achieves ultra-high efficiency without compromising benchmarking capabilities.
BF16 has an average throughput of 1.64 times that of Ling-2.6-flash; an additional 18.6% improvement after FP8 quantification, and the heavier the task, the longer the context, and the greater the advantage of spreading parallel decoding.
This is a double proof of efficiency with Agentic's capabilities: performance and top-of-the-range auto-return models are in the same range and lead in critical interaction tasks, while throughput is 1.64 times that of each other. In cost-sensitive Agent deployment scenarios, this combination has direct real value, especially Agent tasks that often require dozens of rounds of reasoning in a single session, and multiple differences in throughput can gradually accumulate into significant end-to-end delays and cost gaps.
For more information, please refer to the original paper.
What is the meaning of this verse?
The spreading language model has always been a biased one. The enormous advantage in efficiency seems to have been replaced by absolute ability.
But when LLaDA2.2 demonstrates that a distributed language model is capable of performing intelligent body tasks, its absolute efficiency advantages cannot be ignored at all.
For the team evaluating the Agent Technical Route, LLaDA2.2 has become more than just a research sample, but a basic model option that can be seriously placed on a candidate list.
The competition between returning and spreading these two routes will continue for a long time. But it is certain that the Age of Agents is redefining what constitutes a truly excellent model. It needs not only a generator that can give a beautiful answer, but a performer who can continue to judge, detect errors, correct directions, and always remember the end goal.
Under this new evaluation standard, a self-modifying model may be just beginning to show its true imaginative space.