Diffusion Large Language Models (dLLMs) provide a new technological route for efficient code generation with the ability to generate multiple tokens in parallel, particularly in efficient test use cases, to rapidly increase code coverage. However, diffuse language models still face a critical problem: faster generation often leads to decreased generation quality.
To address this issue, the paper presents an accelerated framework for unit testing for distributed language models, DiffuTester. Its core idea comes from a simple but important phenomenon: multiple unit tests generated by the same measurement method often share a large number of similar code structure patterns.
DiffuTester uses the Abstract Syntax Tree (AST) dynamics to extract these structural patterns and retain additional tokens corresponding to these structural patterns during the scattering decoding process, allowing the model to decode more tokens in one noisy step, reducing overall logical steps.
Experiments have shown that DiffuTester achieves a stable acceleration effect on different distributed language models as well as on three programming languages: it can achieve a maximum of about 2 tons3x of generating acceleration while maintaining test coverage and significantly reducing computational costs. The relevant paper has been accepted by EMNLP 2026 and the code is open source.
This study was completed by the AI Agent group at the Institute of Artificial Intelligence of the University of China. The author of the communication is assistant professor Li Xia of the Institute of Artificial Intelligence of the University of China, who mainly studies the development of intelligent software.
DiffuTester: Accelerating Unit Test Generation for Diffusion LLMs via Mining Structural Pattern
Link to the paper:https://arxiv.org/pdf/2509.24975
The code link:https://github.com/THU-Agent/DiffuTester
Background of research
Distributed-language models predict multiple tokens by mutating noise in parallel, and have a stronger parallel generation potential than traditional auto-return models, which is why they are gaining increasing attention in efficient code generation scenarios.
However, there is a clear speed-quality trade-off between dLLM's generating efficiency and generating quality: if each de-noise step retains only a small amount of high-confidence tokens, generating quality is more stable, but parallel advantages are difficult to fully realize; if the number of tokens retained at each step is simply increased, decoding can be accelerated, but generating quality will decrease.
This problem is particularly relevant in unit test generation. Software projects in the real world typically contain a large number of methods that require testing, and developers need to generate multiple test use cases for them to cover different statements and branches, so UTG has a natural need for generating efficiency, and the parallel generation capabilities of dLLM are well suited to this scenario.
Further observations have found that unit testing itself also has a characteristic that is very suitable for dLLM acceleration: a large number of iterative structural patterns, as shown below. Therefore, the core starting point for this article is to: use the widely available iterative structure in UT to help dLLM decode more tokens at each step, accelerating generation without sacrificing test quality as much as possible.
What is the solution?
The central question of this paper is how to accelerate the unit test generation process of dLLM without reducing the quality of production.
The existing training-free acceleration methods of dLLM can be roughly divided into two categories: caching-based methods and sampling-based techniques. The former reduces the cost of one-step reasoning by duplicating similar intermediate representations or KV Cache between different de-noise steps; the latter mainly starts with the sampling strategy of the decoding process by decoding more tokens, dynamically adjusting the sampling process, and so on.
Both methods can effectively increase production efficiency, but they also generally lead to a decrease in production quality. DiffuTester is also an optimization of sampling strategies, but DiffuTester not only does not reduce production quality, but is also compatible with other caching-based methods.
What method was suggested?
In response to the above, DiffuTester: an accelerated framework aimed at dLLM unit testing generation, training-free. Its core idea is to: dynamically explore shared structures between multiple test use cases and use these structures to help decode more tokens, thereby reducing overall noise levels.
As shown below, in each decoding step, DiffuTester first executes the dLLM's original confidence-based decoding, retaining the most robust part of the model token; then further executes the structure-based pattern decoding. Specifically, DiffuTester builds an AST for multiple test use cases that have not yet been fully generated in the current batch and identifies the shared structure in them by merging different ASTs. The tokens corresponding to these public structures will be retained in addition so that the model can decode more content at a time of reasoning.
Since scattered models may contain grammatical errors in generating early code, DiffuTester does not construct ASTs directly for complete test use cases, but uses a step-by-step analysis method to reduce the impact of local grammatical errors on structural drilling.
Additionally, DifffuTester uses two auxiliary designs: one is to maintain confidence only in tokens that are higher than a default threshold to avoid low confidence affecting code quality; and the other is not to perform structural analysis of AST at each de-noise step, but at intervals to further reduce additional expenses.
In general, DiffuTester can be summarized as:
On the basis of the original confidence decoding, structural information from multiple test use cases is introduced and more tokens are decoded at a time to achieve more efficient dLLM unit test generation.
Test results and analysis
In this paper, we experimented with two representative dLLMs, DiffuCoder and Dream, and tested on the TestEval dataset, and in order to verify the effectiveness of the method in different programming languages, we also extended the original dataset to three different programming languages, namely Python, C++, and Java.
Principal experimental results: Significantly accelerated while maintaining test quality. DifffuTester can steadily improve production efficiency in different model and programming languages without affecting peak coverage.
DiffuTester achieved a better balance of speed and quality compared to other sample-based acceleration methods: speed and quality.
Compared to caching-based acceleration methods: overlapping. Another common type of dLLM acceleration method is the KV Cache-based method. DiffuTester mainly reduces the decoding steps that need to be performed, while caching methods mainly reduce the computational cost of each logical step, so the two are naturally compatible.
