WAM-Diff2: Hierarchical AR-to-Diffusion Distillation for Highly Efficient Autonomous Driving VLA
Link to the paper:https://arxiv.org/pdf/2608.01035
This is the first time I've seen this.
Author's Unit: University of Penang, directed by Intelligent Technology Limited
Figure 1: Overview of WAM-Diff2's multi-tasking performance, decoding efficiency and long-term planning results.
Readings
The VLA model has become the mainstream technology route for end-to-end autonomous driving, but the existing architecture based on self-return AR decoding faces a double structural bottleneck: first, sequential generation of tokens leads to delayed sequential length linear growth, severely constraining real-time deployment; second, the training phase relies on the teacher forcing mechanism of the real frontrunner, which exposes the model to errors generated by itself during reasoning, causing exposure bias accumulated over time.
The scattering model supports multi-location parallel updates and can repeatedly modify intermediate results in multiple rounds of de-noise, which is a natural choice to break the above bottleneck. However, existing scattering strategies largely revolve around single planning tasks from scratch, and cannot inherit the semantic knowledge and multi-tasking reasoning skills that have been learned since the return of the model on a large scale. A key question arises from this: can mature multi-tasks from the return of VLA smoothly migrate to parallel scattering architectures without relying on head training?
To address this problem, the University of Padang together with Intelligent Vision has proposed WAM-Diff2, a multi-tasking discrete spread VLA framework based on three-tiered distillation. Its core contributions can be summarized in three ways:
1. General architectural transformation paradigm: the smooth migration of multi-tasks from the VLA back to the discrete spread model was systematically achieved for the first time, with the complete preservation of semantic knowledge and unified interfaces, rather than degradation to single-task spread planners;
2. Block-Causal Attention and Three-Scale Gradual Distillation: to propose a Block-Causal Attention Mechanism as a continuous bridge between linking factor generation and bi-directional generation, and to design a three-scale strategy for progressive block-level adaptation, paradigm-consistent block-level distillation and cross-scale model-level distillation to ensure optimum stability while eliminating exposure biases;
3. System-level efficiency and multi-tasking functionality uniformity: 2.8 times decoding acceleration is achieved by generating paradigm conversions alone, overlapping FlashInfer core optimization with CUDA Graphs with cumulative acceleration of 15.1 times after execution of chart packaging, and driving comprehension, visual perception and motion planning performance are evenly matched to the auto-return baseline, and long-range planning is more robust.
A: The background of the study:
Efficiency of self-return VLA with rubber bottles
The VLA model unifies natural language responses, target positioning and future trajectories into Token sequences, enabling a single network to simultaneously undertake driving scenarios understanding, visual perception and motion planning, and has become an important technological route for end-to-end autonomous driving. However, the mainstream driving general model is largely built on regression vision - there are structural constraints on top of language models.
Efficiency bottleneck: The computational complexity of sequential decoding by Token increases linearly with sequence length, with high rationale for delay directly constraining real-time deployment. In autonomous driving scenarios, delay differences of milliseconds may mean areas of safety and accident.
Rubatic bottlenecks: the training phase relies on the teacher forcing mechanism, with the model using real forecasts as inputs at each step; while the reasoning phase is exposed to the output itself. This training - the fallacy of reasoning distribution - leads to a continuous accumulation of early errors along the length of the sequence, i.e. the exposure deviation problem. In long-range trajectory planning, this deviation increases with the number of predicted steps, severely weakening the reliability of the execution of the closed loop.
The scattering model can theoretically mitigate the two major bottlenecks by simultaneously modifying noise and multipurpose. However, since scattering strategies are mostly trained around a single planning task from scratch, it is difficult to inherit the semantic knowledge accumulated in VLA in driving scene understanding, visual perception and multitasking reasoning. If you switch directly to large block scattering, experiments show that the NAVSIM PDMS will drop from 88.1 to 84.1, proving that the architecture transformation cannot be kicked, it must proceed gradually.
The core idea of WAM-Diff2 is to provide a replicable, low-cost architectural conversion pathway: fully preserving the mature knowledge and unified interfaces learned from the return of VLAs, through Block-Causal Attention and three-tiered distillation, to smoothly convert Token-by-Token rewriting to parallel noise, thus achieving unification of multitasking cognition and parallel execution efficiency.
Second, the core method:
Differential diffusion model with three-tiered distillation
2.1 Block-Causal Attention: Continuous bridges generated in connection sequence and generated in parallel
WAM-Diff2 is based on the Qwen3-VL backbone network, which divides the output sequence into several decoding blocks. The tokens within the block can be interacted in two directions and corrected in parallel, while the blocks remain causal. The block size B thus becomes a continuous adjustment variable for connecting two generating paradigms: B = 1 when equal to the standard auto regression decoding, and B is the larger the tokens that are processed in parallel once.
Natural language queries, 2D detection, and motion planning are all sequentialised into the same Token space without introducing any task specification. Therefore, the conversion object of this method is a complete VLA covering understanding, perception, and planning, rather than a single trajectory generating module. This design ensures the completeness of multi-tasking capabilities during architectural migration.
Figure 2: The overall architecture of WAM-Diff2 with a three-stage AR-to-Diffusion layered distillation process.
2.2 Tier 3 graded distillation: the generating paradigm of gradually changing models
Architectural transformation faces a dual paradigm gap between attention patterns (causal and bidirectional) and optimization goals (next-token prediction and masked denoising). WAM-Diff2 achieves a smooth transition through a three-tier forward strategy:
Stage one: Progressive Block-wise Adaptation
Expand the block size by 1→4→8→16→32 Gradually, each level is initiated with a first-level weighting. The model gradually learns a wider range of bi-directional dependence in small structural changes, avoiding the optimization collapse caused by large-scale switching, ensuring the stability of the attention mechanism conversion process.
Stage 2: Conventional Block Distillation (Block-wise Distillation)
The teacher guides the larger student models with stable small-block scattering. The teacher and the student make predictions on the same damaged sequence and use the symmetrical Jensen-Shannon scattering (JSD) to align Token distribution. Compared to previous mode-covering behaviours that are easily triggered by KL scattering, the JSD is better able to retain the multi-module trajectory distribution characteristics.
Step 3: Cross-Scale Distillation by Model-wise
Using the 8B, Block-32 spread model, 2B, Block-32 students were guided. The study found that the Top-5 Token weighting ratio of 8B spread teachers and 2B students was 58.2%, significantly higher than the 51.2% of 8B recruitment teachers; the corresponding NAVSIM PDMS was 88.3 and 79.9, respectively. The results showed that the consistency of the student-generated paradigm is also a prerequisite for efficient knowledge transfer.
Figure 3: Degree 3 distillation, distillation loss, Token concentration and dissolution results of the teaching method.
Three, system-level acceleration:
Efficiency jumps from 2.8 to 15.1 times
Under the Unified Multi-tasking Model, the standard achievement of Block-32 was 124.8 Tokens/s, up 2.8 times from the 44.5 Tokens/s from the back-to-baseline. On this basis, the team customized the FlashInfer kernel for the Block-Causal Attention Mode to further accelerate 1.7 times through enhanced shared memory throughput; and then used CUDA Graphs to encapsulate the execution chart of fixed de-noise steps to eliminate CPU by-calculator deployment and then accelerate by 3.1 times.
The decoding delay decreased from 22.7 ms/token to 8.1, 4.6 and 1.5 ms/token respectively, resulting in a 15.1-fold cumulative acceleration, with peak throughput of 673.4 Tokens/s, while the movement planning indicator varied by no more than 0.5 percentage points. The experiment further demonstrated that the performance of NAVSIM, DriveBench, COCO and LingoQA tended to collapse largely at 816 de-noise and near-noise steps, and that deployments could achieve adjustable performance controls based on the dynamic configuration budget of the precision delay demand.
Figure 4: Changes in performance under different noise steps, as well as the speed and accuracy of Pareto curves on multiple parameters.
Four: The results of the experiment:
Single models retain multi-tasking capabilities, while long-term planning is more robust
WAM-Diff2 was evaluated on five benchmarks: DriveBench, LingoQA, COCO, NAVSIM and Bench2Drive. Under the Unified Multi-tasking Protocol, all results came from the same freeze checkpoint: 2B, Block-32 models obtained 48.80 DriveBench, 65.80 LingoQA, 36.30 COCO mAP, 87.44 NAVSIM-v1 PDMS and 87.50 NAVSIM-v2 EPDMS, while the standard decoding throughput was increased from 44.5 to 124.8 Tokens/s.
Table 1: Comprehensive results of the unified multitasking model in understanding, perceiving, planning and decoding efficiency.
In the mission-specific NAVSIM setup, Block-32 achieved 88.3 PDMS and 88.6 EPDMS; combined with candidate trajectory scores, it was further upgraded to 91.1 and 90.7, better than advanced dedicated planning models such as ReCogDrive (90 PDMS) ‒DriveVLA-W0 ‒90.2 PDMS. In the Bench2Drive closed-circuit simulation, the model achieved a 49.55% success rate and a 78.93 driving score, and maintained a strong performance in critical safety scenarios such as emergency braking (61.67%) ‒ traffic sign adherence (85.26%) ‒ demonstrating the mitigating effect of distillation frame logic on the exposure bias.
Table 2: WAM-Diff2 results of task specific planning on NAVSIM-v1/v2.
Table 3: WAM-Diff2 results in closed-circuit and multi-capacity tests in Bench2Drive.
Figure 5: WAM-Diff2 results in qualitative visualization of driving understanding and visual perception tasks.
Figure 6: Typical failures of WAM-Diff2 on Bench2Drive.
The long-time field error experiment further revealed the value of the spread-generated value. The team compared the L2 error at future routes in 12,146 NAVSIM paired samples: the average error from the return baseline was 0.5935 meters, the WAM-Diff2 was 0.5589 meters, a decrease of 5.8% overall; the gap between the two was extended from 0.002 meters single-mode at the first path point to 0.082 meters at the eighth path point.
Figure 7: Reduced errors and errors at the L2 trajectory point from the return baseline to the WAM-Diff2.
Summary and prospects
WAM-Diff2 demonstrates a technological path that differs from training the scattering model from scratch: understanding, perception, and planning capabilities that are mature since the return of VLAs, and the smooth migration of those capabilities to parallel executable discrete scattering architectures through Block-Causal Attention and three-tiered distillation. While achieving up to 15.1 times decoding acceleration, the model as a whole retains its multi-tasking capabilities from the return of the baseline and demonstrates stronger error-fighting capabilities in long-range planning.
It should be noted that discrete Token conversion may introduce spatial quantification errors, and the ability of students to disseminate is limited to the level of semantic reasoning of returning instructors, which also constitutes an improvement in subsequent work. Overall, the framework does not rely on specific underlying models, forming a low-cost, scalable general-purpose conversion pipeline that can be directly migrated to existing and future advanced returning VLAs, providing a viable solution for deployment of automated multi-mode intelligence under strict ingestion constraints.
