Imagine a scenario like this: a natural language instruction is given to a robot, and it passes through the corridor, converges, and eventually enters a designated room. Visual Language Navigation (VLN) requires the robot to combine the images seen along the way, converting human language instruction into a series of sequential actions.
Most existing VLN methods introduce specialized spatial reasoning modules, graphical structure modeling, or layered decision-making systems to enhance navigation capabilities, but can the navigation potential of the multi-modal large model itself not be fully activated and further explored? The University of Singapore and the team at the South Pacific University of Technology in Singapore have opened up new research ideas: can we not change the core structure of the multi-modal large model, or add specialized navigation modules, relying solely on clever decision design and training processes, to make the most of the model's original capabilities, and complete navigation tasks?
Simple VLN: Towards Simple and Frontier Vision-Language Navigation With Native MLLMs
It was written by Lingyi Hong, Weiren Cai, Jiayu Chen, Xinyu Zhou, Bailin He, Yunfei Guo, Qiao Sun, Xiang Liu, Boyu Ma, Jianfei Yang, Wenqiang Zhang.
Institutions: the University of Penang, the South China University of Technology in Singapore, and the Institute of Research
Link to the paper:https://arxiv.org/pdf/2610.23203v1
The main page of the project:https://simplevln.github.io/
The code repository:https://github.com/simplevln/SimpleVLN
Figure 1 shows the study problem of the Simple VLN and its performance on two navigation benchmarks.
What do you show a robot and how do you let it go?
After the robot turns a corner, the image in front of it may have been completely different from the previous one. At this point, it needs to both judge its current position and combine the images seen before to plan its next route.
Historical images, however, are not better preserved: they are preserved all the time; only the last few lines are preserved, and early route information is easily lost.
SimpleVLN takes each model call as an independent decision, with inputs containing navigation instructions, current images, and filtered historical observations, without continuing to accumulate past conversations and actions. For historical images, the team uses a combination strategy of global + local: sampling uniformly from early observations while retaining the latest images.
Dissolving experiments showed that entering all historical images in their entirety did not improve the effectiveness; sampling the overall history uniformly did better, and continuing to increase local noise could not continue to improve navigation success rates.
Figure 2 The impact of historical tides and local tides on navigation performance.
As a result, the focus is not on how many historical images to enter, but on how to use limited images to retain critical route information.
In motion outputs, the model predicts a sequence of basic moves that are executed by the environment in succession, known as action blocks. In experiments, the action blocks increase the success rate of navigation from 45.6% to 53.6%. This shows how organizing the information used in decision-making and the moves produced by reasoning affects the performance of continuous navigation.
Figure 3: SimpleVLN directly predicts a set of continuous actions based on navigation instructions, current images and selected visual history.
It's a great way to learn how to use a deviant trajectory.
Supervision of fine tuning (SFT) allows the model to learn navigational movements from expert demonstrations. However, when performing real tasks, robots can easily go out on different routes from the experts demonstrating, encountering scenarios not seen in the demonstration data. It is difficult to cover these incidents by studying only fixed expert trajectories.
SimpleVLN uses the DAgger algorithm to incorporate trajectories generated by model strategies into the training data-building process. The research team screens the trajectory based on the degree of deviation between the strategy trajectory and the actual trajectory: when the deviation is large, the trajectory is guided by an expert; when the deviation is small, the trajectory is retained by the model itself. These trajectories are then converted into decision-level supervised samples, combined with the original expert sample, for further training of the model.
After combining R2R, RxR and ScaleVLN sample data, DAgger increased the model success rate from 57.9% to 65.4% on the R2R-CE data set and from 55.3% to 63.4% on the RxR-CE.
Training data is no longer limited to fixed expert demonstrations, incorporating the model's own trajectory of exploration, significantly enhancing the model's ability to correct errors.
Table 1 exhibits the largest increase in the success rate in the table by comparison with DAgger data.
In the meantime, we'll be able to get robots to interact with each other on the ground, and we'll be able to take navigation to the next level.
SFT and DAgger rely on motion-monitoring signal training models, but this method is limited: even if the model takes a path that is completely different from that of the expert, it can successfully reach the endpoint, and it is not possible to judge a decision better than whether a single-step action matches the reference trajectory.
The core goal of the navigation task is to complete the task in its entirety, reaching the end. One turn will change the follow-up field of view, and an entire set of actions will determine the final position of the robot. Simple VLN thus further introduces reinforcement learning (RL): no longer limited to teaching the model how to go at each step, but allowing the model to actually complete the task in a simulation environment, and then adjusting the strategy according to the final results.
The research team first constructed a completely new navigation task: setting up different starting points and target rooms in a navigation scenario, based on multi-modal models, combining corners, rooms, iconic objects and first-person views to generate supportive natural language navigation instructions.
In these new missions, the model interacts continuously with the environment by moving blocks, continuously making decisions based on instructions, current images, and historical observations until the end of the mission.
In the new mission, the model uses the continuous interaction of the action blocks with the environment, continuous decisions based on instructions, current images and historical observations until the end of the mission. Training feedback is based on the results of the entire navigation mission: judging whether the robot can stop near the target, the mission is rewarded with 1 point for success and 0 points for failure. The model does not have to repeat the expert route, as long as it reaches the destination smoothly, even if the route is different, it can also receive a direct reward.
This has also changed the way models learn navigation. Models need to execute the resulting task results through multiple strategies, gradually adjusting the choice of action in subsequent optimization.
Figure 4 Experts demonstrate that the model performs experience with newly generated navigation tasks and then enters the SimpleVLN training process.
The feedback provided by the mission results led to further improvements. For example, in the Qwen3-VL 4B model, the R2R-CE success rate increased from 65.4% to 73.8%, RxR-CE from 63.4% to 75.1%, after DAPO reinforcement learning optimization, an increase of 8.4 and 11.7 percentage points, respectively.
Despite changing the core architecture, it still delivers excellent navigation performance
The team adapted SimpleVLN to a multi-modal model of several different parameters. In the unprecedented environmental tests of R2R-CE, RxR-CE, Qwen3.5 9B achieved a navigation success rate of 76.7% and 80.1% respectively, respectively, 4.6 and 3.6 percentage points higher than Qwen-RobotNav 8B.
Table 2 shows the results of R2R-CE and RxR-CE not seen in the environmental verification set;† and * respectively corresponds to the RL before and after.
In terms of reasoning efficiency, while maintaining a higher navigation success rate, the SimpleVLN also has a lower reasoning latency. On the NVIDIA A100, the SimpleVLN, based on Qwen3-VL 4B, has a model execution time of 293 milliseconds per action block decision, which is close to the StreamVLN latency of traditional navigation methods.
Figure 3 shows the same model on the A100 for time and navigation performance.
Going into reality, physical robots understand language instructions to navigate
In addition to assessments in simulated environments, the research team deployed SimpleVLN to quadruped machine dogs. Machine dogs can walk straight, turn around, enter the room, and stop near the target, according to multi-step instructions, and exhibit strong generalization capabilities.
Mission One: Go forward along the corridor, turn left and enter the first room.
Mission 2: Move forward, turn left and continue forward, stopping next to the yellow pivot.
Summary
The core idea of SimpleVLN is not to superimpose complex dedicated network structures on multicommodal models, but rather to integrate the training data of the model's self-exploring trajectory, combined with environmental interactive learning, to fully release the continuous navigation capabilities of the native multicommodal models. The framework relies on observational screening, action block decision-making, coupled with a three-stage training program, without the need to modify the model's main roots, to achieve optimal effectiveness in simulation tasks, deployment to virtual robots, and a strong generalization in real navigation scenarios.
© THE