Paper information
Title:Vision-Guided Outdoor Flight and Obstacle Evasion via Reinforcement Learning
Authors:Shiladitya Dutta, Aayush Gupta, Varun Saran, Avideh Zakhor
Institutions:UC Berkeley
Readings
The paper proposes a visual guidance framework for autonomous flight and obstacle avoidance based on reinforced learning. The system consists of a pre-trained coding-only control network and an LSTM programming network that uses a solid vision depth chart and a visual inertial mileage meter as input and direct output of the drone's speed control commands. The training uses a two-stage strategy: the first using privileged learning (privileged learning) to accelerate strategy convergence in simulation, and the second through randomization and incentive plastics to migrate strategies to settings that rely solely on visual observation.
Demonstration of Effects
Figure 1 Actual test platform: The Xinjiang M300 drone is equipped with Zed2i sensors for depth estimation and visual inertial mileage measurement (VIO) and is equipped with Jetson AGX Xavier as a computing unit.
References
Drone autonomous flight is a central research topic in the intersection of robotics and artificial intelligence. With the rapid development of deep learning and reinforcement learning technologies, learning-based flight control strategies have gradually shown the potential to transcend traditional methods. Compared to traditional methods based on geometric models and artificial rules, learning-style strategies can automatically detect complex perceptual-decision mapping relationships from data, with greater adaptability and robustness in the face of unseen environments.
In recent years, there has been significant progress in drone navigation based on deep reinforcement learning (DRL). Some methods use end-to-end deep networking directly from the original image learning control strategy, while others use modular design to separate perception and planning. However, most existing work is focused on indoor environments or structured outdoor scenes, and the performance of these methods tends to decline significantly when faced with the complexity of real outdoor environments (such as lighting changes, dynamic barriers, lack of textural areas, etc.). Simulation training and sim-to-real migration is currently the most mainstream training method, but how to effectively bridge the gap between reality and reality remains a central challenge in the field.
The limitations of existing methods are mainly reflected in three aspects: firstly, insufficient visual perceptual robustness, which can lead to profound estimation errors in areas of light changes and lack of texture; secondly, the strategy's limited generalization capability, which can fail in an off-distributed training environment; and thirdly, the inefficiency of end-to-end training samples, which require interaction data directly from visual inputs into simulations to learn flight strategies.
The core idea of this paper is to use privileged learning to accelerate the training process and to enhance the strategy's generalization capabilities through sophisticated domain randomization and reward plastics. Privileged learning allows for the use of additional information during the training phase (such as ground real-value depth) so that strategies can quickly learn effective avoidance behaviors, and then rely solely on available visual input during the deployment phase. Combined with the visual characteristic compression of the encoder and the time modeling capabilities of the LSTM, it ultimately achieves efficient real-time reasoning and strong zero-sample migration capabilities.
Major contributions
The main contribution of this paper includes the following three aspects:
• Proposed a two-stage reinforcement learning training framework, the first using privileged information (ground real-value depth and obstacle location) to accelerate strategy learning, and the second through domain randomization and reward plastics to migrate strategy to a setup that relies solely on visual input, effectively solving the problem of low-efficiency of end-to-end visual navigation training samples.
• Designed with a pre-trained coder-based and LSTM-based perception-planning-control network architecture, since the coder compresses high-dimensional depth graphs into compact features, the LSTM uses timeline information for smooth speed command prediction, achieving efficient end-to-end mapping from visual input to control output.
• Zero-sample migration of reinforced learning flight strategies in real-world outdoor environments, successfully deployed to obstacle types, environmental layouts and drone hardware platforms never before encountered in training, demonstrating unprecedented generalization and real deployment value.
The method
The methodology framework contains three core components of the perception module, the strategy network, and the training process. The perception module uses a 3D camera to obtain depth maps, combined with VIO to provide unmanned position estimates. The strategy network consists of a pre-trained coding-based control network and an LSTM programming control network that receives depth chart characteristics and target directions as input, output, forward speed, side-to-side speed, and perpendicular speed.
Pre-training of the self-encoder is one of the key steps in the method. The authors first train the self-encoder on large-scale synthesis and real depth charts, learning the coding ability to compress high-dimensional depth charts to represent low-dimensional compact features. The post-encoding features retain the spatial distribution information of the obstacle, while significantly reducing the input dimension and computational burden of the subsequent LSTM network. The LSTM network receives the coding character sequences and target direction vectors, using their timing to model the changing trends in the capturing environment and output smooth speed control commands.
The training process is divided into two phases. The first phase (privilege learning phase) uses privilege information such as ground real-valued depth and obstacle location as additional inputs to the strategy network in a simulated environment, through enhanced learning training using PPO algorithms. The privilege information helps strategies to quickly understand environmental structures and obstacles, significantly improving training efficiency. The second phase (migration phase) removes all privilege inputs while preserving only visual observations, while introducing randomization (automated photography, textures, obstacle shapes and sizes, etc.) and carefully designed rewards for plastic form near-target rewards, obstacle avoidance rewards, speed smoothing rewards, etc.) so that strategics can only make the right flight decisions from visual input.
The results of the experiment
Experimental assessments were conducted in simulated and real outdoor environments respectively. Simulated experiments using AirSim constructed a variety of outdoor scenarios, including forests, urban streets and open fields, containing static and dynamic obstacles. Real experiments were conducted in outdoor environments such as campuses and parks, testing the strategies' generalization capabilities under unprecedented environmental layouts and obstacle types. Contrast methods include traditional geometric planning methods and existing learning-based navigation methods.
The experimental results showed that this method was significantly superior to the counter-method in both simulation and real environments. In zero-sample migration tests in outdoor real environments, this method successfully completed autonomous avoidance flight tasks in all test scenarios, far exceeding other learning-based methods. It is particularly noteworthy that the strategy still excels in environments never seen in training and on unmanned platforms, verifying the effectiveness of domain randomization and privileged learning strategies.
Dissolving experiments verified the contributions of the components. After removing the privileged learning phase, the training time of the strategy significantly increased and the final performance decreased; after removing the domain randomization, the strategy significantly deteriorated in real-world performance; after removing the LSTM, the strategy output speed commands were severely shaken and the flight trajectory became uneven. These results showed that the two-phase training framework, zero-domain randomization and timing modeling were indispensable for achieving the sample migration of the robot.
Summary & Future of Work
The paper proposes a visual guided autonomous outdoor drone flight framework based on reinforced learning that achieves zero-sample migration from simulation to real outdoor environments through two-stage training strategies: privilege learning + domain random migration. The combined architecture of the pre-trained encoder and the LSTM ensures efficient mapping of visual input to speed commands, demonstrating a powerful generalization capability across unprecedented obstacle environments and drone platforms, and an important step towards the real deployment of learner-style drone navigation.
Future work will explore the following areas: the introduction of more sophisticated perceptual models (such as light flow, semantic segmentation) to enhance the robustness of strategies under complex visual conditions, the study of enhanced learning methods for avoiding multi-drone collaboration, and the expansion of the framework to faster and more dynamic flight scenarios.
Readers interested in more experimental results and article details can read the original paper.
This article is for academic sharing only. If there is an infringement, please contact the deleted text.