SLAM distilled to the edge chip: 2mW power to achieve 1000 fps gesture estimate, space perception no

SLAM distilled to the edge chip to achieve a 1000fps gesture estimate of 2mW power consumptionAlthough no single chip can achieve both of these extremes at the same time, it is not possible to achieve these two extremes.Knowledge distillation and innovation in collaboration with dedicated chip designSignificant breakthroughs have been made to bring this vision to fruition. The core path is through the distillation of knowledge, which paves the way for marginal deployment, while maintaining accuracy and significantly reducing computing and memory occupancy.

When it comes to model slimming, the distillation technology has shown a surprising effect: the lightweight model volume can be reduced.47.62%The total number of FLOPs decreased.35.52%What's even more surprising is that the student model sometimes experiments with precision against super-teachers after the model is halved.Gross margin error decreased by 88.24%，Rotation error decreased by 93.80%.This demonstrates that distillation can not only compress models, but also improve generalization capabilities, laying the foundation for ultra-low-power real-time computation.

In the exploration of the nearest limits, various technological routes have been gained.The Navion chipAs a micro-drone dedicated visual inertial mileage accelerator, it was implemented as early as 2019.2mW ultra-low power consumptionAlthough it is a simplified version of SLAM, it has verified the feasibility of spatial perception at the milwatt level.Gideon feature extractorArriving at STM32N6111 FPSIt's an accelerated GPU.GLidE-SLAMSystems are more implemented on embedded platforms9 times the rate of inflationIn addition,The Space-Mate processorAt 303.5mW, it has a capacity of 32.85 FPS.Intelligent space computing chipsThe 3.5 TOPS AI-powered and integrated power-controlled brake-to-brake schemes, powered by these soft-hard synchronous cabinets at 1W, are continuously approaching the distance to the ultimate goal.

The hardware platforms that support these breakthroughs are becoming more mature:Microcontrollers such as STM32N6)It is suitable for ultra-lightweight algorithms.It's the first time I've been able to do this, and it's the second time I've been able to do this.It is customized for SLAM and has the highest efficiency;SoCs such as NVIDIA Jetson, Qualcomm chips)Integrated CPU/GPU/NPU with flexibility and computing capabilities;Spatial computing chips3D vision is highly integrated with AI, representing the future direction. Multi-platform coexistence together forms the underlying basis of edge SLAM.

When SLAM can useMillwatt power consumptionandRate of 100 to 1000 tonnesWhen running at the edge, space perception will really reduce the prevalence of micro-drones, virtual reality convergence of smart glasses, smart mapping of robots, and unmanned patrols of underground mines.2mW+1000fpsWhile the challenge is still at its peak, the results have clearly outlined the technological path to distilling lithium + chip customized lithium. With the acceleration of the transition, it will no longer be a luxury to give the device a blink of an eye and a blink of an eye.Space is no longer a barrier.It will eventually become a reality.

Many fans ask the following questions:

Today, let's talk about this sadness together.Almost nine years3D visual technology circles.

Currently, there are nearly 6,300+ active members, mainly involved in the areas of industrial 3D vision, SLAM, autonomous driving, 3D reconstruction, drones, physical intelligence, large models, and technology fronts.

There are many sub-disciplines, including but not limited to: industrial 3D vision, 3D reconstruction, autonomous driving, physical intelligence, large models, widespread models, and technology fronts, including visual competitions, hardware selection, visual product landing experience sharing, academic & job exchange, etc. We also keep up with the latest cutting-edge technological developments and are a hot topic of discussion on our planet.

Summary of the entire series of video tutorials across three planets

Video tutorials are divided intoTwo large platesThe first block is:Core core curriculumWe've thoroughly honed nearly 20 series of video tutorials, content systems, step-by-step, and more importantly, what you've learned about 3D visual technologies like industrial 3D vision, SLAM, 3DGS, autonomous driving and drones, which we'll cover in more detail below.Technology sharing at the forefrontIn a special event, industry leaders and academic leaders were invited to share live content covering top trends such as physical intelligence, autonomous driving, 3D reconstruction, digital people, distributed modeling, 3D generation, and others. This section is not focused on the content and is no longer discussed in detail.

3DGS exclusive video tutorials

3.0.1 3DGS 3D rebuilding series video tutorials

3.0.2 SLAM algorithm video tutorials based on NeRF, 3DGS three dimensional reconstruction

This video tutorial takes you from theoretical and code realization to learning the principles of zero-entry NeRF/Gaussian Based SLAM, reading papers, coding, etc. Theoretical level, from linear algebra to traditional computer graphics, gives you a glimpse into the theoretical foundations and sources of modern 3D reconstruction; the code level teaches you how to reproduce computer graphics, NeRF-related work through multiple hands-on exercises.

3.1 Basic courses

3.1.1 High-precision camera calibration from theoretical to practical combat system instruction

This course mainly consists of two parts: camera labelling and 3D vision; camera labelling includes single-eyed, double-eyed and fish-eyed camera labelling; 3D vision includes stereo vision, structural light and TOF, based on stereo vision.Courses provide corresponding data and code。

Highlights of the CourseIn addition to the basic principles and code that the camera labels, the planet also adds:High-precision camera-labeled improved methods and real-world warfare techniques, which are quite hardcore.。

3.1.2 ROS2 from entry to live video tutorials

ROS2 provides a comprehensive hands-on instructional training on ROS2 from entry-level to real-world video tutorials, introductions and high-level presentations, and systematic learning opportunities for all.

3.1.3 Quad Flyers: Algorithms and Combat

3.1.4 C++ goes from zero-based to advanced

Try to learn through video tutorials, fully understand some of the more difficult modules in C++, master algorithms such as SLAM, learn C++ related job interview skills.

3.2 Industrial 3D visual series video courses

3.2.1 3D reconstruction of video series based on facial structures

3.2.2 Mechanical arm grabbing, 3D point cloud, 3D reconstruction and so on

3.2.3 Linear structural light (single) and three-dimensional system reconstruction tutorials

This course focuses on real-world warfare instruction, hand-in-hand with the implementation of a linear laser 3D scanning system of its own from scratch. Almost every chapter is equipped with corresponding lectures and codes to help you better understand the technology of linear laser 3D reconstruction.

3.3 SLAM series of videos

3.3.1 How to easily pinch LIO-SAM?

3.3.2 Thoroughly dissect the laser-visual-IMU-GPS fusion SLAM algorithm

This video course explains the technical difficulties and frameworks of the SLAM algorithm for laser radar-vision IMU-GPS integration from both theoretical and code implementation and, based on his years of experience in robotics engineering, explains the methods and techniques for multi-module integration in real-world robotic applications.

3.3.3 ORB-SLAM3 theoretical basis + key technical details

3.3.4 Visual-inertial SLAM: VINS-Fusion Principle Precision

The introductory and practical video tutorials for visual-inertial SLAM, combined with the VINS-Fusion source code, systematically align the basic theoretical knowledge of visual-inertial SLAM. The entire course is taught by first-rate algorithm engineers, from basic theory to code analysis, nursery-level teaching, assisting students to grow from small to large.

3.4 Self-driving

3.4.1 Multi-sensor integrated system tutorials for autonomous driving

Mainly divided into two large modules: theoretical and practical, theoretical sections mainly introduce the sensing hardware commonly used in autonomous driving, time synchronization and spatial synchronization between sensors and knowledge of information fusion rationality between multi-sensors; practical, more oriented engineering applications, how to achieve synchronization and integration between sensors in engineering, etc.

3.4.2 Detection of 3D point cloud deep learning targets for autonomous driving

This video tutorial focuses on 3D point cloud deep learning, system analysis and coding of point-based and Voxel-based 3D targeted detection network architectures to help students gain a faster insight and deeper understanding of point cloud depth.

3.4.3 Single-point depth estimation methods: theoretical and real-world video

Video tutorials are mainly divided into two main sections: theoretical and practical, and due to the difficulty of obtaining the depth of the true value of the supervised method, and the effect of the unsupervised method is almost equivalent to that of the supervised method, we focus on the unsupervised method.

3.4.4 Deep learning models in autonomous driving deploy live video

This video tutorial will use a combination of theoretical and practical thinking, first to explain the programming model of TensorRT and the relevant knowledge of GPU/cuda to the extent that it is known; then the course will use three examples of classification, detection and division to demonstrate the detailed programming process and give the relevant code to achieve a truly landing industry share.

Four 3D visual foundations coming in

Considering that many beginners may find it difficult to configure 3D visual software or a visual library, we have prepared the configuration tutorial documentation for each module for you to learn.

5 3D visual source code summary

Below is the source code for the 3D vision industry on the planet:

6 High-Quality Projects Released and Combined