Multimodal models, world models, and body models are continually reshaping the boundaries of AI's capabilities, but a more basic problem has long been overlooked: visual degradation in the open world; high-speed motion blurring, frequency overlapping, high-light over-exposure, flash interference... Complex degradation phenomena in these real-world environments can lead to image distortion and information loss. Lack of reliability makes this data difficult to use in traditional supervised learning and often becomes a blind spot in AI training. Therefore, even stronger models, AI systems can still make mistaken judgments due to errors or lack of visual information. How to obtain high-quality visual information under limited hardware resources, providing a reliable perceptual basis for machines and intelligence, has become a key issue in the development of physical AI.
Following the release of the cover of Nature, the work related to the celestial sphere goes up to the cover of Nature Sensors Seeing through visual primitives
In 2024, a team from the Center for Brain Computing Research at Tsinghua University developed the Zhenjiang Zhenjiang chip on the Nature Zhenjiang cover, which first proposed a new model of brain complementary visual perception based on visual primitive language, and developed the world's first dual-track complementary visual perception chip based on native language, the Zhenjiang Zhenjiang chip, which provides a new technology for machine vision that differs from traditional RGB imaging.
Two years later, the team's latest advances in this direction once again appeared on the cover of Nature Sensors.
If the work completed in 2024 to complete complementary vision from 0 to 1 perceptual paradigm and chip hardware innovation, then the study achieved a system-level algorithm software ecological leap from 1 to 10 tons. The study drew on the self-monitoring character building mechanism of the human visual prototype with the Internal Model to expand complementary vision into a complete technological system of learning, migration, scaling up. Specifically, the study established a simultaneous methodology for constructing visual prototypes based on cloud data, and proposed a methodology that directly reversed the real-valued decayed data monitoring of visual prototypes without real-valued values, allowing the model to learn effectively from real-valued data and to migrate more efficiently to different visual tasks.
The paper was co-authored by Prof. Shruping and Prof. Liu, Professor of the Department of Precision Instruments at Tsinghua University; co-authored by Dr. Lin Liyang, Senior Assistant Professor in the School of Intelligent Manufacturing at Tsinghua University; and a PhD student at Tsinghua University. In addition, the first author of the Nature of Tsinghua cover article in 2024, Dr. Wang Liu, CTO of Tsinghua Technology, and the core members of Ph.D. students Meng Xiaoming and Chen Chang also made important contributions to the work.
As of now, mainland China's institutions have been the first to achieve this milestone in the field of AI and chips, with only three landmark achievements in the field of AI-Nature-based publications, namely Tencent's Tencent, Tencent's Tencent and DeepSeek-R1. Of these, the Chinese University's Cluster of Brain Computing Research Center has completed the first two. From Tencent's Tencent to the introduction of heterogeneous fusion of brain computing chips, to Tencent's Tencent's introduction of complementary visual novelties, to the continuous improvement of complementary visual algorithms, software and open source ecosystems to this day, the team has always adhered to the principles of evolution, exploring new types of intelligent computing and perception.
Rebuilding self-monitored vision under complementary vision sensing: high-speed high dynamic range imaging instances to achieve high-performance machine vision with low bandwidth, low hardware costs
From 0 to 1: Reacting to visual degradation from perceptual input
In 2024, Liu Nature developed a new model of complementary brain visual perception based on native human visual mechanisms, which dismantled visual information in the open world into visual native language representations, and by combining these native languages, leveraged the core mechanisms of the human visual system to form the two advantages of complementary visual information and complete visual sensing pathways that allow intelligent systems to both achieve high-precision recognition and rapid response to movement. Based on this model, the team further developed the world's first-class brain complementary vision chip, the Liu Chen chip, which breaks down the signal signal into three complementary data models: cognitive orientation pathways (PGBD) RGBD (RGBD) RGBD (RGBD) RGBD (RGBD) RGBD (RGBD) RGBD (RGBD) RGBD (RGBD) RGBD (RGBD) RGBD) RGBD (RGBD) RGBD (RGBD) RGBD (RGBD) RGBD) RGBD (R) RGBD (R) RGBD) RGBD (R) RGBD) RGBD (R) RGBD (R) RGBD (R) and RGBD (RGBD) RGBD (R) RGBD (RD) RGBD, but can also be used to distinguish between high-speed distances and distances, but also to differentiate between distances and distances and distances. Multiple visual native languages complement each other, allowing AI systems to simultaneously acquire evidence of color, structure, and change, forming an effective complement between different degradations. The chips ultimately measure extremely high performance indicators and excellent live effects, becoming an important starting point for exploring new paradigms of visual perception.
From 1 to 10: Let the AI know in the degraded message who's going to write the letter.
Multi-way perception is just the starting point. In the open world, visual degradation is often overlapping, and the extent to which signals are damaged varies. The real difficulty is whether algorithms can judge credibility when the evidence provided by different visual pathways conflicts and form a generalizable internal visual model to form visual quality. In 2026, the Nature Sensors hypothesis drew on the inherent mechanisms of the human brain based on visual language to achieve sound perception, constructing two-stage types of brain perception learning patterns, and building internal models based on self-monitoring and complementary visual intuition.
The types of brain vision sensing proposed in this paper - the self-supervised learning framework for perception: building internal models from the bottom up, modeling sensing tasks from the top down, and achieving robust visual perception based on the Tianjin Core that is open to world data efficiently.
At the bottom-up stage, the algorithm does not rely on non-reality perfect image frames, but draws on human and inherent visual native language to embed different visual precursors into a set of intermediate frames, giving each of them different advantages, thus allowing incomplete evidence to predict each other. For different visibility regressions, point frames to adapt to light changes, motion compensation frames to instantaneous restoration structures, and high-mode range frames to replenish exposed or missing areas. Based on these frames, this article further gains an internal model, IGFNet, which uses cross-traject attention and memory module allocation power, preserves reliable signal and disconnect structures, and ultimately uncovers visual visibility in hidden space, and estimates that the visual value of GTVs can be represented in real-time, high-definition, and high-definition AI.
IGFNet generates an estimated sequence of visual true values in flash interference, over-exposure, motion blur and mixed vision degradation.
General visual true value estimates of real-time data in more extreme scenarios
I'm not going to say that I'm not going to be able to do it.
One pathway uses e-VGT as a new canvas to help the basic model generate semantic labels; another migrates the robotic features formed within the IGFNet, allowing the downstream network to inherit the ability to cope with visual degradation. AI gains not only a clear image, but a perceptual basis for depth, division and positioning.
In single-dimensional depth estimates, the team combined e-VGT marking with real-time data, built Tianmouc-MDE data sets with good single-dimensional depth marking in extreme environments, and accessed the deep network with pre-trained IGFNet encoders. Even if images are affected by visual degradation such as exposure changes, the depth structure remains continuous.
Estimated visual authenticity and the visual characteristics provided by the internal model IGFNet significantly enhance the ability to estimate the depth of a single eye in extreme environments of the open world.
In video instance splitting, the team constructed Tianmouc-VIS data sets and designed YOLO-CVS in a similar fashion. Along with simulated data, the team learned basic detection and splitting capabilities, and while Tianmouc-VIS was small in size, the e-VGT-based real-world labels produced significantly improved detection and instance splitting performance in extreme environments under multiple model training tests.
Based on the algorithmic framework presented in this paper, the team tackled the difficulty of splitting video examples under extreme image degradation conditions, with extremely high landing potential in scenarios such as autonomous driving perception, industrial fault dynamics detection, and low-empty drone imaging.
The ecological matrix reshapes vision:
Multiple relevant results published at international summits
The Nature Sensors series provides a complete set of visual native language-based learning theories and applications, and officially releases the TianMouCV algorithm software library for the TianMouCV service. Currently, the TianMouCV has served multiple top-level publications, gradually forming an ecosystem of algorithm applications around the TianMouCV. Here are some excellent work and application examples:
Visual degradation in cellphone imaging is often caused by night vision exposure, rapid subjects and complex motion. The antenna can simultaneously record spatial and time differences within a single RGB exposure, preserving structural and motion evidence. STGDNet's work on CVPR 2026 has been validated in more than 100 real-world scenarios: the antenna achieves a blurred image by recording the exposure process.
Y Meng*, L Yang*, et al. CVPR 2026) is a two-way merger based on the celestial core.
In robotic scenarios, visual degradation such as strong over-exposure, rapid shifting disrupts character tracking. IROS 2025 works with CSVO to construct asynchronous two-way visual mileage meters using low torque RGB and high-speed spatial differentiation to maintain a more stable trajectory in exposure errors and rapid angle changes, allowing AI to continue to perceive itself and the environment during visual degradation.
Y Lin*, Z Zhang*, et al. IROS 2025) is a single-point visual distance measurement based on the celestial sphere.
In conjunction with the video scattering generation model, the visualization reconstruction effect of the antenna core data also takes on a new stage. In ICCV 2025, the team relies on an advanced data set construction scheme based on digital light processing (DLP) to train a standardized two-way circular scattering model (CBRDM) that can achieve precise, clear and colorful video scattering reconstruction.
Based on the high-quality high-speed blurred video reconstruction algorithm generated by antenna cores and video scattering (Y Meng*, Y Lin*, et al. ICCV 2025)
In the field of wearable devices and psychology, complementary vision can be used to track rapid, small eye changes with scarcity differential signals, significantly reducing computational and power demands, while retaining the iris, tear, and facial appearance. It can be widely used in AR/VR and wearable devices.
Eye tracking based on the celestial sphere
In depth assessment tasks, complementary vision can also take full advantage of mature traditional visual large-scale modeling capabilities, allowing dual path complementary features to be efficiently decoded, integrated and optimized, precise deployment of real-depth restoration scenarios, and significantly improving depth assessment accuracy and stability in complex dynamic scenarios. The program can be perfectly adapted to real-world scenarios such as autonomous driving, indoor navigation, and robotic avoidance, effectively addressing pain points where traditional depth assessment methods are unreliable under extreme visual conditions.
Based on the extreme scenario binary path convergence of the celestial core (X Chen, et al. ACMMM 2026, AC recommended oral)
From 10 to 100: Reimagining the cognitive infrastructure of the era of physical AI
From 2024 to the present day, with the introduction of a new AI visual perception paradigm, the Nature Sensors silicon cover further builds a comprehensive technical framework from sensing, self-monitoring expression to system recognition, and the Chinese team around the silicon silicon silicon core continuously improves the collaborative innovation system covering the chip, algorithm and software ecosystem. This path not only drives the continuous development of complementary research, but also is gradually radiating forward exploration of relevant fields around the globe.
As the core platform for the transformation of the results of the University of Qingdao Cerebral Computing Research Center, it is clear that science and technology will take full responsibility for and continue to advance the industrial application of Cerebral Vision technology. The company's core founding team, CEO, Dr. Chen Zhengyu, CTO Wang Zhen, and chief scientific advisor, Dr. Lin Liang Zhen, a long-time member of the National Brain Intelligence major research project, has participated as a core member in two original articles on the cover of Cerebral Nature. In the company's research team, Ph.D., M.S. and more than 80%, core members have an average of 10 to 20 years of experience in the field of image sensors and chips, and have served Sony (Sony), Great Britain (Nvidia), Samsung, and other top companies in the world.
Based on the visual native language RGB+TD+SD integrated perception and dual-track hybrid chip architecture, clear vision technology has achieved adaptive visual token extraction, while reducing redundant visual information by more than 90%, significantly reducing the multi-modal token delay (TTFT) with the world's first token model by more than 20 times, improving the token processing speed (TPS) by more than 10 times, and reducing the cost of training the model at a quantitative level, providing a more efficient and less costly data base for large-modal visual input.
More importantly, this has led not only to improvements in performance indicators, but also to the transformation of the visual model input paradigm. For a long time, AI visual systems have always used the x-pixel array as a single input, sending bulk redundant pixels into the model, and relying on large-scale algorithms to complete information screening. While brain complementarity for the first time directly organized visual native language into scarce, high-information-density visual tokens, allowing the sensors to output natural and large-scale models such as the Transformer's Token signals, achieving a leap from "gathering all the pixels, then understanding the world", to "directly obtaining intelligible information units".
This means that visual sensors are no longer just data-gathering devices, but are beginning to take on the front-end computing of information coding and semantics organizations, becoming the smart input layer of big models and world models. As this new visual input paradigm continues to mature, it is clear that technology is driving the acceleration of brain complementarity in scenarios such as smart robots, unmanned systems, AR/VR, intelligent manufacturing, and building a new generation of visual infrastructure that is more efficient and robust for Physical AI.
Nature Sensors: Brain-inspired Visual Sensing-Perception for Open-World Environments
TianmouCV is an open source project:github.com/Tianmouc/tianmoucv