It's 10.7 times faster! This end-side reasoning engine is powered on, and the robot's body-running m

Published by Panda

In the second week of September, the circle was very hot.

On the 8th, Shenzhen Power released the HERON-World Model; on the 9th, Xi'an released AGILE 2.0 and GE-Act 2.0; on the 10th, Wu-Tang Open Source UnifoLM-WLA-1.0, 6B parameters, about 2500 hours of machine data, one model to unite 64 tasks; and on the 15th, Shenzhen extended the addition of HERON-CRA.

Meanwhile, robots are accelerating their entrance into the physical world. On September 20, the company launched a new product launch, two personal robots were officially launched, with Q1 priced at $999.99; on September 10, Uber announced that it had won over 50 million overseas orders for products such as the Walker C1, the Uber World U1.

At the intersection of the two clues, a core pain point emerges: how to deploy complex, powerful, large, efficient, stable models to the very limited computational robotic hardware? This is the core value of the lateral reasoning engine.

On 15 September, the Chinese University of China jointly launched APXInf, an end-to-end reasoning engine aimed at body models. It also answers two questions:

In an edge environment where computation, memory and power consumption are limited, how do you get a body model to reach the speeds of reasoning available on the body of a robot?
As the model evolves, how do we build the ability to continuously adapt and optimize on the edge that never fails?

The answer to the first question, which APXInf provided, was a set of numbers: without changing the π0.5 model itself, through end-to-end full-frame optimization, the reasoning delay was reduced from 278ms to 26ms in the Thor chip FP8 configuration, and the end-to-end acceleration was about 10.7 times faster, with a frequency of 38.46Hz allowing robots to control the real-time range.

The answer to the second question is written in the way the APXInf warehouse is built. It incorporates model adaptation, optimization, and verification from a small number of experts into a continuously replicable workflow that can be used by the Agent.

Address of the project:https://github.com/RLinf/APXinf-robo

Why do we need specialized end-to-end reasoning optimization?

To answer this question, we must first look at the location of the reasoning engine in a robot.

One control loop is: the control collects observations, the control box reasoning out a chunk of the action, pushes it to the arm or substrate to execute, and then enters the next loop. The reasoning engine is embedded in the middle of the loop, and it decides how many milliseconds it takes to reason, how many Hz the control frequency can reach, and how hot and expensive the algorithm box is.

This position poses a fairly specific set of requirements for the engine: small batches, real-time delays, small disruptions, and the ability to be controlled by a stable call via websocket or ROS.

The general framework of reasoning is to use a unified intermediate representation to lower the model layer by layer, and then give the back end executable code to generate, a set of compilation packs covering as many models and hardware as possible; vLLM, SGLang, and similar schemes revolve around cloud throughput design. They are each successful, but the benefits are based on the assumption of multiple model types, large batches, and sufficient scheduling space.

So the end-to-end deployment of the body model forms three types of reality bottles.

End-side performance. On the end-side, computing power, bandwidth, power consumption and heat dissipation are simultaneously limited, while a single reasoning is required to complete multi-point perception, model forwarding and motion generation, and master control with a stable rhythm response. Between end-side modules and independent graphics cards, the memory bandwidth is 4 to 8 times different and the power consumption is 5 to 10 times different.

Manpower and time. The deployment of models to end-to-end hardware is not a simple copy run, but a massive system engineering. It requires experience from underlying architectural adaptation, core algorithm compilation, precision quantification, to software hardware performance optimization and simulation verification. This process typically takes several weeks and relies on cross-domain experts who understand both reasoning systems and algorithm optimization, and these are very rare in any hardware company. More importantly, small changes in hardware will lead to exhaustion: once the chip is replaced, all the algorithm selection, memory layout and flow waterline deployments must be reversed.

Stability. Demo running and long-term stable running are two things. The latter face continuous perceptual control, resource constraints and multi-module collaboration, and the worst thing is not to run a little slowly, but to run running with vibrations, crashes or loss of control.

The three columns are stacked together to form a multiplicative characteristic:

Deploy the model to the total workload on the main body ≈ ((one access + one adjustment) x number of main body models × number of chip platforms × number of model iterations.

Each variable on the right is dramatically larger: the core model is increasing, the chip is diversifying, and the model's iteration cycle is also shortening rapidly. In fact, five models are a direct representation of this in the first eight days of the article. The expansion team is not realistic, it requires a layer of infrastructure that can be continuously replicated: both to push single reasoning to the limit of the hardware, and to allow the next model to access the next piece of chip without having to start over.

This is exactly what APXInf needs to do.

Putting reasoning into real-time space

What did APXInf do right?

Let's take a look at the first challenge: how to make reasoning efficient on limited hardware.

APXInf does not focus on unified general IR as its primary goal, but instead prioritizes the construction of model family-specific execution paths. The model structure, weight layout, memory space, algorithm integration and execution sequence are all directly reflected in the code; it is only after multiple model validation that the shared capabilities are further embedded in the shared modules. This design allows optimization to deepen the model structure and hardware characteristics, targeted adjustments in the algorithm selection, memory layout and execution process, reducing the additional expenses introduced to accommodate general scenarios.

When running, the real-time reasoning for the controls that are really needed is maintained only on the edge:

The algorithm executes the entire graph capture and steady state playback using CUDA Graph, which is selected by Kernel to be generated and perpetuated by Autotune;
Memory is maintained by the model layer with a fixed Workspace, fixed shape, pre-allocation, address stability, reducing the movement of data on thermal paths;
The scheduling is based on small batch real-time reasoning, with Continuous Batching and Paged Attention not included in the larger batch.

When running, no complicated initialization is performed, instead the execution path is predictable, reproducible, auditable.

This characterization has been implemented to the built-in link: the compilation will query the computing capabilities of the native GPU, only for the kernel; the CUDA kernel, CUTLASS and FlashAttention source code are built into the warehouse, deployed without the need for Docker and external frameworks, and save off-target GB of mirrors when deployed.

Baseline is 1300ms, followed by torch.compile, pipeline, graph, kernel, and running, eventually dropping to 119ms, an overall improvement of more than 10.9 times.

As a result, the official rating protocol is LIBERO-10 for all 10 missions, 50 episodes each, seed fixed at 7, replan step length at 5, 500 rollouts total. Thor FP8 success rate is 92.2%, Thor BF16 92.8%, Orin BF16 92.0%, and as the reference π0.5 reference implementation is 92.4%.

At the same time, Rust's compulsory RAII features and proprietary systems greatly reduce the risk of memory problems, support more secure robotic lifecycle management, and unsefe is strictly restricted to fixed FFI limits. The system price of this middle layer is to make field pointers, data competition, and hidden bugs like this one, eliminated as soon as possible before they go online; a continuous working machine with 38Hz, which can only run up to 38 times, or 38Hz, can be said to be running continuously after 38Hz.

The model continues to change.

How do you think optimization works?

The characterization path brings performance, but also new problems: writing a path for each model family means updating the model, changing the hardware, and repeating the path. If this part of the work is still done by a handful of experts, it will never be possible to replicate the body model.

The solution to APXInf is to produce the engineering capabilities themselves: access to models that are already scattered in a handful of expert experiences, pre-processing, intrinsic adaptation, performance optimization and deployment verification, embedding code into engineering processes that Agent can understand, call on and continuously iterate.

In this new workflow, human-machine division of labour presents a disruptive pattern:

The agent is responsible for running the process: reading PyTorch Reference, generating Ledger execution, implementing the static path of the model, executing the arithmetic on the stroke and return, and finally completing Autotune, documentation and iteration.
People are responsible for setting standards: boundaries between architecture and module responsibilities, kernel contracts and security specifications, accuracy, performance and acceptance lines for tasks, and deciding when to draw commonalities into shared abstractions.

Handing over to the Agent is no longer a simple auxiliary task, but the most time-consuming part of the execution of the expert, and humans evolved to control the overall "judgment" with "decision-makers".

It is also due to the fact that the implementation can be handed over to the Agent that the rigorousness of the verification system has become the new core fortress.

Full-link layer verification: from single-algorithm, layer to complete model, with cross-pairs of Eager and Graph paths to ensure that every step is accurately controllable.
Fail-closed (fail-closed) Principle: Fail-closed parameters or hardware failures that are not supported, never silently producing incorrect results.
Model family isolation: Sharing capabilities only sink into the kernel layer, and any changes must be reversed through strict sub-layering, locking the risk of death.

What does this mean for users?

For enterprises, it transforms reasoning optimization from a one-time project into sustainable engineering capabilities. Self-development models change one version without having to rearrange experts; a new chip doesn't have to push back the previous round of customization experiences; team size is no longer a hard constraint on access speed. More importantly, expert experience is transformed from a personal touch to a team-repeatable asset.

For the industry, it provides a new way to build infrastructure. In the past, measuring a framework of reasoning to see how many models are supported, how much hardware is suitable, and how complete the arithmetic library is, these indicators are based on the assumption that the cost of access is fixed, so the wider the coverage.

From Mizar to APXInf

End-to-end reasoning technology extends to the physical world

APXInf is not an experimental project from scratch, but the inevitable result of long-term accumulation in the field of end-side reasoning.

The APXInf gene directly inherits Mizar's intelligent-terminal reasoning accelerator. At the beginning of the Mizar stage, the unquestioned kernel around terminals such as AI PCs, AI boxes, and integrated circuits, gradually formed the core capabilities of asymmetric hardware adaptation, local model deployment, logic acceleration, memory optimization, and low-power operation. This technology not only supports local models, doubles the scale of reasoning performance, but also improves intelligence levels by 18% under the same computing resources.

More importantly, this technology pathway has gone through an experimental phase and has passed rigorous mass production testing. In February 2025, Uncharted reached a deep collaboration with Lenovo, implanting Mizar engines in Lenovo's new generation of AI PCs, and in November of the same year reached pre-installation collaboration for over 10 million AI PCs.

AI PCs are not the same as end-to-end loads, but the constraints they face are consistent: reaching reasonable speeds under limited computing, memory, and power consumption conditions.

Addressing the new challenges posed by the body scene, APXInf extended its technology: whether faced with more complex VLA model multi-mode execution links or dealing with the real-time control needs of robots in milliseconds, APXInf was able to fully release hardware performance through underlying optimization. At the same time, faced with the high-speed evolution of the model, it also introduced the engineering process for Agent development. This not only achieved the "seamless displacement" of the original edge capabilities, but also completed the "deep adaptation" for the body industry.

Another important consideration of the ecological layout is the wireless "engineering loop". APXInf chose to start in the RLinf open-source ecosystem because model training and intrinsic reasoning are the two links on the same engineering link.

RLinf covers reinforcement learning training, evaluation and real-world robotic workflows, while training completed body models really need to get to the core, as well as a set of reasoning engines designed for small batches, low latency and limited power consumption. APXInf connects to this rod, extending RLinf's capabilities from training and evaluation to lateral reasoning and deployment, allowing developers to go from training, verification to the entire process of running the core in the same ecosystem.

Next step is open source co-creation

As of today, APXInf has completed a synchronous adaptation of π0-fast, GR00T and Qwen Drive, a model that supports further coverage of different body scenarios such as robotic operation, mobile intelligence, and more.

On the hardware side, AMD and the rear end of the domestic chip have also entered the roadmap. In the future, more body models will support efficient, stable operation on different end-side devices.

In order to truly enter the physical world with physical intelligence, there is a lack of model, physical and intermediate infrastructure. In the face of massive access and verification projects, this is the original intention of APXInf to choose open source co-creation.

If you're deploying π0.5 on a robot with RTX 4090, Jetson Thor or Orin on hand, or you want to access new models, chips for APXInf, or even participate in the build of the Skill and Agentic deployment process itself, welcome hands-on experience, submit Issue, or contribute PR.