The deployment boundaries of the Physical AI model include not only prototype verification, but also offline evaluation, cloud reinforcement learning, Rollout, end-to-end real-time control, and may be in the form of factory MaaS, where multiple robots are served by local edge servers or cloud servers. These scenarios differ in the requirements for latency, throughput, and multiple-card communication, with existing work typically involving multiple code sets for different needs, high migration costs, multiple iterations in engineering development, and inconsistencies in the efficiency of different scenarios.
The paper proposes PhyAI, a unified logic runner for Physical AI. It places the model semantics in the Model Adapter, timing, caching, calculating, executing graphics, and handing them over in parallel to the runner. The paper revolves around four types of deployment scenarios and uses Control-Time Roofline as the main analytical tool for connecting logic performance to control cycles.
The work was carried out jointly by the Beijing Postal and Telecommunications University, Beijing University, Tsinghua University, Nanjing University, GIS and Face-to-Face Intelligence.
PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloud
Address of the paper:https://arxiv.org/abs/2608.03682
It's a great way to get started.
AtomGit address:
PhyAI and the official framework real-time deployed MiniCPM-Robot's demo video, three robots in parallel reasoning scenarios, PhyAI delayed up to 2.3 times, and no cartons completed tasks.
01 Deployment scenarios and problem definitions
Currently, there are four main types of embedded intelligent reasoning scenarios: benchmark evaluation scenarios, focusing on reproduction accuracy; cloud RL post-training rollout scenarios, used to improve model long-range task capability, focusing on multi-batch throughput and multi-GPU utilization rates; side deployment scenarios, focusing primarily on model reasoning time delays; edge or factory MaaS scenarios, focusing primarily on multiple requests throughput efficiency, network, queue, and batch processing affecting time delays.
Batch sizes, model accuracy, and execution equipment for these four types of scenarios vary, but image pre-processing, model reasoning logic, caching, and motion output can and must be repeatable and consistent.
However, existing work for these four types of scenarios tends to maintain four sets of model codes, with high migration costs, multiple iterations of engineering development, and disparities in the efficiency of different scenarios.
02 Unified Reasoning in Operational Design
This paper proposes PhyAI, a logic framework for end-to-end cloud integration. It uses only one set of model operating codes that can be seamlessly migrated and used in four scenarios typical of Physical AI.
PhyAI mainly consists of two key components: Model Runner to preserve visual language conditions, action expert or video action generation, solver, state replication and action conversion; Scheduler to select DP, TP, CFG and device group; Runner to manage KV cache, buffer, CUDA Graph and request state; Layers to merge or distribute arithmetic by shape, dtype with hardware.
The same model path can run in single-card, edge and cloud multi-card environments.
Structure of the PhyAI framework
03 Reasoning the bottle and the Control-Time Roofline
This paper measures the performance of different modules of different models and proposes a Control-Time Roofline.
In this article, the sub-module measurement first gives a bottleneck on the model side. In which, at batch=1, PI0.5 represents only 8.8% of the estimated FLOPs, but 57.2% of the latency; after batch=32, it decreases to 13.5%, with a throughput of about 100 samples/s.
PI0.5 and Cosmos3 Roofline modules on different hardware
However, due to the presence of RTC (Real-Time Chunking), network time delays, and post-image processing delays, the Roofline on the model side still cannot directly give the robot control rate.
The paper proposed the Control-Time Roofline, which is used to measure whether the current robot-controlled bottlenecks come from model reasoning or the physical environment. The paper tested PI0.5 on different devices and found that the main bottleneck on AGX Orin was reasoning, while the main bottleneck on RTX Pro 6000 was the physical environment.
This seemingly straightforward conclusion points out that continuing to reduce latency on computing-rich devices will not increase the ideal frequency of control in proportion. Algorithms, hardware, and infra require synchronous design, and the time saved by framework optimization should be used to support larger models, slower devices, or more simultaneously.
Control-Time Roofline Model
04 Benchmark and RL Rollout
In a single-request comparison of 11 groups with the same model and device, this paper achieved a relatively official acceleration of 1.40x to 4.65x. MiniCPM-Robot decreased from 105.38 ms on H100 to 22.64 ms; Cosmos3-Nano-Policy-DROID decreased from 2.46 s to 1.18 s on 8 H20s, CFG=2, TP=4.
The improvement in reasoning speed can also benefit cloud RL Rollout. RLinf's PI0.5 GRPO uses 4 A800s and 32 environments, and the reasoning time is 955.8 s, accounting for 15.9% of the total RL time. This article will increase the reasoning speed by 2.55x; keep other work unchanged, estimating the reasoning time in RL to be 863.2 s, a decrease of 9.7%.
05 Summary
This paper introduces the logic needs for benchmarks, cloud RL rollouts, end-to-end deployments, and factory MaaS to the same set of logic running, where model adaptation, algorithm optimization, and multi-card support are no longer repeatedly implemented along four paths to build a unified set of logic for intelligent logic.
The proposed Control-Time Roofline limits the actual benefits of acceleration. The operation of robots is not only constrained by reasoning speeds but also by environmental factors, and the bottlenecks of the two will change as the computing capabilities of the device and the size of the model change.
The paper notes that attention should be paid to the Control-Time Roofline in order to coordinate algorithms and hardware design.