Over the past few years, the core of AI infrastructure's competition has been GPUs. Larger models, higher parameters, and stronger reasoning have driven data centers to build GPUs.
But as AI Agents begin to perform increasingly complex tasks, new computational needs arise. An Agent no longer just completes a model reasoning, it needs to continuously plan tasks, call tools, execute code, process data, and stay in context between multiple steps.
According to OpenRouter data cited by NVIDIA, the average length of Prompt in real AI traffic has grown by about four times, and the number of tokens that Agentic requests to consume is up to 15 times that of regular chat requests.
Just this morning, NVIDIA first announced the results of the Vera Rubin performance test based on real chips.
In tests for Agent workloads, Vera Rubin achieved 30 times the maximum throughput per megawatt compared to the previous generation GB300 NVL72 and lowered the token cost by a maximum of 35 times.
At the same time, SpaceXAI announced the deployment of the NVIDIA Vera CPU and plans to expand the AI infrastructure supporting Grok based on the Vera Rubin architecture to further explore the deployment of the optimized Vera Rubin NVL72 system to the orbital computing environment.
From Agent Performance Testing to SpaceX's orbital computing exploration, NVIDIA shows a new direction: the key to future AI competition is not only model capabilities, but also the computing infrastructure that supports Agent's ongoing operations.
Vera Rubin faces Agent workload testing for the first time
In the past, large model performance testing focused more on reasoning speeds below fixed input lengths. However, AI Agent has changed this standard. Measuring the ability of the Agent's infrastructure no longer relies on traditional fixed Prompt testing, but needs to be close to the workload of the real production environment. This is also the focus of NVIDIA's testing.
NVIDIA uses the AgentX workload from SemiAnalysis to evaluate the Agent performance of the Vera Rubin NVL72. AgentX is a benchmark test aimed at Agentic Coding Inference, which evaluates the computing system's ability to process real Agent requests by playing real-world production-style Agent sessions.
The Intelligent Body (Agent) session will generate a sequence of reasoning that changes dynamically.
Unlike traditional chat tasks, Agent sessions have distinct long-link characteristics: a single task may include continuous model calls, tool use, and continuous accumulation of upstream context. AgentX retains the upstream context length, input-output sequence, reasoning time, and delay in tool calls in the original task to simulate the actual state of the Agent's operation.
For example, when a code agent completes a development task, it may need to understand the requirements, analyze the code library, modify the program, perform tests, and continue to adjust the program according to the test results. As the task progresses, the system not only needs to process the longer context, but also needs to continually repeat the information that has already been processed.
In a real Agent trajectory displayed by NVIDIA, the main Agent's background can grow from about 60,000 tokens to nearly 400,000 tokens while interfacing with multiple sub-agent requests. That is, for the Agent, the "higher background" is a state of operation that is naturally formed during the mission progression, rather than an occasional extreme situation.
NVIDIA defines such metrics as AI factory-grade "throughput per megawatt" ("tokens per megawatt"). AgentX also looks at metrics such as throughput, end-to-end latency, and first token time to measure whether the system can maintain a good interaction experience at high efficiency.
In the test, NVIDIA used the DeepSeek V4 Pro model to run the Agent X workload. The results showed that at the 160 tokens/s interaction target per user, the Vera Rubin NVL72 achieved a maximum of 30 times higher AI factory-level throughput per megawatt than the GB300 NVL72, meaning that it could support more Agent reasoning tasks under the same energy budget.
NVIDIA defines Vera as "the first CPU built for AI Agent". It uses 88 NVIDIA-designed Olympus cores and comes with high-bandwidth LPDDR5X memory, with a maximum of 1.2TB/s of memory bandwidth.
Of course, Vera is not going to replace the GPU, but it will take on a new role in AI systems: GPUs continue to be responsible for large-scale model computing, while Vera is responsible for supporting task scheduling, code execution, and data processing during Agent's run.
Where did 30 times come from?
If only 30 times attributable to the Rubin GPU itself, Vera Rubin's actual changes would be underestimated. NVIDIA's performance is essentially a rack-scale system result rather than a single chip result.
The Agent runs into the later stages and will continue to process longer and longer contexts. The reasoning system not only needs to perform Transformer computations, but also needs to efficiently store and replicate KV Cache, and exchange model experts, upstream context states, and intermediate results between different GPUs. As the length of the upstream context, user interaction speed, and concurrent scale continue to increase, the bottleneck has also begun to expand from mere GPU computing to memory, communications, cache management, and system scheduling.
For this purpose, the Vera Rubin NVL72 combines the Rubin GPU, HBM4, NVLink 6, and the new logic software package for a collaborative design. The Rubin single GPU configures 288 GB HBM4, with a memory bandwidth of 22 TB/s; the NVLink 6 provides a 3.6 TB/s scale-up bidirectional bandwidth for the single GPU. For the NVL72, which consists of 72 GPUs, a larger unified scale-up domain allows MoE experts to run parallel, distributed KV Cache and other mechanisms throughout the frame.
Changes at the software level are equally important. The key mechanisms listed by NVIDIA include Prefill/Decode separation, Distributed KV Cache, KV Cache offloading, KV-aware routing, and large-scale Expert Parallelism.
As an example of KV-aware routing, when an Agent re-issues a request, the system makes every effort to route the task to the GPU that has already saved the corresponding context cache, thus avoiding recalculating the amount of tokens it has previously processed. For the Agent that may continue to grow to hundreds of thousands of tokens in the context, this cache replication will directly affect the throughput, delay and actual cost of each token.
In other words, Rubin's real upgrade isn't just a GPU, it's a complete "token production stream": from reading the context and executing the model, to switching KV Cache and MoE experts, to continuing to generate tokens, each layer will ultimately decide how much Agent to serve.
NVIDIA is also further dismantling the token's production stream.
On the same day as the announcement of the Vera Rubin Agent performance test, NVIDIA announced that Groq 3 LPX is entering full-scale production and incorporating it into the Vera Rubin platform. Groq 3 LPX is positioned as a low-latency reasoning accelerator for interactive AI, which does not replace the Rubin GPU, but is specifically optimized for another class of computational features in Agent reasoning.
Big model reasoning can generally be roughly divided into two phases: first, Prefill, which reads and processes Prompt and a large amount of context that has already accumulated; then Decode, which generates new tokens one by one.
For agents with hundreds of thousands of tokens in the context, these two phases require different hardware requirements. Prefill requires higher computing power and memory throughput, while Decode focuses more on single token generation delays and tokens/s that users can actually access.
As a result, NVIDIA is beginning to allocate different types of Agent computing to different processors: Vera CPU is responsible for out-of-model task compilation, code execution, and data processing; Rubin GPU is responsible for large-scale model computing, long-term context, and high-throughput reasoning; Groq 3 LPX is further optimised for high-interaction, low-latency token generation.
In a context test on the Gemma 4 31B model, 100K, the third-party Artificial Analysis measured Groq 3 LPX to 3431 output tokens/s; in NVIDIA's own SPEED-Bench test, its coding task median output was 4767 output tokens/s.
More importantly, the Groq 3 LPX can run directly with the Vera Rubin NVL72 combination. One option is to have Rubin in charge of the Prefill and then hand over the Decode to the Groq 3 LPX; it can also be further subdivided into Attention and FFN calculations, or to have LPX take over the draft model in speculative decoding.
This means that the computing system behind the Agent may no longer be run by a single general-purpose processor, but will increasingly be approached by an asymmetrical computing factory: different chips process the most appropriate computing stages of their own, and then high-speed connectivity, caching, and software systems combine them into a complete token production line.
In fact, NVIDIA's current Vera Rubin platform has expanded to include a multi-chip system, including the Vera CPU, Rubin GPU, Groq 3 LPU, NVLink 6 Switch, BlueField-4 DPU, ConnectX-9 SuperNIC, and Spectrum-6 Ethernet.
The GPU is still at the heart of the entire system, but it no longer assumes all the competitive tasks of AI infrastructure independently.
SpaceX joins: From AI Factory to Orbital Computing
Another important application of Vera Rubin comes from SpaceXAI.
NVIDIA announced that SpaceXAI will deploy the Vera CPU to accelerate the next generation of Agentic AI applications, including task compilation, code execution, and data processing. At the same time, SpaceXAI plans to expand its AI infrastructure based on the NVIDIA Vera Rubin architecture to support Grok's AI infrastructure, and plans to expand its computing capacity to GW as the computing infrastructure continues to expand.
According to Musk, SpaceX and NVIDIA have designed a set of Vera Rubin NVL72 systems optimized for the space environment, scheduled to be launched into orbit in the fourth quarter of next year and deployed on a larger scale in 2028.
However, orbital computing is not simply about moving ground servers into space. Space environments pose completely different requirements for computing systems: limited energy, more complex heat dissipation conditions, higher reliability requirements, and long-term unmanned maintenance capabilities.
Therefore, SpaceX and NVIDIA are exploring how to adapt the Vera Rubin architecture to the orbital environment while maintaining a uniform software ecosystem.
AI Infrastructure Competing into the Agent Age
In the past, competition in the AI industry revolved around models. Whoever has a stronger model has an advantage. But the Agent era changed the rules of competition.
When AI starts performing long-link tasks, computing systems need to address performance, cost, energy efficiency, and task scheduling at the same time.
Vera Rubin represents NVIDIA's judgement on the next phase of AI infrastructure: future AI factories will be not only a place to train models, but also a computing platform where a large number of Agents will continue to operate.
SpaceX's exploration of orbital computing further expands the boundaries of this architecture. From ground-based AI factories to possible future orbital computing nodes, AI infrastructure has entered a new stage.
The link is as follows: