When you wake up, the news about OpenAI is really skyrocketing.
Exclusive to X user Leo@synthwavedd, OpenAI has completed a new pre-training, code-named "Bel", whose parameters are said to exceed 10 trillion, or Doug's successor, which could be the base model for Astra/GPT-6...
A round of searches found no accurate information. But there is no doubt that OpenAI's first self-developed logic chip performed well.
OpenAI has just released the latest test results of its first self-developed logic chip, Jalapeño: a major breakthrough in the design of a large-language logic model, with a completely new architectural design that can simultaneously increase throughput and reduce latency, while maintaining high energy efficiency.
In response, Sam Altman, CEO of OpenAI, said on X: "We've made a chip that's fast".
It should be noted that as the first self-developed reasoning chip from OpenAI, Jalapeño was not designed to train the next generation of GPT models, but to optimize it for the post-launch phase of the model.
To put it simply: the training phase is to enable the model to learn, while the reasoning phase is to enable the model to respond quickly to user requests.
According to OpenAI, traditional hardware systems often have to trade between two indicators:
Higher throughput: unit time to process more requests;
Lower latency: enabling users to respond faster.
For example, large-scale batch processing can improve overall efficiency, but may increase single-request wait times; while pursuing extremely low latency may also sacrifice resource utilization. Jalapeño's goal is to optimize both in one architecture at the same time.
In terms of specific performance, OpenAI said that Jalapeño is better than the NVIDIA GB200 and GB300 systems in several large model reasoning tests. These include: OpenAI GPT-OSS 120B; DeepSeek R1 670B; Moonshot AI Kimi K2.5 1T.
In particular, the emergence of DeepSeek R1 and Kimi K2.5 seems to illustrate that Jalapeño is not only aimed at optimizing OpenAI's own models, but also at adapting to large model workloads.
Interestingly, Jalapeño can be translated as "Mexican soup", and immediately after its release, it caused a stir and excitement among netizens across the street.
"Are you going to name your chip by the nickname? I can't wait to experience it".
For OpenAI, the launch of Jalapeño suggests that OpenAI is trying to open up a whole chain of "models, chips, software, chips, data centers".
It is also worth noting that the Jalapeño chip from OpenAI is a collaboration with Cerebras on hardware.
Tibo said: "This ability is thanks to our deep collaboration with Cerebras and their unique hardware architecture. In the future, this collaboration will further push the boundaries of the 'ultra-fast' experience".
I look forward to continuing our collaboration to push the boundaries on the Cerebras platform to explore how to run the best model we can at the fastest speed possible and deliver this experience to the most demanding customers".
In fact, OpenAI is not the first AI company to move towards self-developing chips. Many major AI manufacturers have already started making chips. For example, Google launched TPU; Amazon developed Trainium; Inferentia; Microsoft developed Maia; Meta also deployed self-developing AI accelerators...
The logic behind this is very similar: a general-purpose GPU is flexible enough, but not optimized for all AI workloads. For companies that run bulk AI requests every day, there is an opportunity to further reduce costs if they can design chips for their own models, software systems and service modes.
Below we take a closer look at the capabilities of Jalapeño.
Faster and more energy efficient
The core advantage of Jalapeño is that it does more AI work at the same power consumption while responding faster. Existing systems usually have to switch between throughput and latency, while Jalapeño tries to achieve higher throughput and lower latency with the same architecture.
OpenAI emphasizes that this advantage is not only present in the model itself but also covers models developed externally, proving that Jalapeño is a more general set of reasoning architectures.
In the three open models GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T, Jalapeño achieved 1.5 tons of 1.9 times the working power per watt of AI at peak throughput, with end-to-end latency reduced to 1/1.7 tons 1/3.6 of the contrasting system; for highly interactive workloads, the performance advantage was 2.1 tons (4.1 times).
On the GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T, Jalapeño's peak throughput per watt is 1.9 times, 1.7 times and 1.5 times that of the best existing systems, respectively.
On the largest-scale Kimi K2.5, Jalapeño's peak performance per watt increased by about 1.5 times and end-to-end delay decreased by about 3.4 times.
On the Kimi K2.5 1T, with single-user decoding speeds improving, Jalapeño's per-watt throughput advantage over GB300 has expanded from 1.5 times its peak to a maximum of 56.1 times.
OpenAI not only compares the peak performance of a single chip, but focuses more on one indicator: how much useful AI work can be done per unit of power when meeting user and interactive Agent delay requirements.
The reason is also related to the Agent. Agents often need to complete multiple steps in succession, and seemingly small delays in a request will accumulate throughout the execution of the task.
This is also where Jalapeño's performance in the low-latency range is most evident. In the example of DeepSeek R1, under the conditions of the previous best TBT response of the contrast system, Jalapeño's throughput per kilowatt reaches 12,258 mixed TPS/kW, which is about 104.3 times the difference between GB300's 118.
On the DeepSeek R1 670B, the Jalapeño peak had a throughput of about 1.7 times GB300 per watt; at the same decoding speed, the maximum advantage expanded to 104.3 times.
OpenAI uses SemiAnalysis' open benchmark InferenceX to test and compare it with the current leading commercial system. From high-throughput services to high-interaction, low-latency scenarios, Jalapeño achieved better per-watt performance and delay combinations on all three open models, located on the Pareto frontier along the Pareto frontier.
At the different operating points of the DeepSeek R1 670B, the Jalapeño combines better per-watt throughput with end-to-end interaction latency at the Pareto frontier.
Jalapeño's rated power output was 700W, but during this test workload, the actual continuous power output was always kept at 550W or below.
Living for an agent.
Jalapeño was designed from the outset around the big language models of the present and future, especially the interactive Agent.
Different phases of LLM reasoning have different bottlenecks: Prefill is more dependent on computing capabilities, Decode is more limited in memory bandwidth, and data movements between cores and chips increase latency.
As a result, OpenAI co-designed chips, memory, networks, software, and rack-level systems. The model states, including KV Cache, can be clearly placed and kept locally as much as possible, allowing Jalapeño to simultaneously adapt to Prefill and Decode and adjust to changes in their workloads.
OpenAI believes that this is a key feature of the Agent workload.
Interestingly, AI is not only the object Jalapeño is serving, but is also directly involved in the development of the chip.
Using the different stages of the model, the OpenAI team took only nine months from the initial design to the Tape-out. The model was used to explore different implementations, shorten design, measurement and verification cycles, and participate in algorithmic circuits to optimize the chip.
Jalapeño was also designed as a clear, predictable programming goal for both humans and models. Using the GPT-Astra-powered Codex, the team took just two months to get the three open-weight models that were not in Jalapeño's initial production plans to run to high performance.
In some GPT-OSS Attention and MoE modules, the codex was implemented 1.5 times faster than the original human expert handwriting version (1.8 times faster).
At the end of the year, the second and third generations are already on the way
OpenAI plans to start deploying Jalapeño in its own computing infrastructure by the end of 2026. The team is still in production verification, improving software, preparing for mass operation, and continuing to verify performance on more models.
Subsequent products have also been launched, Gen 2 has entered the deep development stage, and Gen 3 has begun to take shape.
OpenAI summarizes the efficiency changes brought about by Jalapeño in three grades: Ultra-fast mode, which can achieve the efficiency that only Fast mode has in the past; Fast mode, which can achieve the efficiency that only Batch mode has in the past; and Batch mode itself, which further improves efficiency.
However, self-developed chips do not mean that OpenAI is ready to get rid of NVIDIA.
OpenAI has made it clear that to meet the growing demand for AI, which requires more computing power from all available sources, the company will continue to deploy NVIDIA and other partners' accelerators on a large scale, while covering training and reasoning workloads.
What do you think? Welcome to the comments section!
The link is as follows:
