AI has begun making its own chips, and OpenAI only took nine months to complete the entire process from scratch to production.

The development of custom chips has become an unstoppable trend.

OpenAI has announced the first round of public test results for Jalapeño, its first in-house AI inference chip co-developed with Broadcom.

On the public benchmark InferenceX released by SemiAnalysis, Jalapeño ran three models: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. The results show that Jalapeño delivers 1.5 to 1.9 times more AI workload per watt than NVIDIA GB200 and GB300, with end-to-end latency reduced to approximately 30% to 60% of that of the reference system.

When running DeepSeek R1, the inference process for one 8K input and 1K output dropped from 5.99 seconds to 1.65 seconds, and the minimum token interval was reduced from 5.90ms to 1.43ms.

With Jalapeño, in the future you may see more than just the 1.5x speed "fast" output mode on ChatGPT. As long as sufficient resources are invested, a 3x speed mode is also achievable.

Its performance is so outstanding that Sam Altman described this achievement very concisely:

"We built a chip, and it's extremely fast."

The High-Performance Jalapeño, Built Exclusively for Inference

Different from more general-purpose GPUs, Jalapeño has only one single goal from the very beginning of its design: to undertake the inference workload of modern large language models.

This goal is closely related to the current popularity of AI Agents.

Anyone who has observed the task chains of Agents at work will find that a single task often requires the model to be called dozens of times in a row: reading files, searching for information, operating software, checking results, and then proceeding to the next step based on the results.

The accumulated inference latency of each step makes Agents spend a lot of time even processing relatively uncomplicated daily tasks.

This is why the homepage interface of ChatGPT is divided into "Chat" and "Work" —

Simple daily Q&A can output results quickly, while complex tasks that require Agent invocation must be separated independently to prevent Agents that "think too much" from slowing down the daily user experience.

If the inference speed of Agents can be further improved, their response will be as fast as regular chat. In principle, ChatGPT at that time will return to its original form: there will only be one dialog box on the homepage, and there is no need to distinguish between chat and work modes.

To achieve faster inference speed, Jalapeño is specially optimized for the real workload of large model inference from the initial design stage.

As we all know, for chips under the von Neumann architecture, the computing speed is often far faster than the data transmission speed.

In the article published the day before yesterday, we used a metaphor: even if the speed of cutting and stir-frying in the kitchen is as fast as lightning, as long as the speed of transporting ingredients cannot keep up, even the most skilled chef can only wait in front of an empty stove.

Xiaomi released its full lineup of Xuanjie AI chips, and also officially announced the Xiaomi "Ultra Foldable" device.

The inference process of large language models is mainly divided into two stages: the first stage is Prefill, in which the chip reads the entire prompt input by the user at one time, which mainly tests the computing capability; the second stage is Decode, in which the model starts to generate tokens one by one. This step requires repeated reading of weights and KV Cache from the memory, which will inevitably hit the "memory wall" where the data transmission speed cannot keep up with the computing speed.

Figure | OpenAI

This also means that to approach the upper limit of computing capability, it is necessary to do as OpenAI did when designing Jalapeño: co-design the computing cores, HBM memory and internal network as a whole, to minimize the time spent on data reading.

According to the disclosure from SemiAnalysis, Jalapeño is equipped with 216GiB HBM4, with a memory bandwidth of 15.4TB/s. This figure is nearly twice that of Microsoft Maia 200 and Google Ironwood. Such an aggressive bandwidth design is to ensure that data can keep up with computing, and reuse the theoretical computing power that was wasted waiting for data.

Figure | SemiAnalysis

In addition, Jalapeño divides the computing cores and HBM into multiple corresponding regions, so that model weights and KV Cache can stay in the local memory responsible for processing them as much as possible; cross-region synchronization will only transmit data through a dedicated high-speed communication network, which reduces the steps of data transmission back and forth between cores, memory and chips, and cuts down the time that computing units spend waiting for data.

Figure | SemiAnalysis

Since computing, memory and network resources are all considered in the chip architecture from the initial design, Jalapeño achieves an extremely impressive energy efficiency ratio — its nominal power consumption is 700W, and the sustained power consumption in actual tests is even lower than 550W; for reference, the nominal power consumption of NVIDIA's GB200 and GB300 are 1200W and 1400W respectively.

In other words, Jalapeño can deliver higher inference performance with roughly half the power consumption of NVIDIA chips.

Power consumption translates directly to electricity costs, which means huge operational expenses. Considering the massive computing demand that data centers face every day, any optimization of energy efficiency can significantly reduce OpenAI's spending on computing resources.

Figure | Computex 2026 keynote

OpenAI plans to deploy Jalapeño to its computing infrastructure before the end of this year, while the second-generation product is already in the middle of development, and the third-generation has also taken initial shape.

With the widespread adoption of Jalapeño, we are also expected to see Tibo press the "reset" button more frequently.

3 Years for Chip Development, Only 9 Months for Tapeout

The "speed" of Jalapeño is not only reflected in its performance, but also in its design and production cycle, which is so fast that it hardly looks like a semiconductor product.

According to Reuters, this project officially started roughly from the end of 2024 to the beginning of 2025, and many of its initial team members came from Google's TPU project.

A year before that, OpenAI had already invited Richard Ho, the former head of Google TPU, to form its hardware team —

Obviously, long before Gemini once overtook its competitors with its own TPU computing resources, OpenAI had already realized the importance of self-developed chips.

Eventually, the project was officially unveiled last October, when OpenAI announced that it would cooperate with Broadcom to deploy 10 gigawatts of custom AI accelerators, "integrating cutting-edge model and product R&D experience directly into hardware"; the news of its official tapeout came out last November.

Combining the official blog disclosure in June that this project only took 9 months from initial design to tapeout, Reuters' estimate that it takes about 6 months from tapeout to finished products, and SemiAnalysis' information that OpenAI only carried out about 3 months of power-on debugging before the official test in August, we can sort out a clear timeline:

Jalapeño officially entered the design cycle around February 2025, completed tapeout in November, OpenAI received the first finished chip in May this year, and finished debugging and launched public test in August.

Every step was carried out at full speed, approaching the upper limit of chip design speed.

According to the estimation of Semiconductor Engineering, an AI acceleration chip with such high complexity as Jalapeño usually takes 18 to 24 months from formal design to tapeout; OpenAI compressed the formal design cycle to 9 months, which is almost twice as fast as the industry average, making it the best portrayal of the rapid development in the AI era.

One key factor to achieve such speed is AI itself.

In OpenAI's words, AI directly participated in the development of Jalapeño.

Its models were used to explore implementation solutions, participate in verification and modification of plans, and optimize the arithmetic circuits of the chip during the design phase; after the finished chips were delivered to the lab, OpenAI used Codex and the unreleased Astra model to complete the adaptation of several models that were not in the original adaptation plan of Jalapeño, including GPT-OSS, DeepSeek R1 and Kimi K2.5, within two months.

As the symbol and product of the "great acceleration era", AI is greatly shortening the R&D process of chips. The second and third generation Jalapeño, as well as dedicated chips from other large model companies, are likely to meet us in the next few years.

Why are all large model companies developing their own chips?

The launch of Jalapeño is undoubtedly stunning, but OpenAI is not the earliest player on the self-developed chip track.

The Google TPU, which provided countless initial talents for this project, is a well-known example. This product that was first deployed in data centers ten years ago has now iterated to the 8th generation.

As early as the 7th generation product in 2025, it was Ironwood, the first TPU chip designed exclusively for inference; and the 8th generation product released this year is further divided into TPU 8t for training and TPU 8i for inference.

Figure | Google Cloud

Google even believes that TPU is not specialized enough.

According to The Information, Google is developing a server chip codenamed Frozen v2, trying to solidify part of the Gemini architecture directly into the hardware, and its token generation per watt is estimated to be 6 to 10 times that of the latest TPU.

Google is embedding its large AI model directly into the chip.

On the other side, Amazon AWS has also successively developed Inferentia chips for inference and Trainium chips that support both training and inference, supporting its cloud business by selling computing resources externally.

As Amazon's largest external ally, Anthropic has previously obtained computing power from nearly 1 million Trainium chips through Project Rainier.

But Anthropic does not intend to rely entirely on Amazon. Earlier this month, they confirmed that they are building an in-house dedicated chip R&D team to develop custom AI chips for the next generation of Claude large models.

It is clear that betting on self-developed chips has become a common direction for leading large model manufacturers.

The most direct reason for this change is naturally that computing efficiency is closely related to