GPT-5.6 is about to be launched, with its inference speed surging to 750 Tokens/s, and it is suspected to span 100 wafers
【Introduction】GPT-5.6 boasts an astonishing inference speed of 750 tokens per second! Industry insiders reveal explosive details: it will operate across 100 wafers. As AI shifts from deliberate thinking to instantaneous output, has the era of real-time intelligence truly arrived?
According to multiple leaks, GPT-5.6 is on the verge of public release for all users.
Recently, widespread speculation about this model has been trending on X (formerly Twitter).
On June 26, OpenAI officially announced the launch of its brand-new GPT-5.6 family of models.
An official blog post included this striking statement: OpenAI plans to launch its cutting-edge new model, GPT-5.6 Sol, this month on custom hardware from chip giant Cerebras, delivering a staggering inference speed of 750 tokens per second!
This means complex Agent operations that previously required minutes to complete can now be finished in the blink of an eye.
Clearly, OpenAI has taken a groundbreaking, disruptive step forward in the co-design of hardware and models.
Coupled with the recent unveiling of Jalapeño, the industry's first self-developed AI inference chip, it's evident that OpenAI harbors ambitions to build a full-stack AI empire.
Speed Defeats All: The Dimensional Leap of 750 Tokens/s
What does "750 tokens per second" actually mean?
For humans, that translates to reading and outputting roughly 500 to 600 characters of text in a single second.
GPT-5.6 Sol can generate the full text you are reading right now in less than 0.1 seconds.
On X, prominent developer Caleb Shepherd expressed excitement: "This is the most thrilling news I've seen. GPT-5.6 Sol running on Cerebras isn't just about faster code writing—it's a qualitative leap in how we interact with computers. We'll never again have to wait two full minutes for an AI to click a single button."
For a long time, even as large language models grew more capable, "inference latency" remained the biggest bottleneck for deploying real-time, multi-step Agent tasks.
When models scale up to trillions of parameters, traditional GPU clusters often hit physical limits in inter-node communication (via NVLink interconnects).
OpenAI's solution is revolutionary: instead of forcing models to adapt to hardware, they are merging hardware and models into a unified system.
According to initial official disclosures, GPT-5.6 Sol will be made available to a limited number of select clients in July, with broader rollout planned as production capacity ramps up.
As widely speculated online, this will be an extremely high-cost service, a premium privilege tailored exclusively for top-tier enterprises willing to pay for unprecedented speed.
How to Fit a 3-Trillion-Parameter Behemoth Onto Chips?
When news of the 750 tokens/s speed broke, Peter Gostev, head of LLM Arena, raised a question that puzzled everyone:
What exactly is happening with GPT-5.6 Sol running on Cerebras? From what I understand, this appears to be the full, uncompromised model (with full multimodal capabilities including vision), not a stripped-down version like the earlier GPT-5.3-Codex-Spark that removed vision and context support.
But my understanding is that a single Cerebras chip can only fit a model with 700 to 900 billion parameters at most. So is the model smaller? Are there new types of chips I haven't heard of? Or is this some innovative multi-chip coordination technology?
This question immediately sparked widespread discussion across online communities.
Some joked that people were conducting "forensic-level chip audits at midnight," commenting: "If this really is the full unmodified model, it's like someone squeezed a supertanker into a glass bottle and won't tell you how they did it."
Shortly after, veteran technical expert Bleys Goodson published a highly compelling technical analysis—
GPT-5.6 Sol is not contained on a single chip, but spans 70 to 100 Cerebras wafer-scale chips!
The Ultimate Deployment Elegance: "One Wafer, One Network Layer"
Industry experts estimate that GPT-5.6 Sol has an extraordinarily large scale:
- Total parameters: Approximately 3 trillion
- Activated parameters: Approximately 150 billion
- Network layers: Approximately 70 to 90 layers
To achieve optimal inference performance, OpenAI and Cerebras adopted an extremely extravagant and groundbreaking deployment strategy—each individual neural network layer is deployed entirely on a separate full Cerebras wafer.
As one user pointed out, by adding more pipeline stages, you can theoretically scale models to any size as long as you have enough wafers to link together. This won't reduce token generation speed, and will only cause a minor impact on Time To First Token (TTFT).
Radical Architectural Overhaul: The Forced Innovation of Lightweight KV Cache
However, massive numbers of wafers alone are not sufficient. A defining feature of Cerebras chip architecture is its abundant on-chip SRAM (Static Random Access Memory), which offers extremely high speed but comes at a premium in terms of capacity.
If OpenAI had used traditional heavyweight KV caching in GPT-5.6 Sol as it did in previous models, that valuable SRAM bandwidth would be instantly exhausted.
This leads to the core strategic shift in this partnership: model reconstruction tailored specifically for specialized hardware.
Bleys Goodson notes that since OpenAI was deeply involved in hardware co-design, it is highly likely they abandoned traditional attention mechanism caching schemes in favor of more advanced lightweight designs.
The most plausible solutions include:
An architecture similar to DeepSeekV4: Extremely optimized cache footprint.
Hybrid SSM design: Combining linear-time-complexity models like Mamba with Transformers to completely eliminate the historical burden of KV Cache.
Additionally, prominent developer John Lam proposed a brilliant hypothesis—decoupling attention calculations from FFN operations.
He speculates that OpenAI may be using conventional GPUs to handle attention calculations, while leveraging massive numbers of Cerebras wafers to brute-force accelerate the feed-forward neural network portion of computations.
This is not unfounded speculation. Users quickly uncovered details from Cerebras' earlier blog posts about Kimi K2.6 deployment:
Cerebras stored the original Kimi K2.6 weights in 4-bit on its CS-3 systems, while performing calculations in 16-bit floating point to preserve precision. Weights are distributed across multiple wafers, and activations are streamed between wafers. Full inter-layer communication relies entirely on on-wafer network structures, delivering over 200 times the bandwidth of NVLink on Nvidia NVL72! Combined with custom operators and speculative decoding, they can run trillion-parameter MoE models at nearly 1000 tokens/s.
Official specifications show that the revolutionary CS-3 system not only delivers unmatched speed, but can also easily scale to 24-trillion-parameter models on a single logical device!
As one observer marveled: "If this really is the full version of Sol running on Cerebras, then the perceived upper limit of model size that everyone has taken for granted has been completely shattered tonight."
The Real Hidden Ace: OpenAI's First Self-Developed Chip "Jalapeño"
Just recently, OpenAI officially launched the first self-developed chip in its history: Jalapeño.
The arrival of this chip directly explains the deeper logic behind OpenAI's partnership with Cerebras: through exploration on third-party state-of-the-art inference hardware, OpenAI has fully mastered the critical strengths and value of dedicated inference architectures, and translated that knowledge into its own fully controllable underlying platform.
Jalapeño is one of the milder chili pepper varieties. OpenAI's choice of this name clearly signals: this is just the appetizer.
This chip is a custom ASIC designed exclusively for large language model inference. From the very first circuit layout, every transistor is optimized for one single purpose: running large AI models.
Surprisingly, Jalapeño not only runs OpenAI's own models, but its architecture is also compatible with LLMs across the entire industry, demonstrating the company's grand platform ambitions.
Remarkably, the entire design and tapeout process for this chip took only 9 months.
Behind this achievement lies an extraordinarily powerful industry alliance:
Architecture Leadership: OpenAI personally led the underlying architecture design.
Chip Implementation and Interconnects: Semiconductor giant Broadcom provided robust manufacturing capabilities and advanced network interconnect technology support.
System Integration: Celestica handled final board manufacturing and rack-level physical integration.
Taking Over the Entire Industry Chain: OpenAI's Full-Stack Empire Ambition
Training models in-house, designing chips independently, optimizing inference on its own terms, and controlling deployment end-to-end.
Clearly, OpenAI's ultimate goal is to build a massive full-stack AI empire.
But OpenAI's ambitions are even more audacious than Apple and Google's, as it possesses an unprecedented super flywheel: using AI to accelerate the construction of AI infrastructure, then leveraging that more powerful infrastructure to run even stronger AI systems.
According to OpenAI's published ambitious roadmap, the first gigawatt-scale super data centers will start deployment in partnership with key collaborators like Microsoft by the end of 2026.
The total power consumption of a mid-sized city will be dedicated to powering inference racks for Jalapeño and next-generation "chili" chips.
Get ready: soon we will witness GPT-5.6 Sol blazing across Cerebras wafers at 750 tokens per second, shattering the physical limits that once constrained model parameters and inference speed.
References: https://x.com/bleysg/status/2073937651150029084
This article is sourced from WeChat public account "AI Era", authored by ASI Revelation; edited by Aeneas, republished with authorization from 36Kr.