The details of Zen6 have finally been fully unveiled: how AMD pushes EPYC into the era of agent AI
In the data center sector, AMD has already gained overwhelming competitive advantages.
The market share of AMD server CPUs has skyrocketed from roughly 0.2% in 2017 to 46% at present. Over the past decade, AMD has grown from a barely noticed challenger to a market player that holds half of the market share.
This year, AMD has stepped into an ultra-high-speed development mode, launching a series of new products that all point to the same direction: Agentic AI.
The first product is the Zen 6 architecture, which introduces a flexible computing base specifically designed for the agent era at the architecture layer.
The second product is the 6th-generation EPYC processor Venice, which has up to 256 cores and launches a CPU product family oriented to Agentic AI at the chip layer.
The third product is the Helion rack, which takes CPU as the core to schedule and execute agent tasks, and launches a rack-level reference design specially built for Agentic AI at the rack layer.
On November 4 local time in the United States, at the AMD In Depth conference held in Las Vegas, AMD elaborated on the architectural design of Zen 6, the product layout of Venice, and the rack-level deployment ideas for AI workloads, and also announced the product roadmap extending to 2030.
It is worth mentioning that this time AMD introduced the MEXT memory optimization technology in depth for the first time. According to the test results provided by AMD, in specific agent AI workloads, the DRAM usage is reduced by 50%, while the system performance can still reach 99% of that of the full DRAM configuration. This also indicates that AMD's focus is no longer limited to the computing core itself. How to make data flow to CPUs and GPUs more efficiently and at lower cost has become a key part of AMD's AI infrastructure layout.
From cores to processors, from memory to racks, the entire link is built around data centers in the AI era. Behind these new AMD products, it is not only performance upgrades, but also AMD's rethinking and trade-offs for computing infrastructure in the AI era.
Zen 6 makes a stunning debut: AMD's thinking on architecture in the AI era
Let's first look at the most hardcore part of this launch, the Zen 6.
Previously, AMD had announced that the 6th-generation EPYC supports up to 256 cores, but the specific internal adjustments to the cores were not clearly explained. This time, AMD further disclosed the details of the front end, execution units, cache and instruction set.
Mike Clark, AMD Architect, said on site that Zen 6 has enhanced two design points at the same time: one is the throughput capacity of the entire processor, and the other is the execution capacity with high frequency and low latency. Corresponding to the products, they are Zen 6c and Zen 6.
First, in terms of core count, high density supports concurrent processing while high frequency accelerates key steps. Zen 6c expands a single core complex (CCX) from 16 cores to 32 cores, the shared L3 cache increases from 32MB to 128MB, the maximum frequency is raised from 3.7GHz to 4.1GHz, and the maximum number of cores of the entire processor is expanded from 192 to 256. The CCX of Zen 6 increases from 8 cores to 12 cores, the L3 cache increases from 32MB to 48MB, and the maximum boost frequency is raised from 5.0GHz to 5.3GHz.
These two design paths correspond to the AI scenarios in the agent era. A large number of sessions, sandboxes and tool calls run at the same time, requiring more cores to undertake the work; certain code execution, database access or serial steps will directly affect the completion time of the entire task, which requires stronger single-threaded performance.
There is also an interesting detail here: the floating-point scheduler of Zen 6c has 3 groups of 60 entries each, while the high-frequency version of Zen 6 has 46 entries per group. Clark explained that the higher the frequency, the shorter the time left for the circuit to complete work in each cycle, and the internal structure also needs to meet tighter timing requirements. Since the frequency target of Zen 6c is relatively lower, AMD uses this part of the design space to increase scheduling resources, so that the processor can find executable work from more pending operations.
Secondly, as the number of cores increases, cache and memory access capabilities also need to keep up. For the cache of Zen 6c this time: the number of cores is doubled, and the L3 cache is directly expanded to 4 times the original size. Converted by the number of cores, the average L3 capacity per core increases from 2MB to 4MB, and each core retains 1MB of private L2 cache. The 128MB L3 cache is shared by 32 cores, and active cores can use it on demand. For programs that need to share data, a larger complex also helps reduce cross-complex communication. The role of cache is not difficult to understand. The CPU calculates very fast, but it needs to wait when fetching data from main memory. The databases, runtimes and tool programs called by Agent contain a large number of branches and irregular memory access. Keeping frequently used data near the cores can reduce these waiting times. AMD has also adjusted the prefetch strategy: the prefetch is more aggressive when there are few threads and the memory is idle; when a large number of cores compete for bandwidth, it controls invalid data movement to avoid the prefetch itself blocking the data path. The space for finding executable instructions inside the core has also been expanded. Zen 6 inherits the dual basic block fetch and pipeline foundation laid by Zen 5, expands the Op Cache that stores decoded operations to 6.75K Ops, can provide up to 18 operations per cycle, and the subsequent dispatch width is 8-way. The out-of-order window of Zen 6c increases from 448 entries to 640 entries, the integer physical registers increase from 240 entries to 352 entries, and the Load/Store Queue increases from 64/96 entries to 80/128 entries. The out-of-order window can be understood as the to-do list in front of the CPU: if the previous work is waiting for data, it can find the subsequent work that can be done first. The expansion of the window and supporting resources leaves more room for scheduling work. This is highly related to the load at the agent execution layer. The real software it calls usually contains a large number of branches, function calls and irregular memory access, and the processor needs to continuously find promotable work in these complex paths.
Finally, in terms of vector computing, Zen 6 continues the 512-bit floating-point path and adds full AVX-512 FP16 computing support. The INT8 VNNI execution pipeline is expanded from 2 lanes to 4 lanes, and the Bit Matrix Multiply capability is added. Data loading supports two 512-bit operations per cycle, or four 256-bit operations. The total amount of data moved by the two modes is equivalent, and the improvement is that the existing 256-bit code can also make full use of the path. Enterprise software updates slowly, and this kind of compatibility is very practical.
Putting all these changes together, AMD's idea is quite clear: high-frequency cores speed up key steps, high-density cores undertake a large number of concurrent tasks, cache, execution window and data access capabilities help the cores keep working continuously, and vector and system capabilities expand the scope of application.
So how much improvement have these changes brought? In tests at fixed frequency and fixed 16 cores, compared with Zen 5, Zen 6c achieves a geometric average IPC improvement of about 20% on the selected enterprise and cloud workloads, about 31% on AI workloads, and an overall improvement of about 27%. Clark frankly said that AMD has long believed that CPUs will occupy a place in the AI software ecosystem, so it continues to strengthen general-purpose computing, vector capabilities and system efficiency. The rise of agents amplifies the value of these designs. This also corresponds to AMD's positioning for the Zen 6 architecture: Flexible Computing for the Agentic Era.
EPYC 9006 product lineup: AMD's thinking on CPUs in the AI era
At the AMD in Depth event, AMD showcased a 96-core Venice with a maximum boost frequency of 5.3GHz. This high-frequency Venice is only one member of the 6th-generation EPYC product portfolio. SP7 also has a high-density version with up to 256 cores and supports 16-channel memory; SP8 covers 8 cores to 128 cores, targeting enterprise servers; Venice-X uses 3D V-Cache to serve EDA and high-performance computing; Verano adopts LPDDR5X, focusing on energy efficiency for AI host nodes.
The reason why AMD prepares so many types of CPUs is related to how agents work. In the past, users raised questions and the model gave answers. Now, to let an Agent complete a task, it may first query the database, then write code and run tests, and if it finds that the result is incorrect, it will go back and re-execute. The user only issues one instruction, but the background runs several rounds of operations. The model is responsible for planning and judgment, and executing these plans will mobilize a large amount of CPU resources.
AMD divides CPUs in the agent infrastructure into three roles: the first type is AI Host Node, which acts as the host for GPUs; the second type is Agent Sandbox, which provides a sandbox for code execution and tool calls; the third type is general-purpose computing node, which runs the original enterprise software.
To be a host for GPUs, the first requirement is high speed. If steps such as preparing data, submitting tasks, and processing requests get stuck on the CPU, the subsequent GPUs may also be idle. For this kind of work, improving the processing speed of key threads is often more useful than continuing to add cores. The 96-core Venice with a maximum frequency of 5.3GHz demonstrated on site is targeted at this type of demand. AMD has also prepared another solution for host nodes: Verano. It provides up to 192 Zen 6c cores or 72 Zen 6 cores, adopts 24-channel LPDDR5X-8533, and reduces the energy consumption of data movement between chiplets through EFB interconnection. Of course, the memory uses replaceable SOCAMM modules. The LPDDR5X here is worth elaborating. Judging from the agent CPUs launched by AMD, Intel and NVIDIA this year, most of them are equipped with LPDDR5X products. For example, NVIDIA Vera also uses LPDDR5X and SOCAMM, with a maximum memory bandwidth of about 1.2TB/s, which is mainly because data centers are facing increasingly severe power and heat dissipation constraints. LPDDR5X was originally applied to mobile and edge devices. Now, while providing higher memory bandwidth, its power consumption is only a fraction of that of traditional DDR5 implementations. The transition to LPDDR5X is a key evolution of server architecture.
For Agent sandboxes, AMD brings out high-density cores again. Agents execute code, process files, and call tools, which usually need to be completed in an isolated environment. These tasks may only run for a few seconds, but will be started hundreds or thousands of times repeatedly. The server needs to start quickly and accommodate enough tasks at the same time. The 256-core, 512-thread configuration of SP7 is targeted at the concurrent demands of agents. According to the test results, for short-time tool tasks, the performance of the Xeon 6980P platform is set as 1, the 192-core Graviton5 reaches 1.4, the 192-core Turin reaches 1.7, and the 256-core Venice reaches 2.5.
The demands of enterprise servers are different from dedicated AI clusters. Databases, business systems, virtual machines and retrieval services will not stop just because Agents are deployed. Software that was originally clicked and queried by humans may now be called continuously by Agents. The more in-depth the AI application is, the busier these background systems will be. SP8 is the configuration provided by AMD. According to AMD, the design of SP8 comes from in-depth communication with large enterprises and OEM customers. It covers 8 cores to 128 cores, adopts 8-channel memory with 2 DIMMs supported per channel, and the maximum TDP of the platform is 400W. According to the specific configuration, a single-socket system can provide up to 128 PCIe 6.0 lanes, and a dual-socket system can provide up to 192 lanes.
In enterprise and cloud workload tests, with the 128-core Xeon 6980P as the benchmark, the 256-core EPYC 9996 delivers about 3.6 times the performance in MongoDB (YCSB), about 2.8 times in MySQL transaction processing test (TPC-C), and about 3.5 times in Redis read test. There is a very interesting trade-off here: compared with SP7, the number of memory channels of SP7 is reduced from 16 to 8, but the upper limit of single-socket PCIe lanes is increased from 96 to 128. Storage and business servers may not need 256 cores, but they need to connect a large number of NVMe SSDs, network cards and other devices. SP8 reserves configuration space for these purposes, so that enterprises do not have to purchase the highest computing specifications that they do not use just for I/O expansion. In a set of server consolidation calculations demonstrated on site, AMD's results show that the performance of a single server is about 7.4 times that of the original solution, the number of servers is reduced by about 86%, and the total 5-year total cost of ownership is reduced by about 37%.
