A rising star in AI memory, SOCAMM2 makes its debut
It is reported that NVIDIA has canceled the promotion of its first-generation SOCAMM memory module and shifted its development focus to a new version called SOCAMM2.
Not long ago, NVIDIA said it planned to deploy 600,000 - 800,000 SOCAMM memory modules for its AI products this year. However, it is reported that technical problems were subsequently discovered, the project was put on hold twice, and no actual large-scale orders were placed. Currently, the development focus has shifted to SOCAMM 2. NVIDIA has started collaborating with Samsung Electronics, SK Hynix, and Micron to conduct sample tests on SOCAMM 2.
01
What is SOCAMM2? What are its advantages?
The SOCAMM technology is positioned as a new high-bandwidth, low-power memory solution for AI servers. Its design goal is to provide performance similar to that of HBM (High-Bandwidth Memory) while effectively reducing costs. By using LPDRAM in combination with Compressed Attached Memory Module (CAMM), it offers excellent performance and energy efficiency in a revolutionary new form factor. It saves more space compared to traditional DDR5 RDIMM configurations and consumes one-third less power.
Previously, NVIDIA had clearly included SOCAMM-related information in its product documentation. The GB300 NVL72 specification sheet shows that this product can support up to 18TB of SOCAMM based on LPDDR5X, with a bandwidth of up to 14.3TB/s. However, due to the motherboard design changes of NVIDIA's GB300 "Blackwell Ultra", the SOCAMM module will not immediately enter commercialization. Nevertheless, this technology is still expected to become part of the Vera Rubin platform, providing a more flexible and easily maintainable non-onboard memory option for the Vera CPU.
It should be noted that SOCAMM and HBM are not in direct competition but complementary. SOCAMM solves the flexibility problem, while HBM addresses the need for extreme performance through advanced packaging integration with GPUs.
SOCAMM 2 may support the LPDDR6 memory specification
From a technical perspective, SOCAMM 2 inherits the 694 input/output (I/O) port design of SOCAMM 1 in its hardware architecture. This stable underlying architecture lays the foundation for its performance upgrade. Notably, the data transfer speed of SOCAMM 2 has significantly increased to 9,600MT/s, representing a remarkable increase of approximately 12.5% compared to SOCAMM 1's 8,533MT/s. This speed increase means that SOCAMM 2 can transfer more data in the same amount of time, providing stronger data throughput capabilities for the high-speed computing of AI servers.
More forward-looking, SOCAMM 2 is very likely to support the next-generation low-power memory, LPDDR6. As the successor to LPDDR5X, LPDDR6 has achieved qualitative leaps in many key performance indicators. In terms of the channel architecture, LPDDR6 abandons the 16-bit single-channel design of LPDDR5 and innovatively adopts a 24-bit wide channel (two 12-bit sub-channels) architecture, with each sub-channel equipped with an independent 4-bit CA bus, significantly improving the parallel processing capability. Its burst length is set to 24 (BL24), forming a 288-bit data packet, of which 256 bits carry valid data, and the other 32 bits are assigned special functions for error checking, memory marking, or implementing the Data Bus Inversion (DBI) technology, ensuring data accuracy while significantly reducing power consumption during write operations.
In terms of performance, the memory bandwidth of LPDDR6 is as high as 38.4GB/s, which is a significant improvement compared to LPDDR5. JEDEC has confirmed that its power consumption will be approximately 20% lower than that of LPDDR5. In light-load scenarios, the operating mechanism of the Efficiency Mode introduced in LPDDR6 can enable only one sub-channel to access all 16 memory banks, while the other sub-channel enters standby or deep sleep mode. The flexible resource configuration further optimizes energy consumption. At the same time, the newly added ALERT signal line in LPDDR6 enables DRAM chips to actively report errors to the processor, supporting real-time reporting of 15 different types of faults, greatly enhancing the data integrity guarantee capability.
If SOCAMM 2 successfully integrates LPDDR6 technology, the two will form a powerful synergistic effect. On the one hand, the high-performance characteristics of LPDDR6 will help SOCAMM 2 achieve further breakthroughs in bandwidth, energy efficiency ratio, and data processing stability, further widening the gap with traditional memory solutions and better meeting the strict requirements of AI servers for high-speed and stable processing of massive data. On the other hand, the modular design of SOCAMM 2 will also expand new application spaces for LPDDR6, enabling it to be more conveniently applied in the field of AI servers and accelerating the commercialization process of LPDDR6. However, currently, LPDDR6 is still in the technical verification stage, and there are certain uncertainties in its compatibility with existing platforms, mass production costs, and stability after large-scale application. This is also an important reason why relevant suppliers have not officially confirmed this function.
02
How to integrate into the AI server ecosystem?
SOCAMM2 is not an isolated memory product. Its full value can only be realized through in-depth collaborative adaptation with the AI server ecosystem. From a hardware perspective, the motherboard design of AI servers, the compatibility between CPUs and memory controllers, and the layout of the cooling system all need to match the characteristics of SOCAMM2. Take NVIDIA's Vera Rubin platform as an example. As the core architecture for next-generation AI computing, this platform places higher requirements on the flexibility and maintainability of memory.
The non-onboard memory design of SOCAMM2 precisely meets the requirements of the Vera Rubin platform for independent upgrade and replacement of memory modules without the need to replace the entire motherboard, significantly reducing the maintenance cost and upgrade cycle of servers.
At the software level, the optimization of the operating system, AI frameworks, and device drivers is also crucial for SOCAMM2 to exert its performance. For example, in AI training scenarios, mainstream frameworks such as TensorFlow and PyTorch need to adjust algorithms according to the memory bandwidth characteristics of SOCAMM2 to achieve efficient data scheduling and cache management, avoiding the waste of computing resources caused by underutilization of memory performance. At the same time, the memory management device driver needs to support the multi-channel parallel transmission mode of SOCAMM2 to ensure that memory resources can be reasonably allocated during multi-task concurrent processing, reducing data latency and conflicts.
In addition, the collaboration between SOCAMM2 and AI acceleration chips is also particularly important. Currently, mainstream AI acceleration chips such as NVIDIA's GB300 series chips and AMD's MI300 series chips need to consider compatibility with new memory during the design stage. Take NVIDIA's GB300 NVL72 as an example. Its specification sheet clearly supports up to 18TB of SOCAMM based on LPDDR5X, with a bandwidth of up to 14.3TB/s. This parameter setting is the result of the pre-collaborative research and development between chip manufacturers and memory manufacturers. Through the in-depth adaptation of chips and memory, the data transmission link can be optimized, signal interference and transmission loss can be reduced, and the overall computing efficiency of AI servers can be further improved.
03
Competition among major memory manufacturers heats up
With the explosion of AI applications, the demand for high-bandwidth, low-latency memory has increased sharply. Traditional DDR memory is difficult to meet the requirements of AI servers for fast reading and writing of massive data. Although HBM has excellent performance, it cannot be fully popularized due to high costs and limited production capacity. The emergence of SOCAMM2 is just in time. Its integration of LPDDR technology advantages is expected to provide performance close to that of HBM at a more affordable cost, filling the market gap.
Against this background, many major memory manufacturers have placed their bets on the SOCAMM2 track.
Micron, with its technical accumulation and market response speed, took the lead in the SOCAMM field. In March this year, it announced that it would deliver SOCAMM modules based on LPDDR5X to customers, becoming one of the earliest manufacturers in the industry to achieve the delivery of such products. This move has enabled it to accumulate market application experience and establish cooperative relationships with some AI server manufacturers, laying the foundation for the subsequent promotion of SOCAMM2. However, whether its first-mover advantage can be maintained depends on its subsequent product iteration and cost control capabilities.
As leading enterprises in the memory industry, Samsung and SK Hynix, although slightly behind Micron in the launch progress of SOCAMM products, are continuously making efforts in the R & D and mass production preparation of SOCAMM2 with their technical reserves and industrial chain integration capabilities.
The SOCAMM2 design scheme presented by Samsung focuses on product compactness and cooling management optimization. Through improvements in packaging technology and the application of cooling materials, it enhances the stability and reliability of the product in high-density server environments to meet the requirements of long-term high-load operation. SK Hynix, on the other hand, focuses its R & D on improving memory speed and stability. By improving the manufacturing process of memory chips and optimizing circuit design, it enhances the stability of SOCAMM2 in data transfer speed, latency control, and multi-module collaborative work. Currently, both enterprises maintain technical cooperation with NVIDIA, participate in the sample testing of SOCAMM2, and optimize product design based on the test feedback. They plan to promote mass production early next year.
Domestic storage manufacturers are also actively participating in the market competition of SOCAMM2. As a representative enterprise in the domestic storage industry, Foresee, based on its technical accumulation in storage chip design, main control chip development, packaging testing, and production and manufacturing, has launched its self-developed SOCAMM2 product. This product, while retaining the general advantages of SOCAMM technology, is optimized for the characteristics of the domestic server market. For example, it removes the trapezoidal structure protruding from the top of LPCAMM2 to reduce the overall height and improve the compatibility with the common server installation environment and liquid cooling systems in China. This localized optimization strategy may help it win cooperation opportunities with domestic AI server manufacturers, expand its share in the enterprise-level storage market, and promote the breakthrough of domestic storage technology in the field of AI memory. With the development of the domestic AI industry and the increasing demand for self-controllable storage products, domestic manufacturers may narrow the technical gap with international giants by leveraging their advantages in the domestic market.
04
How far is commercialization?
In the process of commercialization, SOCAMM2 faces challenges in technology, cost, and ecosystem, and also has certain market development opportunities. Its development trend will have an impact on the pattern of the AI memory industry.
Technical compatibility is the primary challenge for the commercialization of SOCAMM2. As a new memory module, SOCAMM2 needs to be compatible with the hardware architecture, operating system, and AI application frameworks of existing AI servers. Currently, the memory controllers of some old servers may not support the high transfer speed and special instruction set of SOCAMM2. If enterprises want to adapt to SOCAMM2, they need to upgrade or replace the hardware, which will increase the initial investment cost. This may cause cost-sensitive small and medium-sized enterprises to postpone adoption. At the same time, the adaptation and optimization of the operating system and AI frameworks require time and resources. If the software adaptation progress lags behind the launch of hardware products, the performance of SOCAMM2 cannot be fully released, affecting market acceptance.
Cost control is also crucial for the commercial promotion of SOCAMM2. Although SOCAMM2 is positioned as a "low-cost alternative to HBM" solution, compared with traditional DDR5 memory, there is still a gap in R & D and manufacturing costs due to the use of LPDDR5X/LPDDR6 chips and modular design. Especially, the LPDDR6 chips are in the technical verification stage, and the cost is relatively high in the initial stage of mass production, which will directly affect the pricing strategy of SOCAMM2: If the price is set too high, it will lose the cost competitive advantage compared with DDR5 memory. If the price is set too low, it may compress the profit margin of memory manufacturers and affect their enthusiasm for R & D investment. In addition, the production yield of SOCAMM2 will also affect the unit product cost. A low yield will further push up the cost and restrict large-scale commercialization.
From the perspective of market opportunities, the continuous expansion of the AI industry will drive the growth of demand for high-bandwidth memory.
According to the latest research by TrendForce, in 2025, due to the continuous strong demand and the relatively high average selling price of products, it is expected that the value of the AI server sub - market will rise to $298 billion, and AI servers will account for more than 70% of the total value of the entire server industry. As a core component, the demand for memory will increase synchronously. SOCAMM2 has the characteristics of "high bandwidth, low power consumption, and low cost", which is in line with the needs of mid - to high - end AI servers and has application space in scenarios such as AI training, inference, and cloud computing.
In addition, the development of SOCAMM2 may promote the technological iteration of the memory industry. Facing the competitive pressure brought by SOCAMM2, memory products such as HBM and DDR5 may accelerate technological upgrades to further improve performance and reduce costs, forming a healthy competitive pattern in the industry. This competitive situation may prompt the memory industry to break through technological bottlenecks, provide better memory solutions for the AI industry, and promote the overall improvement of AI computing performance.
This article is from the WeChat official account "Semiconductor Industry Insights" (ID: ICViews), author: Fang Yuan. It is published by 36Kr with authorization.