The development of the world's first RISC-V in-memory computing standard has been launched.

In the current era when large AI models are developing rapidly and the demand for computing power is growing exponentially, the Chinese chip industry is facing three core pain points. The blockade of advanced chip process technologies and the inherent limitations of traditional architectures have made computing power density, software and hardware ecosystems, and data bandwidth the "three major obstacles" restricting industrial upgrading.

36Kr learned that the 3D-CIM (Three-dimensional Computing-in-Memory) technology introduced by Micro-Nano Core at the RISC-V Computing-in-Memory Industry Forum and the Launch Conference of the Application Group, which is deeply integrated with the open-source and flexible RISC-V architecture, is becoming the core path to break through the predicament and promote the iteration of domestic chips.

1. The "Three Major Obstacles" of the Domestic Chip Industry

1) Bottleneck in computing power density due to the lack of advanced processes

Currently, the domestic 3nm/5nm advanced processes are still in the R & D stage and are difficult to mass-produce in the short term. If traditional process chips continue to use the von Neumann architecture, the computing power density is relatively low, far from meeting the operating requirements of large models with hundreds of billions of parameters.

2) Bottleneck in software and hardware ecosystems due to parasitic development ecosystems

The domestic AI chip industry has long relied on the closed-source ecosystems of the United States and the West. In particular, the CUDA ecosystem almost monopolizes the software link for AI model training and inference. Once the external ecosystem is restricted, even if an enterprise has high-performance chips, it will face the dilemma of having hardware but no software.

3) Bottleneck in software and hardware bandwidth under the traditional von Neumann architecture

Under the traditional von Neumann architecture, the computing and storage units are separated, and data needs to be frequently transferred through the bus, forming a "memory wall" bottleneck. When the parameter scale of large models reaches hundreds of billions, the data transfer volume increases exponentially, and insufficient bandwidth will lead to a sharp decline in inference efficiency.

2. The Three-dimensional Computing-in-Memory Technology Can Break the Deadlock

In response to the above pain points in the chip industry, on September 9th, Professor Ye Le, the Chief Scientist of Hangzhou Micro-Nano Core, gave a report titled "Three-dimensional Computing-in-Memory 3D-CIM: Empowering the RISC-V AI Ecosystem" at the RISC-V Computing-in-Memory Industry Forum and the Launch Conference of the Application Group.

He said that the three-dimensional computing-in-memory technology can be understood as embedding computing power in the memory. It is an innovative chip technology that emerged in the context of the exponential growth of computing power demand from large AI models. This technology can complete calculations within the memory through SRAM computing-in-memory + DRAM three-dimensional stacking, which can fundamentally eliminate the data transfer overhead and is regarded as the core path to continue the growth of computing power in the post-Moore era.

The core breakthrough of the three-dimensional computing-in-memory technology lies in the SRAM computing-in-memory design. By integrating the computing unit and the storage unit, tensor calculations (accounting for 99% of the computing tasks in AI scenarios) can be completed in-situ within the memory, which can significantly improve the computing power density. After multiple tape-out verifications by SMIC, it can achieve a computing power density equivalent to that of traditional NPUs/GPUs at 7nm under the 22nm process, and the computing energy efficiency is improved by 5 - 10 times. In terms of cost, based on the fully domestic supply chain, the cost of this 22nm SRAM computing-in-memory chip is reduced by 4 times compared with that of 7nm chips.

The core R & D team of this three-dimensional computing-in-memory technology is led by Professor Ye Le, the Chief Scientist of Micro-Nano Core, and brings together top talents in the fields of chip design and architectural innovation. The team has been deeply involved in the field of computing-in-memory for more than a decade. In the past six years, they have continuously published 14 measured results of AloT chips that break the current world records at the "International Solid-State Circuits Conference" ISSCC. Their achievements were selected for the "2021 ISSCC Best Chip Presentation Award" (the first time won by a domestic team, along with chips from Intel) and the "2024 ISSCC Best Paper Award" (also the first time won by a domestic team).

3. The Ecosystem and Application Prospects of the Three-dimensional Computing-in-Memory Technology

In addition, the open-source and flexible RISC-V architecture can form a natural complement to the three-dimensional computing-in-memory technology. The integration of the two can not only precisely meet the requirements of large AI models for high parallelism and low-power computing but also effectively relieve the pressure of external process blockades, creating a virtuous cycle for the iteration and performance improvement of domestic chip technologies.

RISC-V is an open-source instruction set architecture originating from the University of California, Berkeley. Its core advantage lies in being "open, flexible, and scalable". Different from X86 (closed-source monopoly) and ARM (licensing fees), RISC-V allows global developers to freely modify and expand the instruction set without paying high licensing fees.

As the leading unit of the RISC-V Computing-in-Memory Application Group, Micro-Nano Core is collaborating with upstream and downstream enterprises in the industrial chain to promote the ecological implementation of the three-dimensional computing-in-memory technology and the RISC-V architecture.

Micro-Nano Core divides the application prospects of the three-dimensional computing-in-memory technology into short - term, medium - term, and long - term. In the short term, it will first enter the application of large models on the edge side, empowering terminal devices such as large model AI mobile phones and large model AIPCs. In the medium term, it will expand to the application of large models on the cloud side. The 3D-CIM chips combined with domestic CPUs/GPUs have the opportunity to bypass or even surpass NVIDIA's benchmark solutions, providing more competitive computing power support for the training and inference of large cloud models. In the long term, it will enter the new field of embodied intelligence (AI robots) applications.

On the application side, Micro-Nano Core has cooperated with many leading mobile phone, PC, and server enterprises. On the supply side, it is integrating industrial chain resources by collaborating with leading domestic process enterprises, RISC-V partners, and leading domestic memory enterprises to accelerate the transformation of the three-dimensional computing-in-memory technology from the laboratory to the market.

(Author: Feng Yaling)