Nearly 2 billion yuan raised within 5 months, MoCore, the domestic leader in the sparse computing track, has secured a new round of financing.

Reported by TechCore on November 3, today, domestic sparse computing chip developer MoXi Artificial Intelligence (hereinafter referred to as "MoXi") announced that it has recently closed a financing round of nearly 1 billion RMB. This round of financing came only 5 months after the completion of its previous financing of nearly 1 billion RMB, bringing the total amount of the two rounds of financing to nearly 2 billion RMB.

This round of financing for MoXi was jointly participated by industrial investors including Ren'ai Capital, Begonia Fund, Dongheng Petroleum, as well as well-known state-owned and financial investment institutions such as Biqiu Capital, Paila Capital, Zhongcai Capital. The raised funds will be invested in the full-stack upgrade of sparse dedicated inference chips, focusing on the pre-research of next-generation chips, and promoting the construction of the sparse computing industry-university-research alliance at the same time.

Why are capital players increasing their bets at this point in time? What changes have taken place in MoXi's commercialization? How does sparse computing turn into visible numbers on customers' account books? With these questions in mind, TechCore had an in-depth conversation with Guo Weijun, Vice President of Marketing at MoXi.

01.

Seize the Bursting Point of Inference Demand

Comprehensive Acceleration of Commercialization

To understand this round of financing of MoXi, we need to look at it against a larger industry background.

Over the past year, large models have truly entered production environments, and the volume of inference calls has kept rising, leading to a corresponding surge in computing bills. Nowadays, Token consumption is no longer project-based, but a fixed monthly "computing power bill" just like utility bills for water and electricity.

Against this backdrop, customers have put forward higher requirements for next-generation inference computing power. Guo Weijun sums up customer demands into three points: first, reduce the comprehensive inference cost, which includes not only the Token bill, but also the long-term accompanying expenses such as electricity cost, deployment cost and compatibility cost; second, ensure flexible computing power supply and deterministic delivery; third, control the adaptation cost, so that existing models and business systems can complete migration with low barriers.

At present, there are many ways to improve the overall inference efficiency of the system, and sparse computing is regarded as one of the most noteworthy directions. What sparse computing aims to solve is actually a simple problem: large models often have tens of billions or even hundreds of billions of parameters, but only a small part of them are actually activated and participate in calculation in each inference. Traditional chips make no distinction between these parameters, which is called "dense computing" in the industry; sparse computing takes the opposite approach, only calculating the part that actually works and skipping the rest. For the same inference task, the amount of calculation is reduced, and the cost will drop accordingly.

MoXi has been deeply engaged in the technical verification of sparse computing for many years, and recently, these accumulations have begun to be cashed out intensively. These two rounds of financing are an important watershed for MoXi to move from years of single-point technical verification to large-scale and commercialized delivery.

At the technical level, MoXi's "algorithm-software-hardware" co-design idea has been verified in real products, and the end-to-end software closed loop is also taking shape gradually. MoXi's S30 and S40 computing cards have won the first place in three consecutive MLPerf Inference tests, and its engineering capabilities are endorsed by public tests.

At the delivery level, a growing number of intelligent computing centers and industry customers are also including sparse computing power in their candidate options. MoXi's dedicated sparse inference chips have been deployed across four major regions: Northwest China, Southwest China, East China and North China, with thousand-card clusters deployed. It has also established cooperation with leading telecom operators to integrate the sparse computing inference solution into the other party's computing power service system, and MoXi's large-scale delivery capability has also been verified in these projects.

At the market level, Guo Weijun also observed a trend: not every industry needs a large general model. With the continuous enrichment of AI applications in vertical industries, sectors including finance, healthcare, exploration and meteorology all have their own industry-specific models, almost all of which can benefit from sparse computing, and their deployment scenarios are showing the possibility of cross-industry replication.

02.

Sparse Computing Enters Real Production Environments

MoXi Helps Customers Figure Out Clear Cost Accounts

In the past few months, MoXi's list of deployed projects has grown rapidly, ranging from university scientific research platforms to national computing power projects. We also talked about several real customer cases.

In scientific research scenarios, MoXi's sparse computing power has obtained a set of quantifiable measured results. For the cutting-edge field of spatiotemporal AI and spatiotemporal big data, MoXi, in collaboration with the Institute of Earth and Space Technology of Shanghai Lingang International Innovation Center of Peking University, completed the adaptation and actual measurement of two internationally mainstream basic models for intelligent analysis of remote sensing image data on MoXi's computing cards based on self-developed sparsification technology.

The data shows that under the conditions of almost no loss of business accuracy and no need to retrain downstream tasks, the model inference throughput is increased by up to 2.88 times, and the model compression ratio reaches up to 7.64 times, which significantly reduces the computing power consumption and processing time of large-scale remote sensing image interpretation.

If the Peking University project verifies the effect of a single scenario, the next computing power project tests the integration capability of sparse computing power with the existing system. MoXi participated in the country's first offshore computing power Token factory built by a Shanghai state-owned platform company: this project uses green power from offshore wind power and seawater cooling to reduce PUE and electricity bills, but the money saved by green power may be offset by the inference cost of general-purpose GPUs.

MoXi's approach is to reduce the inference cost per Token at the chip level, and then cooperate with the hybrid heterogeneous computing power mode, so that "cost reduction from green power" and "cost reduction from architecture" are superimposed to further amplify the cost reduction effect.

It is worth noting that when communicating with customers, MoXi does not deliberately emphasize the technical concept of sparse computing. "Customers are most concerned about whether the product is easy to use, and will not pay too much attention to whether it adopts the sparsification route in the early stage of the project."

In actual commercial cooperation, customers pay more attention to the overall input and output: how many servers need to be deployed in the computer room, what the power consumption and operation and maintenance costs are, and whether the final result can meet the business accuracy and ROI requirements.

To this end, MoXi converts product value into specific cost calculation, and starts to build a standardized pre-purchase simulation test system, so that customers can obtain corresponding simulation data before procurement to evaluate the actual benefits of the product in their own business scenarios.

Compared with performance indicators, migration cost is often a higher threshold for customers to make decisions. Most of the existing evaluation scripts and scheduling systems of customers are built around the dense computing architecture. Replacing chips not only requires re-regression testing, but also involves engineering transformation and business grayscale release, bringing additional time and capital investment.

To solve this problem, on the one hand, MoXi helps customers build a cross-architecture evaluation and supplier comparison system through sparse evaluation standards and simulation tools; on the other hand, it also provides a heterogeneous adaptation toolchain to promote the collaborative work of sparse cards and customers' existing dense cards. For customers who have already deployed a large number of stock chips, this method can reduce the cost of system migration and repeated construction, and also provides another path for the implementation of sparse computing.

03.

From "Usable" to "User-Friendly"

Software and Ecology Provide Sustained Momentum for Commercialization

Reducing the usage threshold for customers is only the first step for sparse computing to move towards commercialization. To achieve wider deployment, continuous product iteration, improved software toolchain and joint support from the industrial ecology are also required. MoXi is promoting the layout from three directions.

In the academic world, a mature collaborative system has not yet been formed among the algorithms, mathematical theories, hardware architectures and software toolchains of sparse computing, and many key issues remain to be further studied. To solve this problem, in August this year, MoXi initiated the establishment of China's first sparse computing industry-university-research alliance, and has launched a number of joint research projects with universities including Fudan University, Tsinghua University, Xi'an Jiaotong University, Nankai University and Shanghai Jiao Tong University.

Guo Weijun revealed that the joint projects within the alliance are arranged in combination with the disciplinary characteristics of each university: the School of Computer Science focuses on the direction of how the combination of "algorithm + hardware architecture" can improve the performance of sparse computing; the School of Mathematics focuses on the calculation efficiency of matrix multiplication algorithms and the positioning of sparse parameter rules; the software discipline focuses on the development of sparse toolchains, compilers and high-performance operators.

▲ Planning of the Sparse Computing Industry-University-Research Alliance

The task undertaken by the industry-university-research alliance is not only scientific research cooperation, but also to fill the basic elements required for the industrialization of sparse computing. In Guo Weijun's view, at present, the development of the industry still faces three shortcomings:

First, there is a shortage of compound talents. Talents who understand both algorithms and sparse mathematics, and are familiar with chip architectures are relatively scarce, and the training cycle for relevant talents usually takes 3 to 5 years;

Second, the evaluation and acceptance standards have not been unified. The existing computing power delivery standards are mainly formulated around dense computing, which is difficult to accurately measure the actual capability of sparse computing. Guo Weijun said that MoXi may launch acceptance standards for sparse computing power in the future;

Finally, basic resources such as the toolchain and datasets for sparse computing still need to be improved.

Guo Weijun believes that although these basic works are low-profile and not highly concerned, they are indispensable links for the large-scale development of the sparse computing industry.

Beyond the ecosystem, MoXi has not stopped the pace of iterating its product lines. Its new generation board card SP200 is scheduled to be launched before the end of this year. Guo Weijun revealed that the real pain points exposed in previous customer usage have been incorporated into the design of the next-generation architecture.

In terms of the software stack, MoXi's upgraded software stack SECA (Sparse-native Efficient Computing Architecture) is about to be released. The entire software stack, from the operator library and compilation to software-hardware co-design, is redesigned around the sparse architecture, and the supporting MaaS platform and model framework adaptation management tools will also be launched one after another.

From ecological construction and product iteration to software stack upgrade, MoXi's sustained momentum in commercialization is worth paying attention to.

04.

Conclusion: Has the Inflection Point for Sparse Computing Arrived?

The application space of sparse computing is rapidly expanding from large models to more industries. In Guo Weijun's view, models in vertical fields are customized for specific businesses, and do not need all the capabilities of general large models, making them naturally suitable for sparse computing.

For example, in the industrial quality inspection scenario, what the vision model needs to identify is often only the local defect area in the image, and a large amount of background information does not directly participate in the judgment. Sparse computing is expected to reduce unnecessary computing overhead. Embodied intelligence and world models also have similar demands. Guo Weijun analyzed that the human brain does not process all the information in the surrounding environment to the same extent, but filters and pays attention to key information according to the current task, which coincides with the mode of sparse computing.

For MoXi, it has passed the key node of proving that sparse computing is effective. Now, the company is proving through one real commercial case after another that sparse computing can be deployed on a large scale, continuously reduce costs, and bring quantifiable business benefits to more industries.

This article is from the WeChat Official Account "Core Things", written by Chen Junda, and authorized for release by 36Kr.