Xinglian's portfolio company Jiliu Technology has completed its Pre-Series A financing

Jiliu Technology completed its Pre-A round of financing at the beginning of this year, led by Lightspeed Photon Partners.

"We are very honored to have the strong support of numerous mentors and friends on Jiliu's entrepreneurial journey, which has given Jiliu the opportunity to participate in the construction of super systems and witness the implementation of general artificial intelligence in China. I am extremely proud of the team's hard work. In the wave of the rapid development of artificial intelligence, we have left our mark," said Hu Xiaohe, CEO of Jiliu Technology, with emotion.

Hu Xiaohe studied at Tsinghua University from undergraduate to doctoral and postdoctoral levels. He studied under Researcher Li Jun and conducted ten years of research on high-performance network systems in the Cybersecurity Laboratory. During his visit to the University of California, Berkeley, he studied under Academician Scott Shenker, the proposer of the SDN network. He is highly proficient in distributed computing and high-performance networks. Before starting his business, he had already implemented the first operator-level Tbps programmable network product in the country and ran a domestic thousand-card large model in a supercomputing environment.

Focusing on the construction of large-scale computer systems was the goal set by Hu Xiaohe at the beginning of his entrepreneurship. What Jiliu Technology is currently developing is a distributed GPU system designed for artificial intelligence, also known as a computing power cluster.

"Since starting the business a year and a half ago, Jiliu Technology has built the largest private single computing power cluster in China," said Hu Xiaohe. "We have broken many established consensuses in the industry. For example, we have proven that AI training is not latency-sensitive but bandwidth-sensitive. We have achieved large model training over a wide area network across 30 kilometers without any loss of computing power, and we can maintain 98%-99% of the computing power across 50 kilometers." This is groundbreaking worldwide.

Zhu Jia, a partner at Lightspeed Photon Partners, said that with the rapid development of large AI models, the demand for high-performance computing power is increasing day by day. However, the barriers to building large-scale cluster computing power are high, and there are currently very few teams in China with the technical ability to build large-scale clusters of over a thousand cards. Jiliu Technology is currently a solution provider for medium and large computing power clusters. It has built and optimized more than a dozen clusters in total and has experience in implementing ten-thousand-card clusters, which is very rare in China.

01

Building a High-Computing-Power Super System

In 1967, Gene Amdahl, a computer architect at IBM, proposed an empirical formula, indicating that the potential for system performance improvement is limited by the parallelizable part of the system. Even if the number of parallel processors increases infinitely, the upper limit of overall performance improvement is greatly restricted.

Simply put, the computing speed of a computing power cluster cannot be infinitely stacked with the increase in the number of GPUs. Just as one person can build a house in ten days, ten people only need one day, but 100 people still need one day - the other 90 people may have to idle because they can't fit into the construction site.

The same is true for training large models. According to a report by Gartner, during the training process of GPT-3.5, a high-performance computing power cluster composed of ten thousand NVIDIA A100 GPUs was used, and this increased to approximately 25,000 A100 GPUs for GPT-4. However, the computing power utilization rate was only 32% to 36%, resulting in serious waste of computing power.

The work of Jiliu Technology is to design a system that can organize thousands or even tens of thousands of people to build more houses as quickly as possible.

Hu Xiaohe said that Jiliu Technology's products are mainly targeted at three dimensions, including the computing power management and scheduling platform, the computing power optimization and operation and maintenance platform, and high-speed interconnection hardware. Currently, in addition to the complete set of solutions for building computing power clusters, the company has also productized and gradually implemented products at the three levels of cluster management, computing engines, and high-speed networks, helping AI enterprises to organize GPUs reasonably and improve delivery efficiency and GPU utilization as much as possible.

Currently, Jiliu Technology's computing power cluster solution can improve the performance of GPU clusters by more than 20%, helping customers save tens of millions of yuan in a thousand-card environment and hundreds of millions of yuan in a ten-thousand-card environment.

02

Early Layout in the Trillion-Dollar Market

According to statistics from IDC, in 2022, the total computing power of global computing devices reached 906 EFlops, with a growth rate of 47%. The computing power industry is booming, and it is expected that the global computing power scale will grow at a rate of more than 50% in the next five years. By 2025, the total computing power of global computing devices will exceed 3 ZFlops (Note: 1 ZFlops = 10E9 TFlops).

In the era of the artificial intelligence explosion, the importance of computing power is no less than that of coal and oil in the industrial era. In the digital era, what is transmitted on the Internet is the information flow, which is the structured abstraction after the rough processing of data by computing power. In the intelligent era, what is transmitted on the Internet is the intelligent flow, which is the model abstraction after the in-depth processing and refinement of data by computing power. It can be said that the recent concentrated explosion of artificial intelligence is inseparable from the "intelligent emergence" generated by the continuous stacking of computing power and data.

However, there are also differences between different types of computing power. Hu Xiaohe mentioned that there is a significant difference between the computing power requirements of traditional Internet services and those of artificial intelligence. Therefore, the network architecture needs to be redesigned, and the relevant technologies are still in the initial stage. There is still a lot of room for imagination in distributed computing, scalability, and even hardware.

He said that in AI computing, especially during the pre-training process of large models, a task needs to run simultaneously on multiple nodes of multiple machines, and this task cannot be split. Therefore, it is necessary to optimize parallel strategies and computing and communication efficiency to improve GPU utilization. This process has very high requirements for scalability.

On the other hand, the scale of some AI computing is very large, possibly involving tens of thousands of connection points. When a computing task runs across multiple nodes, multiple machines, and multiple hops, if a hardware connection point fails, the entire task will fail directly. This requires the design of a new distributed computing engine to meet the needs of fault tolerance, monitoring, and fault resolution.

"This change in the technical paradigm actually stems from distributed computing. Many manufacturers in the market do not have a clear understanding of the technical changes, and there is also a lack of consensus among different roles in the industrial chain," Hu Xiaohe believes. "In our view, artificial intelligence is a development opportunity for ten or twenty years. The development of computing power infrastructure and models has just begun, and market perception will gradually converge and become unified after a period of time."

According to the calculation results of the China Academy of Information and Communications Technology, in 2022, the total computing power of computing devices in China reached 302 EFlops, accounting for approximately 33% of the global total. The growth rate has exceeded 50% for two consecutive years, higher than the global growth rate. The demand for intelligent computing power is showing an explosive growth trend, and its proportion in the computing power scale will become higher and higher. The compound annual growth rate in the next five years will reach 52.3%.

The huge demand for computing power has created a rapidly expanding market scale. IDC data shows that in 2022, the market scale of China's data center services reached 129.35 billion yuan. It is expected that the market scale will exceed 300 billion yuan by 2027, and the market share of servers for inference will reach 62.2%, with the market scale expected to exceed 180 billion yuan.

03

Breaking Through Technical Barriers and Establishing Core Advantages

With the explosive growth of the computing power market, Jiliu Technology has focused its development on the specific implementation of projects. It actively participates in the construction and operation and maintenance of medium and large computing clusters, tries to turn the tools accumulated in the early stage into more standardized products, and explores the adaptation of domestic hardware and going global.

Zhu Jia noticed that Jiliu Technology's high-performance computing power network system based on an open ecosystem already has the advantages of high performance and low comprehensive cost. The company has only been established for a little over a year, and its products have been verified by many customers, showing a very fast development speed.

"The competition in AI computing power is not only about the capabilities of single cards and single GPU chips. The core of AI computing power lies in the network cluster capabilities formed by GPUs. Jiliu Technology's products can enable a large-scale computing cluster to operate efficiently and solve the bottleneck problem in GPU computing power," Zhu Jia said. "We look forward to the company's continuous iteration and growth, and hope it can become the most important infrastructure provider for global AI."

In the year and a half since Jiliu Technology was established, the projects have been implemented in the production environments of first-tier manufacturers. The company has designed, built, optimized, and maintained computing power clusters for multiple data centers, serving multiple manufacturers such as Zhipu AI, SenseTime, Yindun Cloud, and Century Internet, with a total of several thousand to ten thousand computing power clusters. It has also launched a solution for a hundred-thousand-card cluster.

"We hope to form a high-performance computing power network by building such a super system, which will ultimately support the application implementation of artificial intelligence models and the IT iteration of enterprises."

"High-performance computing power infrastructure is the general trend. In future competition, technology will be our core competitiveness," Hu Xiaohe believes. "There are similarities between entrepreneurship and scientific research: 'In scientific research, we need to follow a major direction and make breakthroughs at key points to gain the recognition of review experts; in entrepreneurship, we also need to find a major direction, establish our own advantages in the field, and come up with solutions and products that enterprises need to ultimately gain the recognition of customers and investors.'"

Hu Xiaohe summarized: "From this perspective, whether it is scientific research or entrepreneurship, 'Talk is cheap, show me the code' is the most important thing. This industry has just started to develop. Our products and technologies are in a leading position in the domestic open market, but there are many challenges to be solved in the future, such as expanding and optimizing the established computing power clusters, achieving 'backward compatibility', improving the automation capabilities of computing power scheduling, operation and maintenance, and fault location, and supporting the implementation of long-distance distributed computing power clusters. We will forge ahead in the direction of high-performance computing power networks, contribute to domestic computing power, and support the implementation of domestic large models. We believe that Jiliu will definitely have a place in the future hundred-thousand-card and million-card clusters, and we believe that Jiliu can enter the era of general artificial intelligence together with domestic leading large model manufacturers."

This article is from the WeChat official account "Xinglian Capital", author: Ma Wenpei, Entrepreneurship Nation. It is published by 36Kr with authorization.