The graphics cards are not fully utilized yet, but the data centers are already bottlenecked by the "network cables".

Really? Nowadays, can the Micro LED technology used in TVs be made into network cables for data transmission?

Well, recently, Tony came across a news item online stating that Microsoft is researching the use of Micro LED optical interconnection technology (MOSAIC) to solve the bottleneck problem of data transmission in computing power centers.

Well, although it sounds quite abstract, the computing power centers responsible for data processing are actually held back by data transmission because there aren't enough network cables in the computing power centers...

You might be curious. When I search for a Category 6A 10 Gigabit (10Gbps) network cable on JD.com, it only costs around ten yuan. How can there not be enough?

But in fact, the connection speed of the network cables at our homes is several orders of magnitude lower than those used in data centers.

The maximum speed that the network cables at our homes can usually support is 1000 Mbps - 2500 Mbps, which is 1 Gbps - 2.5 Gbps. In recent years, some regions have been promoting the "10 Gigabit network" and have provided 10 Gbps network cables, but this is almost the upper limit of the speed for civilian network cables.

However, the port switching speed in data centers has long reached 100 Gbps as the mainstream, and the switches in AI computing power centers need to start at 400 Gbps.

Actually, it's because AI large models are getting bigger and bigger. During training and inference, a large amount of data needs to be exchanged between servers and between GPUs, which increases the demand for bandwidth.

To transmit such a large amount of data, the copper cables and optical fibers currently used in data centers are really not enough...

Let's start with copper cables. The characteristic of this material is that it can only prioritize either transmission speed or effective distance. To achieve the high speed required by data centers, the length of high - speed copper cables is usually only 1 - 2 meters. That's why copper cables are often used to connect GPUs inside a cabinet.

As the transmission speed increases, the effective distance of copper cables gradually shortens (a), and the power consumption of optical fibers gradually increases (b)

Optical fibers, on the other hand, can transmit data quickly and over long distances and can connect across cabinets. However, they involve complex "optical - electrical conversion". The related circuits consume a lot of power, are sensitive to temperature, and are prone to aging. In the extremely high - temperature environment of a computer room, they are very prone to failures.

Microsoft's paper mentions that if all optical fiber interconnections are used, the power consumption of NVIDIA's GB200 NVL72 cabinet will increase by 17%. In a very large - scale GPU cluster, a link failure will occur every 6 - 12 hours...

Therefore, after comprehensive consideration, NVIDIA's GB200 NVL72 finally adopted a copper cable connection scheme. However, as a result, the 72 GPUs inside can only be stuffed into a single rack, which puts a great deal of pressure on the power supply and heat dissipation of the entire cabinet.

Moreover, maintenance is also very troublesome. After all, with such a high level of integration, once something goes wrong with a GPU or any part of the transmission link, the operation of the entire cabinet will be affected when trying to repair it.

From this example, you can also see that traditional copper cable and optical fiber communication can no longer meet the requirements of data centers for high - bandwidth, low - power, and long - distance connections at the same time.

The emergence of MicroLED optical communication is to solve this problem.

The MOSAIC proposed by Microsoft essentially uses MicroLED pixels as light sources. You can imagine the glowing MicroLED pixel array as stacked displays.

Since MicroLED pixels can emit light independently, each pixel is an optical channel for data transmission.

So the transmitting end controls the on and off of the pixels, where on represents 1 and off represents 0. The receiving end then records the brightness changes of each pixel, restores the received long string of 0s and 1s into the original data, and can transmit information through optical signals.

It sounds similar to the communication principle of traditional optical fibers, but different from the "narrow - bandwidth, high - speed" of optical fibers, the transmission mode of MOSAIC is "wide and slow".

Let's first talk about the "slow" part.

MOSAIC stipulates that there is no need to push the single - channel speed to an ultra - high speed of 50 Gb/s or even 100 Gb/s like traditional optical fiber communication. Each MicroLED pixel can operate at a "low speed" of 2 Gb/s.

The ability to achieve high - speed transmission with such a slow speed relies on another characteristic of MOSAIC, "wide".

In the past, to achieve a bandwidth of 800 Gbps, 8 high - speed channels of 100 Gbps were required. Although the speed of a single channel in MOSAIC is reduced to 2 Gbps, by making the MicroLED array with 400 pixels, a bandwidth of 800 Gbps can be achieved.

However, don't think that the volume and power consumption of the MicroLED optical communication module will also get out of control.

The reason why MOSAIC dares to use scale to exchange for speed is, on the one hand, that MicroLED pixels themselves are only a few micrometers to dozens of micrometers. Even if an array of 400 pixels is made, the volume of the core light - emitting chip is less than 1 mm³. For traditional 800 Gbps - level optical modules, the volume of the core light source/modulator can reach several tens of mm² or even several tens of mm³. It's like comparing a millet grain with a rice grain on a table.

Micro LEDs have smaller pixel sizes and are more densely arranged at the same pixel pitch

Even if the connection speed of MicroLEDs is expanded to 1.6 Tbps or even 3.2 Tbps, which is 4 times the current mainstream transmission speed in data centers, the volume of the entire MicroLED optical module will not be larger than that of traditional optical fiber modules...

On the other hand, the transmission cables of MOSAIC are also quite advanced. They directly introduce the "multi - core imaging fiber" used in medical endoscopes into the computer room. Simply put, the increase in MicroLED optical channels will not make the cables take up more space.

Because the inside of this kind of optical fiber contains thousands of tiny cores, which can fully meet the connection requirements of hundreds of MicroLED optical channels.

MicroLED multi - fiber imaging fiber can carry hundreds of optical channels

Taking the 800 Gbps bandwidth as an example, the traditional optical fiber scheme needs to package 16 single - mode fibers (8 for transmission + 8 for reception) together. The "multi - core imaging fiber" can achieve high - bandwidth transmission with just one cable.

Moreover, this kind of multi - core imaging fiber can achieve an effective transmission distance of 50 meters, which far exceeds the limit of copper cable connections.

In addition, since the structure of MicroLED is simple, the current switches for controlling the on and off of pixels can also be made simple, eliminating many high - power - consumption circuits in traditional optical modules.

According to Microsoft's data, to achieve the same bandwidth, the power consumption of MOSAIC can be reduced by up to 68% compared with traditional optical fiber interconnection, and the failure rate can be reduced to 1/100 of the original...

It can be said that with this technology, when building cabinets or servers in the future, there is no need to struggle between "bulky copper cables" and "high - power - consumption optical fiber modules". Instead, there is a third option that strikes a better balance between power consumption, distance, and bandwidth.

Common connection schemes in data centers: optical fibers are used to connect switches, and copper cables are used inside cabinets

Currently, MicroLED optical communication is still in the technology verification stage. Manufacturers such as TSMC, Avicena, and Zhaochi are still working on prototypes and industrial layout, and large - scale commercialization has not been realized yet.

However, Tony is quite optimistic about the development prospects of this technology. After all, MicroLED optical communication can truly reduce power consumption, and the power shortage in foreign countries has always been a big problem.

More importantly, as mentioned at the beginning, the computing power centers are increasingly being held back by communication efficiency. Or from another perspective, a revolution in communication efficiency can make up for the disadvantages in computing power.

For example, in Huawei's 384 super - node, the performance of a single Ascend AI processor is not very strong. However, by connecting 384 NPUs in series to form a computing power cluster, the performance of the entire machine can be comparable to NVIDIA's GB200 NVL72.

Then I think whether we can "overtake on a curve" with a new optical communication protocol, making data transmission faster, more power - saving, and more reliable, may be the second half of the AI competition and the computing power "game".

This article is from the WeChat official account "ChaPingX.PIN". Author: Levi, Editors: GuoTiao & MiLuo & MianXian. Republished by 36Kr with permission.