The hundred-billion-parameter MoE video model is open-sourced, and video generation has begun to enter the next Scaling Law.

If we turn back the clock two years, few people would have predicted that video generation would become the fastest-evolving direction in this round of AI competition.

At that time, AI-generated videos were often only a few seconds long, with distorted characters, actions that defied physical laws, and chaotic shot transitions, not to mention generating complex content such as dialogues and ambient sounds. Waiting for dozens of minutes or even longer only to get a short video full of "AI flavor" was the most profound impression many people had of video models back then.

In just two years, this technology curve has risen sharply at a speed almost no one expected.

Today, models ranging from OpenAI's Sora and Google's Veo to Chinese models such as Keling and MiniMax Hailuo can all generate content of dozens of seconds or even several minutes with high consistency and high controllability. On international third-party rankings such as Artificial Analysis, Chinese video models have long been in the global first tier, jointly defining the development direction of this round of video generation technology with international manufacturers like Google.

However, while the industry is constantly refreshing the limits of image quality, duration and controllability, another challenge has also emerged.

The characters, actions, shots and scenes that video models need to process are becoming increasingly complex. After unifying audio and video, they also need to understand dialogues, ambient sounds and background music at the same time. As the scale of the model continues to expand, training costs, inference costs and cross-machine communication overhead also rise rapidly. The previous path of improving capabilities by continuously expanding the scale of Dense Transformer is gradually approaching its boundary.

Large language models have already gone through a similar problem once.

One of the solutions proposed by large language models is MoE (Mixture of Experts). It allows the model to have a larger total parameter scale, but only activates a small number of experts during each inference, thus partially decoupling the model capacity from the single calculation amount, and reopening the Scaling Law.

Then, when video generation also reaches the crossroads of model scale and computing cost, will it also enter its own MoE era?

Sand.ai's newly released and open-sourced hundred-billion-level MoE video generation model MAGI-2 Preview may be trying to answer this question. It has a total parameter of about 114B, but only activates about 6B parameters in a single forward calculation; more importantly, it is one of the few teams in the world that has truly applied the hundred-billion-level MoE to a unified audio and video generation model, and fully open-sourced the model weights, code and training system.

On the Image to Video list of the global video generation model ranking Artificial Analysis, MAGI-2 Preview ranks sixth in the world.

For a unified audio and video model with 114B parameters, what is really unexpected is that its cost does not grow linearly as the model expands. According to Sand.ai's calculation, the cost of generating 1080P video is about 0.5 yuan per 10 seconds.

For Sand.ai, this is another iteration of the MAGI series; for the entire industry, it may mean that video generation is beginning to try another scaling path.

Video Generation is Hitting the New Scaling Law

Since the beginning of 2024, video generation has been advancing rapidly along a clear development line.

From the initial rough attempts of "moving images" to today's minute-level high-definition generation, precise character consistency, and extremely complex shot scheduling; further, it is crossing the pure image rendering stage and evolving towards "unified audio and video generation" that integrates dialogues, ambient sounds and background music. Teams including OpenAI, Google, ByteDance, Kuaishou, MiniMax and other domestic teams are constantly refreshing the upper limit of this track.

Behind this round of explosion, the only iron law supporting the rapid development of the entire industry is to make the model larger.

Whether it is large language models, embodied intelligence or video models, the underlying belief of this round of AI wave has always been only one — the Scaling Law.

The expansion of parameter scale means that the model can describe more extensive physical laws and capture more subtle light and shadow and micro-expressions. The industry once firmly believed that as long as data and computing power are continuously invested, the scale effect will eventually be converted into amazing capability leaps.

However, in 2026, this path of forcing the expansion of Dense (dense model) scale to obtain capability improvement has hit the dual walls of physics and business.

The key crux is that video generation itself is the most computationally intensive task in the AI field, without exception. Compared with one-dimensional text, video requires high-intensity feature modeling in both two-dimensional space and one-dimensional time at the same time; and after unified audio and video generation becomes the new industry standard, multi-modal data including text, image and audio are integrated into the same model, and the amount of information contained in a single video increases exponentially.

Under the traditional Dense architecture, every time the model processes a Token, it needs to run hundreds of billions of parameters completely. When modalities are superimposed and the number of Tokens expands exponentially, the computing power cost, communication bandwidth overhead between GPUs and inference latency are rapidly getting out of control.

The industry has to face a cruel reality: the era of extending the Scaling Law by simply stacking model scales is reaching its peak. When the computing power budget is approaching the limit, how can the model continue to scale?

For this question, the adjacent large language model (LLM) track has already handed in the answer first. From Mixtral to DeepSeek and Qwen, the MoE (Mixture of Experts) architecture has gradually become the absolute mainstream. Its logic is elegant and intuitive: by building an extremely large total parameter library, but only accurately activating a small number of expert modules according to the task during each inference, it successfully physically decouples the model capacity from the single calculation cost, making the Scaling Law continue to be valid.

But why has MoE, which has achieved great success in the LLM field, been slow to be implemented in the video generation field?

The reason is that video generation cannot simply copy the routine of text models, and the biggest difference comes from Tokens.

The first is the dimensionality reduction attack of data magnitude. A few hundred words of text will only produce a few hundred Tokens after being discretized by the Tokenizer; while a high-definition audio and video of more than ten seconds, even after high-magnification compression, the number of discretized Tokens can easily surge to tens of thousands or even hundreds of thousands.

The second is the extremely complex spatiotemporal correlation. Text Tokens mainly process the semantic relationship between contexts, while video Tokens need to undertake the smoothing of spatial pixels, the continuity of the cross-frame time axis, and the millisecond-level alignment of audio and video synchronization at the same time. After the unified audio and video model appears, every Token also needs to keep synchronizing with information such as dialogues, lip shapes, and ambient sounds.

In other words, the video model not only processes more data, but also has more complex relationships that need to be modeled between each Token.

Such characteristics will bring engineering challenges to MoE. When a huge number of Tokens flood into the MoE system with an extremely complex cross-frame attention mechanism, a disaster will occur. Every Token needs to perform expert selection, routing and allocation in real time. If the hundred-billion-level MoE is disassembled onto hundreds or thousands of GPU cards, tens of thousands of Tokens frequently switch experts between different nodes, which will instantly burst the communication bandwidth between devices, and the routing overhead will even directly offset the computing savings brought by MoE.

In addition, there are issues such as cross-machine load balancing, extreme optimization of video memory fragments, expert collapse and gradient explosion that may occur at any time during training... For the hundred-billion-level video MoE, writing the algorithm formula of MoE may only take a few days, but to make a set of underlying system supporting hundred-billion-level MoE video training run stably, there was no ready-made reference answer in the whole industry before.

This also explains why for a long time in the past, although the whole industry coveted the cost dividend of MoE, the practical verification of hundred-billion-level MoE in the video generation field has always been blank.

Three Consecutive "Unconsensus" Choices: Sand.ai Has Been Answering the Same Question

If we regard Sand.ai's newly released MAGI-2 Preview as an independent update, it is easy to ignore its real significance.

In fact, this is already the third time that Sand.ai has continuously chosen a technical route that was not the mainstream of the industry at that time. Looking back, these choices seem to have occurred in model architecture, modal fusion and training method respectively, but in fact they are all answering the same question: how video models can continue to approximate the real world.

The earliest divergence appeared in the competition of video generation routes.

In fact, the original spark that made this wave of AIGC explode overnight and sweep the world was Diffusion. Therefore, for a long time in the past, Diffusion was almost the undisputed underlying paradigm in the field of video generation. Its core logic is to "reconstruct the picture from noise" — the model starts from a piece of chaotic random noise, and gradually generates high-definition and highly continuous video frames through dozens of rounds of extremely precise denoising prediction.

However, Sand.ai chose another path — Autoregressive — when it was founded at the end of 2024.

In Sand.ai's view, video is not a group of independent images, but a continuous sequence that unfolds continuously over time. If the model can predict the future frame by frame just like generating text, it will be easier to learn the causal relationship between actions and easier to extend to longer video generation. This judgment was finally implemented on Magi-1 released in April 2025, which also made Sand.ai one of the early teams that publicly verified the autoregressive video route.

Shortly after that, Sand.ai pushed the question one step further.

The real world never only has pictures. When a person speaks, lip shape, sound, action and environmental changes always occur synchronously. But for a long time in the past, most video models still generated pictures and sounds separately, and then spliced them through post-processing.

In September 2025, Sand.ai began to take unified audio and video as the next technical direction, and tried to put text, video and audio into the same Transformer through Gaga-1, so that different modalities continuously exchange information from the first layer of the model to jointly complete the generation process, instead of adopting the popular industry practice of adding sound after the picture is generated. The daVinci-MagiHuman released in March 2026 further verified this direction.

By the time of MAGI-2 Preview, the problem has changed again.

As the unified audio and video model continues to expand, the training and inference costs brought by Dense Transformer begin to grow rapidly. As mentioned earlier, continuing to expand the model scale means that every calculation needs to activate all parameters. The larger the model capacity, the higher the overhead of computing power, communication and inference will increase synchronously.

Therefore, Sand.ai placed the third technical judgment on the model scaling method.

MAGI-2 Preview adopts the Multi-Head MoE architecture, which splits a Token into multiple subspaces, and each subspace independently selects experts to participate in the calculation. This means that although the model has a total parameter of 114B, only about 6B parameters are actually activated in one forward calculation.

This also changes the way the model continues to scale.

How to understand this? Multi-Head MoE allows the model to have more internal experts, but only activates a small part of them during each calculation, which realizes a certain degree of decoupling between model capacity and single calculation amount. For the unified audio and video model, this means that the model can use more parameters to learn more complex temporal relationships, physical laws and multi-modal interactions, without making every inference bear all the calculation costs. This also gives video models a new path different from Dense Transformer when they continue to scale.

Looking back, Sand.ai's technological evolution in the past few years is actually continuously breaking through three boundaries of video models: time, modality and scale. Autoregression solves the problem of temporal continuity, unified audio and video solves the problem of multi-modal collaboration, and Multi-Head MoE solves the scale problem faced by the continuous scaling of the model.

However, for the hundred-billion-level Multi-Head MoE, only the model architecture is far from enough. The expert routing, cross-device communication and training stability mentioned earlier will all become new engineering bottlenecks as the model scale expands.

In order to solve these problems, Sand.ai has developed a complete set of training systems synchronously. The MagiMoE Kernel Library is responsible for expert routing, Token sorting and expert calculation, so that a large number of Tokens can find the corresponding experts and complete the calculation more efficiently; Head Parallel reorganizes the calculation mode between different Heads for the communication pressure brought by a large number of fine-grained experts, reducing frequent data exchange between GPUs; the distributed optimizer MagiMuon is responsible for ensuring the training stability under the conditions of hundred-billion-level parameters and massive experts, so that the super-large-scale MoE model can continue to converge without crashing halfway due to gradient oscillation or load imbalance.

The previous several technical judgments are further converging at this moment.

What Is Really Worth Paying Attention to Is Not Only the 114B Parameter

A very interesting change is taking place in the large model industry: models are getting stronger and stronger, and there are more and more papers, but what can really affect the direction of the industry is often not the self-definition of a certain company, but whether there is a set of public, repeatable and verifiable standards.

The world model is a typical example.

During the WAIC last year, almost every video model company and robot company was talking about world models, but different companies showed different lists: some demonstrated VBench, some emphasized WorldModelBench, and some showed Physics-IQ or robot task success rate. It was not until more and more public evaluation systems appeared that the industry gradually formed a common coordinate system. Whether a model is strong or not is no longer completely determined by the press conference, but is jointly defined by public data and community verification.

The same is true for technical routes.

In the past few years, Sand.ai has continuously chosen three routes that were not the mainstream of the industry at that time: autoregression, unified audio and video, and Multi-Head MoE. But no matter how complete the paper is and how outstanding the model effect is, these judgments essentially come from the technical choices of a single company.

The only way to truly make a route viable is to let more people verify it, and to get more people willing to move forward along it.

Llama promoted the open-source large language model ecosystem, Stable Diffusion gave birth to the AIGC image community, and DeepSeek made more and more teams rethink the technical routes of large model training and inference.

Sand.ai's open source of MAGI-2 Preview this time may also be striving for the same kind of influence.

Therefore, there are many key points in this MAGI-2 Preview: the parameter is one, MoE is one, and the synchronously developed engineering capability is also one, but there is another point that should not be ignored, which is open source. This time, Sand.ai has opened up the model weights, code and technical report. Multi-Head MoE has become a public technical route that the whole industry can verify, reproduce and continue to evolve.

Researchers can verify whether this fine-grained MoE is really more suitable for video generation; developers can continue to fine-tune, quantify and develop workflows around the model; inference framework teams can continue to optimize expert routing, video memory usage and communication efficiency; enterprises can also carry out private deployment according to their own data environment.

More importantly, these feedbacks from the community will in turn test the route itself; so how far this route can go in the end no longer depends only on Sand.ai, but on whether the entire developer ecosystem is willing to continue to evolve along it.

Can the