The hundred-billion-level MoE video model is open-sourced, and video generation has begun to enter the next Scaling Law.

If we turn the clock back two years, few people would have expected that video generation would become the fastest-evolving segment in this round of AI competition.

At that time, AI-generated videos were often only a few seconds long, with distorted characters, actions that defied physical laws, and chaotic shot cuts, not to mention generating complex content such as dialogues and ambient sounds. Waiting for dozens of minutes or even longer only to get a short clip full of "AI artifacts" was the most profound impression many people had of video models back then.

In just two years, this technology curve has shot upward at a speed almost no one anticipated.

Today, models ranging from OpenAI's Sora and Google's Veo to China's Kling and MiniMax Hailuo are all capable of generating content of dozens of seconds or even several minutes with high consistency and controllability. On leading international third-party rankings such as Artificial Analysis, Chinese video models have long occupied the global top tier, jointly defining the development direction of this round of video generation technology with international manufacturers including Google.

But while the industry is constantly pushing the boundaries of image quality, duration and controllability, another challenge has emerged.

Video models need to process increasingly complex characters, actions, shots and scenarios. After unifying audio and video, they also need to understand dialogues, ambient sounds and background music at the same time. As the model size continues to expand, training costs, inference costs and cross-machine communication overheads are also rising rapidly. The previous path of improving capabilities by continuously scaling up the size of Dense Transformer is approaching its limit.

Large language models have already gone through a similar problem once.

One of the solutions proposed by large language models is MoE (Mixture of Experts). It enables the model to have a larger total parameter size, but only activates a small number of experts during each inference, thus partially decoupling model capacity from single computation volume, and reopening the Scaling Law.

So when video generation also reaches the crossroads between model size and computational cost, will it also step into its own MoE era?

The release and open-sourcing of the hundred-billion-parameter MoE video generation model MAGI-2 Preview by Sand.ai this time may be an attempt to answer this question. It has a total of about 114B parameters, but only activates about 6B parameters in a single forward computation; more importantly, it is one of the very few teams in the world that has applied the hundred-billion-parameter MoE to a unified audio and video generation model, and fully open-sourced the model weights, code and training system.

MAGI-2 Preview ranks 6th globally on the Image to Video track of Artificial Analysis, the world's leading ranking for video generation models.

For a unified audio and video model with 114B parameters, what is truly surprising is that its cost does not grow linearly as the model expands. According to Sand.ai's calculation, the cost of generating 1080P video is about 0.5 yuan per 10 seconds.

For Sand.ai, this is another iteration of the MAGI series; for the entire industry, it may mean that video generation is starting to try another scaling path.

Video Generation is Hitting the New Scaling Law Boundary

Since the beginning of 2024, video generation has been advancing at a breakneck pace along a clear development line.

From the initial rough "GIF-like" attempts to today's minute-level high-definition generation, accurate character consistency, and extremely complex camera scheduling; further, it is moving beyond simple frame rendering to the "unified audio and video generation" that integrates dialogues, ambient sounds and background music. Teams including OpenAI, Google, ByteDance, Kuaishou and MiniMax are all constantly raising the upper limit of this track.

Behind this round of explosion, the only iron law supporting the rapid development of the entire industry is to build larger models.

No matter it is large language models, embodied intelligence or video models, the underlying belief of this round of AI wave has always been the Scaling Law.

The expansion of parameter size means that the model can describe more complex physical laws, and capture more subtle light, shadow and micro-expressions. The industry once firmly believed that as long as data and computing power are continuously invested, the scale effect will eventually be converted into amazing capability leaps.

However, in 2026, this path of forcibly expanding the size of the Dense (dense model) to improve capabilities has hit the dual walls of physics and business.

The key crux is that video generation itself is the most computationally intensive task in the AI field, without exception. Compared with one-dimensional text, video requires high-intensity feature modeling in both two-dimensional space and one-dimensional time; and when unified audio and video generation becomes the new industry standard, multi-modal content including text, image and audio is integrated into the same model, and the amount of information contained in a single video surges exponentially.

Under the traditional Dense architecture, every time the model processes a Token, it needs to run hundreds of billions of parameters completely. When modalities are stacked and the number of Tokens expands exponentially, computing power costs, communication bandwidth overhead between GPUs and inference latency are rapidly getting out of control.

The industry has to face a cruel reality: the era of continuing the Scaling Law by simply stacking model sizes is reaching its peak. When the computing power cost is approaching the bearing limit, how can the model continue to scale?

To this question, the adjacent large language model (LLM) track has already handed in its answer. From Mixtral to DeepSeek and Qwen, the MoE (Mixture of Experts) architecture has gradually become the absolute mainstream. Its logic is elegant and intuitive: by building an extremely large total parameter library, but only accurately activating a small number of expert modules according to the task during each inference, it successfully physically decouples model capacity from single computation cost, and keeps the Scaling Law valid.

But why has MoE, which has achieved great success in the LLM field, been slow to land in the video generation field?

Because video generation cannot simply copy the routine of text models, and the biggest difference lies in Tokens.

The first is the dimensionality reduction attack of data magnitude. A text of several hundred words will only generate several hundred Tokens after being discretized by the Tokenizer; while a high-definition audio and video of more than ten seconds, even after high-magnification compression, will easily generate tens of thousands or even hundreds of thousands of Tokens after discretization.

The second is the extremely complex spatiotemporal correlation. Text Tokens mainly process the semantic relationship of context, while video Tokens need to undertake the smoothing of spatial pixels, the continuity of cross-frame time axis, and the millisecond-level alignment of audio and video synchronization at the same time. When the unified audio and video model appears, every Token also needs to keep synchronizing with information such as dialogues, lip shapes and ambient sounds.

In other words, video models not only process more data, but also need to model more complex relationships between each Token.

Such characteristics will bring engineering challenges to MoE. When a huge number of Tokens flood into the MoE system with an extremely complex cross-frame attention mechanism, problems will arise. Every Token needs to perform expert selection, routing and allocation in real time. If the hundred-billion-parameter MoE is disassembled into thousands of GPU cards, tens of thousands of Tokens frequently switch experts between different nodes, which will instantly burst the communication bandwidth between devices, and the routing overhead will even directly offset the computing savings brought by MoE.

Coupled with cross-machine load balancing, extreme management of video memory fragments, expert collapse and gradient explosion that may occur at any time during training... For the hundred-billion-parameter video MoE, writing a MoE algorithm formula may only take a few days, but to make a set of underlying systems that support hundred-billion-parameter MoE video training run stably, there was no ready-made reference answer in the entire industry before.

This also explains why for a long time in the past, although the entire industry coveted the cost dividend of MoE, the actual verification of hundred-billion-parameter MoE in the video generation field has always been blank.

Three Consecutive "Unconventional Choices", Sand.ai Has Been Answering the Same Question

If we regard the MAGI-2 Preview released by Sand.ai this time as an independent update, it is easy to ignore its real significance.

In fact, this is the third consecutive time that Sand.ai has chosen a technical route that was not the mainstream of the industry at that time. Looking back, these choices seem to have occurred in model architecture, modal fusion and training mode respectively, but in fact they are all answering the same question: how video models can continue to approximate the real world.

The earliest divergence appeared in the competition of video generation routes.

In fact, the original spark that made this wave of AIGC explode and sweep the world is Diffusion. Therefore, for a long time in the past, Diffusion was almost the undisputed underlying paradigm in the video generation field. Its core logic is "reconstructing frames from noise" — the model starts from chaotic random noise, and gradually generates high-definition, high-continuity video frames through dozens of rounds of extremely precise denoising prediction.

However, at the beginning of its establishment at the end of 2024, Sand.ai chose another path — Autoregressive.

In Sand.ai's view, video is not a set of independent images, but a continuous sequence that unfolds along time. If the model can predict the future frame by frame just like generating text, it will be easier to learn the causal relationship between actions, and easier to extend to longer video generation. This judgment was finally implemented in Magi-1 released in April 2025, making Sand.ai one of the earliest teams to publicly verify the autoregressive video route.

Shortly after, Sand.ai pushed the question one step further.

The real world is never only made of images. When a character speaks, lip shape, sound, action and environmental changes happen synchronously. But for a long time in the past, most video models still generated images and sounds separately, and then spliced them through post-processing.

In September 2025, Sand.ai began to take unified audio and video as the next technical direction, trying to put text, video and audio into the same Transformer through Gaga-1, so that different modalities continuously exchange information from the first layer of the model and complete the generation process together, instead of adopting the popular industry practice of adding sound after the image is generated. The daVinci-MagiHuman released in March 2026 further verified this direction.

When it comes to MAGI-2 Preview, the question has changed again.

As the unified audio and video model continues to expand, the training and inference costs brought by Dense Transformer begin to grow rapidly. As mentioned earlier, continuing to expand the model size means that every computation needs to activate all parameters. The larger the model capacity, the higher the overhead of computing power, communication and inference.

Therefore, Sand.ai placed its third technical judgment on the model scaling method.

MAGI-2 Preview adopts the Multi-Head MoE architecture, which splits a Token into multiple subspaces, and each subspace independently selects experts to participate in the computation. This means that although the model has a total of 114B parameters, only about 6B parameters are actually activated in one forward computation.

This also changes the way the model continues to expand.

How to understand this? Multi-Head MoE allows the model to have more internal experts, but only activates a small part of them in each computation, realizing a certain degree of decoupling between model capacity and single computation volume. For the unified audio and video model, this means that the model can use more parameters to learn more complex temporal relationships, physical laws and multi-modal interactions, without making every inference bear the full computational cost. This also gives video models a new path different from Dense Transformer when they continue to scale.

Looking back, Sand.ai's technical evolution in the past few years has actually been constantly breaking through three boundaries of video models: time, modality and scale. Autoregression solves the problem of temporal continuity, unified audio and video solves the problem of multi-modal collaboration, and Multi-Head MoE solves the scale problem faced by the continuous expansion of models.

However, for the hundred-billion-parameter Multi-Head MoE, the model architecture alone is far from enough. The expert routing, cross-device communication and training stability mentioned earlier will all become new engineering bottlenecks as the model size expands.

In order to solve these problems, Sand.ai has developed a complete set of training systems simultaneously. The MagiMoE Kernel Library is responsible for expert routing, Token sorting and expert computation, so that a huge number of Tokens can find the corresponding experts and complete the computation more efficiently; Head Parallel reorganizes the computation mode between different Heads to reduce frequent data exchange between GPUs, targeting the communication pressure brought by a large number of fine-grained experts; the distributed optimizer MagiMuon is responsible for ensuring training stability under the condition of hundred-billion-parameter and massive experts, so that the ultra-large-scale MoE model can keep converging without crashing halfway due to gradient oscillation or load imbalance.

The previous several technical judgments are further converging at this moment.

What Is Really Worth Noticing Is Not Only 114B

A very interesting change is taking place in the large model industry: models are getting stronger and papers are becoming more and more numerous, but what can really influence the industry direction is often not the self-definition of a certain company, but whether there is a set of public, repeatable and verifiable standards.

The world model is a typical example.

During the WAIC last year, almost every video model company and robot company was talking about world models, but different companies showed different rankings: some demonstrated VBench, some emphasized WorldModelBench, and others presented Physics-IQ or robot task success rate. It was not until more and more public evaluation systems emerged that the industry gradually formed a common coordinate system. Whether a model is really strong is no longer completely determined by the press conference, but jointly defined by public data and community verification.

The same goes for technical routes.

In the past few years, Sand.ai has consecutively chosen three routes that were not the mainstream of the industry at that time: autoregression, unified audio and video, and Multi-Head MoE. But no matter how complete the paper is and how prominent the model effect is, these judgments essentially come from the technical choice of one company.

The only way to make a route full of vitality is to let more people verify it, and to attract more people to move forward along it.

Llama promoted the open source large language model ecosystem, Stable Diffusion gave birth to the AIGC image community, and DeepSeek made more and more teams rethink the technical route of large model training and inference.

The open sourcing of MAGI-2 Preview by Sand.ai this time may also be striving for the same kind of influence.

Therefore, there are many key points in MAGI-2 Preview this time: the parameter count is one, MoE is another, and the synchronously developed engineering capability is also one. But there is another point that should not be ignored, which is open source. This time, Sand.ai has opened the model weights, code and technical report. Multi-Head MoE has become a public technical route that the entire industry can verify, reproduce and continue to evolve.

Researchers can verify whether this fine-grained MoE is really more suitable for video generation; developers can continue to perform fine-tuning, quantization and workflow development around the model; the inference framework team can continue to optimize expert routing, video memory occupation and communication efficiency; enterprises can also carry out private deployment according to their own data environment.

More importantly, these feedbacks from the community will in turn test the route itself; so how far this route can go in the end no longer depends only on Sand.ai, but on whether the entire developer ecosystem is willing to continue to evolve along it.

Can the model really continue to scale? Is the engineering complexity low enough? Can the training cost be further reduced? These questions no longer need to be answered by Sand.ai alone, and the entire developer community will jointly give the answers