From NVIDIA to Pika, why are Silicon Valley giants "seeking reverse leverage" from this domestically developed model?

Few people would be surprised that a Chinese AI company has trained its own large video model using NVIDIA GPUs and referring to the technical roadmap of Silicon Valley.

But what if the opposite happens?

The research team of NVIDIA, the world's leading GPU giant, has rarely selected an open-source multimodal model from a Chinese company, investing manpower and time to develop a video generation model. That makes things quite interesting.

From Startups to NVIDIA: Why Was H3 Selected?

In August this year, MiniMax opened the weights of its multimodal model H3. This model can uniformly understand multiple types of materials including text, images, audio and video, while balancing generation quality, speed and cost. From the perspective of the technology ecosystem, opening model weights enables external teams to directly participate in post-training, distillation and inference optimization, expanding the ways they can participate in model R&D. Within one month of open-sourcing, H3 was downloaded more than 24 million times, with over 300 public derivative models developed based on it.

The first to be attracted were video generation companies. Most of their customers use video generation models to make advertising and marketing materials, and they care about generation speed, as well as the consistency of products and characters. No matter how gorgeous the technical demonstration is, in the actual commercial production scenario, customers often care about more practical issues: Will the text on the product packaging be deformed? Can the same character remain consistent after switching shots? When modifying a section of advertisement, is it necessary to regenerate the entire video? If thousands of short videos need to be produced in batches, can the time and cost be afforded? These issues determine whether AI video is just an amazing technology, or a real business that can operate on a large scale.

The industry is highly competitive, and these video generation companies are motivated to adjust the model more targeted according to customer needs.

Fal, a generative media infrastructure company, started from these demands, trained H3 Max based on MiniMax H3, and optimized the inference system around the model, aiming for faster generation speed. Video generation company Pika wanted to better meet the needs of professional creators on its platform, and on October 29, it released H3 Flex which was post-trained based on H3, compressing the generation steps.

Somewhat surprisingly, unlike these companies in vertical industries, NVIDIA is re-developing a video generation model based on H3. In late October, the relevant NVIDIA research team released Open-H3-Regenerate-2K based on H3: it first generates low-resolution drafts with H3, then regenerates high-resolution videos through an additionally trained refinement module and three-step denoising. The project retains the original weights of H3.

NVIDIA rarely invests so many resources in multimodal models, but it has carried out systematic development on H3.

Previously, it launched the StreamRender-H3 streaming video rendering solution and fast video decoder, exploring interactive generation and decoding acceleration respectively; the inference optimization project improved the generation process while keeping the H3 model weights, prompts and generation configuration unchanged.

Video generation companies and global giants like NVIDIA have different application scenarios for models, but they both set very high standards for selecting base models: the selected model can not only handle complex multimodal input and video generation tasks, but also open weights, allowing researchers to run the model locally, add refinement modules or modify the inference process. Even if a model is fully open, it is difficult to become the basis for further R&D if its generation quality is not good enough, its architecture is difficult to expand, or it cannot adapt to new training tasks.

There are not many multimodal models that meet this standard, and the performance and scalability of H3 make it selected repeatedly. In September this year, H3 ranked first in the Arena image-to-video list with a score of 1495, receiving more than 57,000 evaluation votes in total.

But NVIDIA's goal may go further than that. It is good at and dares to invest in and cultivate the ecosystem in the early stage of new technologies. One example is CUDA: through resource tilting and commercial means, it allows developers to build applications around NVIDIA GPUs, forming a software ecosystem with high barriers.

The multimodal model field may be the next CUDA-like opportunity: by doing inference acceleration and generation applications around H3, more video models and developers will be brought into the NVIDIA technology stack. NVIDIA is optimistic not only about the video generation market, but also about the ecosystem and computing power demand behind it.

In the Open-Source Ecosystem, Partners Are Also "Nodes" to Enter Larger Markets

When leading enterprises integrate video models into their production processes, they face different requirements, and the same enterprise will put forward new demands as its business changes. If model manufacturers meet these requirements one by one and continue to maintain and update, their costs will keep increasing.

According to a survey by video generation platform Runway, some enterprise customers hope to directly own the model weights, deploy the model in their own environment, and then fine-tune it with their own data. Interviews conducted by venture capital firm a16z with enterprise AI heads also mentioned that control and customization for specific businesses are important reasons for enterprises to choose open models. It also pointed out in the report that enterprises are paying more and more attention to open models, not only because of the price, but also because they can fine-tune according to their own data.

Models with open weights provide a new solution. Open weights enable leading enterprises to put models into their own production systems. They can adjust the models around existing customer needs, and then deliver capabilities through their own platforms, products and services.

This points to a vast market that far exceeds the service capacity of a single company. These partners are connected to customers, platforms and workflows respectively. For model and product manufacturers such as MiniMax, partners may become nodes to enter more industries. When H3 enters more customers and scenarios along with partners' products, MiniMax can also continuously expand its own industry applications, allowing more technical teams familiar with vertical scenarios to participate, carry out targeted optimization on the basis of H3's existing capabilities, and jointly promote the implementation of video generation technology in different fields.

This is also a unique commercialization method in the open-source ecosystem. MiniMax can expand its revenue sources by means of authorization, deployment support or joint development with the help of these important companies that act as "nodes".

This is a business path that has been verified by many industries. Horizon Robotics, an autonomous driving chip and algorithm company, opened BPU chip IP and perception algorithm authorization, and also provided chips, toolchains, solutions and joint development. Car companies and partners can build differentiated products accordingly, which not only expanded the company's "circle of friends" and influence, but also directly supported its commercialization.

According to the "MiniMax H3 Community License Agreement", the agreement grants a royalty-free limited license within the applicable territory, allowing users to use, modify the model and develop derivative models in accordance with the terms; if the annual revenue of commercial products or services using H3 exceeds 20 million US dollars, prior written authorization from MiniMax is required.

As the ecosystem becomes more active, the commercialization of MiniMax H3 is also developing rapidly.

When video generation changes from an occasional creative tool to a new link in the enterprise production chain, directly related industries such as advertising, e-commerce, film and television, games and education, as well as broader fields, will give birth to new tools and services respectively. Open weights make H3 the base that these teams jointly modify, and also give MiniMax the opportunity to enter more industries along with partners' products.

This echoes the mission that MiniMax has always emphasized: "Intelligence with Everyone". The development of AI is not only about a few model companies continuously launching stronger versions, more developers and more companies will also be nodes in this wave of technological innovation, and everyone is a user of new technologies as well as a promoter of the wave.

For a commercial company, what is more important is that the market may reassess the commercial value of an open-source model: in addition to API revenue, when a model can become the base for enterprise customization, deployment and product development, enter the workflows of more industries, and drive revenue from authorization, technical services and joint development, its commercial potential may be greater than imagined.

This article is from the WeChat Official Account "AI Front" (ID: ai-front), written by Dongmei, and published with authorization from 36Kr.