There is a time gap of native multimodality between Kimi K3 and DeepSeek V4.

Text | Li Zhaofeng

Edited by Zhang Yuxin

"If long-chain tasks only rely on feedback from the code level, errors may accumulate continuously, and the final result will be very poor." Talking about the significance of native multimodality, a multimodal researcher said, "Vision is a more accurate form of feedback, and it is also closer to user intent."

Over the past year, the capabilities of Coding and Agent have been constantly rewriting the rankings of large models, and have become one of the scenarios where AI realizes commercial value the fastest. At the same time, as Agents begin to take over more long-chain tasks, more and more general large models are beginning to be equipped with native multimodal capabilities.

In January this year, an enterprise usage report released by OpenAI showed that among the three most commonly used ChatGPT tools in R&D positions, image uploading ranked third after search and data analysis.

As Agents begin to generate web pages, operate software and check running results, the role of vision is gradually evolving: it is no longer just an image that users submit to the model, but has also become feedback for the model to check work, find errors and adjust actions.

In the just-passed July, Moonshot AI released Kimi K3. This MoE (Mixture of Experts) model with a total parameter of 2.8 trillion and supporting 1 million token context, besides its outstanding Coding and long-horizon Agent capabilities, excellent native multimodal capability is another label of K3.

The so-called native multimodality means that data such as images and texts jointly shape the main model from the pre-training or continuous pre-training stage, and continue to be optimized in post-training, and finally participate in the perception and decision-making of the Agent.

After its release, K3 topped the Arena Frontend Code list with a score of 1679. This list is ranked by users who blindly select interactive web pages generated by models, and the evaluation covers not only whether the code can run, but also the final effect of the page.

The browser development platform Puter once created 5 visual deviations in the test web page. K3 compared the screenshots of the target page and the running page, found all of them without false positives. Thanks to its excellent native multimodal capability, K3 can understand the visual problems after the code runs, providing a basis for the next round of modification. Kimi calls this iterative method between code and screenshots "vision in the loop": after the model finishes writing the code, it checks the page, and then continues to adjust according to the visual result.

None of these can be achieved without the multimodal capability of the model.

In addition to Moonshot AI, Alibaba and ByteDance are also following the native multimodal route, taking vision as the basic capability of general models. On the latest Qwen3.8-Max and Doubao-Seed-2.1, visual understanding and multimodal input are presented as core functions.

On the other side of the top tier of domestic models, the latest general base models of DeepSeek, Zhipu AI and Tencent Hunyuan still mainly support text input.

Previously, Liang Wenfeng, founder of DeepSeek, once said: "To do AI training well, you don't need a world model, or even multimodality." But he also said: "Multimodality will eventually be implemented."

These two sentences are not contradictory. There is no big difference among companies on the long-term value of multimodality, and the real divergence lies in the timing and cost.

In the stage of rapid Coding iteration, whether to train vision synchronously and how much model capacity, data and computing power to invest for it are affecting the technical routes of leading domestic models.

 Why Agents Need Eyes

Is it really necessary to develop native multimodality? As the task chain of current Agents gets longer and longer, this question has become more and more specific.

"Most of the time, I prefer to input screenshots." A researcher from a leading base model team said, "Maybe text can also meet the same requirement, but you have to spend a lot of context describing the visual relationship."

Visual input can bring more convenience, but different users have different perceptions of it. "Ordinary people may not feel it, but for developer groups, having multimodality is very convenient."

However, this demand is not limited to developers. The researcher mentioned that some users around him who are not AI practitioners use the model most frequently to make PPTs. "More professional people may need more capabilities and more modalities, but ordinary users will also use it in specific scenarios."

The pure text model itself is still "blind". The actual system can call OCR or independent VLM (Vision Language Model) to first convert the image into text, labels, coordinates or structured fields, and then submit them to the text model for reasoning. These tools can be accessed through the Agent framework or MCP to obtain visual capabilities in the form of "plug-ins".

From this perspective, the proposition of "how to enable Agents to obtain visual input capability" is not only a base model R&D issue, but also a product issue.

If the system is tuned well enough, a large upstream language model acts as the hub, and a smaller visual model is placed downstream, just like a boss assigning a small task to an employee, which is enough to solve a large number of common tasks.

But in the view of multimodal researchers, this modular solution still has limitations. "If you call a tool, there are always two models in total: one LLM is 'blind', and the downstream is equipped with a subordinate who can see clearly." He explained that the communication channel between the visual input inside the native multimodal model and the language backbone is "wider", instead of first compressing the picture into text and then submitting it to another model for judgment.

This difference will also extend to post-training. "If the model can do it by itself, it is definitely a better thing." Chen Yu said that the native multimodal model can re-read the pages or images it generates in visual reinforcement learning, and incorporate the visual results into the same training trajectory; if relying on external tools, the perception results need to be transmitted between the two models.

In long-chain tasks that require repeated viewing of the screen, this difference will be more obvious. The external visual tool needs to first convert the picture into text and then submit it to the main model for judgment; the native multimodal model can directly understand the page changes and then directly decide what to do next.

In the communication, a researcher recalled to the media Intelligence Emergence the "shocking" experience when he first let GPT-5.6 Sol operate the PC-side Agent. GPT-5.6 Sol can not only automatically open the browser, but also handle authentication and permissions, push local code to the platform, and even log in to the training platform directly and start tasks.

Willison commented that the model "knows a lot of tricks", and will almost use all available methods to achieve the goal.

"Without native multimodality, the model has no eyes, and the feedback is only limited to text." The above multimodal researcher said, "But many user-oriented outputs are visual, such as web pages, images and videos. The model needs to understand these results and give feedback to itself."

In fact, the industry has never reached a consensus on whether vision is only a tool for models to complete tasks, or an indispensable part of understanding the world.

Ilya Sutskever, co-founder and former chief scientist of OpenAI, once called text "a projection of the world": humans have compressed a large number of real-world laws into language, and models that continue to learn text can still gain an understanding of the world.

Yann LeCun, Turing Award winner and chief AI scientist of Meta, holds almost the opposite judgment: "The vast majority of human knowledge is not expressed in text." Language is only information that has been filtered and summarized by humans; to understand objects, spaces and actions, models ultimately need to learn from images, videos and real-world interactions.

Text is highly summarized information, while images and videos are more like raw signals. Raw signals contain the world that text has not recorded, and more data, computing power and patience are needed to refine them into generalizable knowledge.

"Multimodality is definitely not decisive, but it is very important." A model researcher said, "Basic image and video question answering tasks have been developed quite maturely. The next step is to integrate multimodality into production-oriented actual scenarios and complex tasks."

On the other hand, as the Agent ecosystem is fully popularized among mass users, users' expectations for flagship models are also changing.

At the end of June, Tang Jie, chief scientist of Zhipu AI, solicited users on Platform X about "what new functions must be added to the next version of GLM", and the answer that repeatedly appeared in the comment area was "vision".

Previously, many industry insiders told Intelligence Emergence that the next-generation general base models of Zhipu AI and Tencent Hunyuan may further move towards native multimodality.

The trend has emerged, but whether to fully strengthen the native multimodal route, base model teams are still facing the problem of resource allocation.

The Cost of Models Growing Eyes

"For LLMs (Large Language Models), multimodality is more like an accessory." A researcher from a leading base model manufacturer said, "The core of model development is not multimodality, but the ability to complete tasks."

From the perspective of task completion, the fact that a model can see more does not mean that its working ability is necessarily stronger. A more realistic problem is: after adding multimodal capabilities to an LLM, it may even weaken its original language, reasoning and Coding capabilities.

"The more tasks you have, the harder it is to balance." A multimodal researcher said, "If you only do one task, you can keep adjusting in one direction; if you do two tasks at the same time (vision + text), you either make the model larger, or add more data."

Enabling the model with vision is not simply adding an input interface. The data structure, information density and learning speed of images and texts are different. When they jointly train the same set of backbone parameters, visual data will compete with text, code, mathematics and reasoning for model capacity, and different training targets may also interfere with each other.

The Kimi K2.5 technical report recorded this effect in an ablation experiment on fusion timing: if visual data is added in the middle and late stages of training, the text capability of the model will first decrease and then gradually recover.

"If you want to build a unified native multimodal model, the resources you pay must be 1+1 greater than 2." The above researcher said, "You can't get the same effect just by adding up the training volume and parameters of two separate models."

In addition to the difficulty of training, native multimodality, while expanding usage scenarios, also means more computing power consumption.

The technical report of Apple's MM1 (Multimodal Large Model) found that the visual encoder, image resolution and the number of visual tokens will all affect the model effect; higher resolution and more visual tokens usually bring higher training and inference overhead.

For tasks that only need to read a few fields, letting the general model repeatedly view the entire image may not be more cost-effective than OCR (Optical Character Recognition) or professional models.

K3 is the most intuitive sample at present. This "all-round model" that pursues vision, Coding and Agent capabilities at the same time has a total parameter of 2.8 trillion, with about 104 billion parameters activated per token.

From the perspective of training methods, K3 continues the early fusion route similar to K2.5. Experiments on K2.5 show that when the total number of image-text tokens is fixed, introducing vision at an earlier stage and at a lower ratio is usually better than accessing it in the later stage of training. Its final solution always mixes text and visual data at a fixed ratio in the joint pre-training of about 15 trillion mixed image-text tokens.

K3 also trained the visual encoder MoonViT-V2 with about 400 million parameters from scratch, no longer relying on the initialization weights of SigLIP (Sigmoid Loss for Language Image Pre-Training), and let the visual representation directly enter the next token prediction target. The technical report states that this approach improves training stability while maintaining visual effects.

"Every time multimodal data increases a little, text understanding, Coding or mathematics may get worse, and you have to keep adjusting the ratio later. It tests the version integration capability of the base model team." A researcher summarized in this way.

Weeks after the release of K3, DeepSeek came up with another answer. The official version of DeepSeek V4-Flash has a total parameter of 284 billion, with 13 billion parameters activated per token, which are about one-tenth and one-eighth of K3 respectively. It did not adjust the architecture and parameter scale, but focused the updates on post-training.

In a number of Coding and Agent evaluations released by DeepSeek, the official version greatly outperforms the preview version, and even surpasses the previous V4-Pro preview version. On the Arena WebDev list that examines web development and multi-round tool operation, its public score once rose to 1577 points, close to GLM-5.2, and only a few models such as K3, Claude Fable 5 and GPT-5.6 Sol are ahead of it.

DeepSeek V4-Flash proves that without adding vision and significantly expanding the parameter scale, post-training can still significantly improve the core task capabilities such as Coding.

When this route is still generating returns rapidly, there is no unified answer yet to how many resources base model companies should allocate to vision in advance.

At the same time, the newly released Qwen3.8-Max from Alibaba has chosen a similar "large and comprehensive" direction as K3: with a total parameter of 2.4 trillion, 95 billion parameters activated per token, and it supports native vision, Coding and Cowork capabilities at the same time.

It is not difficult to see that the technical route choices of base model manufacturers are not entirely determined by technical conclusions. In this regard, industry insiders said: "The size of the model and the selling point each company chooses are ultimately decided by a few people, or the boss."

The research background of core members, product scenarios and training costs will all affect the priority of a company.

The talent market is also sensing the changes taking place in the industry. Intelligence Emergence learned that after the release of K3, there are more headhunters in the cafes and restaurants near the office of Moonshot AI, and "multimodality" has become the key word for their visits.

But this does not mean that all companies will immediately follow the route of super-large parameters + native multimodality, nor does it mean that multimodality will replace Coding as the center of competition.

Many industry insiders believe that Coding is the threshold for entering the game. If the Coding capability cannot be improved, there may be no future opportunities.

For a large number of domestic base model manufacturers, this does not stem from the divergence of the long-term route of AGI, but more from different considerations of development timing and cost. At least for now, the pressure of ranking, product and commercialization still falls more on Coding and Agent capabilities.

This competition is like turning two clocks at the same time: the rankings of Coding and Agents change every few months or even weeks, while the evolution of models from language to the world requires a much longer cycle.

The former clock determines who can keep up with the current speed, and the latter clock may determine where these companies can finally reach.