Xu Zhuhong, Head of Qwen Intelligence: The competition in the Agent smartphone sector is not only about models.
Over the past two years, many mobile phone manufacturers have integrated large models into their smartphones.
"The smartphone is the smart terminal closest to the Personal Agent" has almost become the industry consensus at the current stage: the penetration rate of smartphones is close to 100%, they follow users at all times, connect communication, payment, travel, imaging and various life services, occupy 90% of the usage time of most people, and naturally precipitate the most complete personal context of a user.
But most of the time, smartphones equipped with large models still only provide users with a smarter "mobile assistant": they can answer questions, summarize documents, and edit images, but it is difficult to truly complete a complex task for users.
Nowadays, the emergence of Agent is changing this situation.
Agent enables smartphones to be equipped with task planning, tool invocation and long-distance execution capabilities — users no longer need to tell the phone every step to do, they only need to give the target, and the rest of the problems will be understood and disassembled by the model, which will invoke system capabilities, applications and services to complete the task.
This also makes the focus of competition in AI smartphones begin to shift — what determines the user experience is no longer just whether the model is smart enough, but the entire set of systems outside the model: how Context is organized, how Memory is invoked, how tools and Skills are arranged, how GUI operates, and how to recover after task failure, etc.
Qwen Intelligence, the mobile smart business released by Alibaba at this year's Yunqi Conference, is exactly such a system.
Alibaba does not produce smartphones by itself, and Qwen Intelligence is not just a model or an Agent, but a full-stack AI smartphone solution for mobile phone manufacturers. Based on the Qwen large model, it provides terminal manufacturers with a complete set of "intelligent layers" from underlying models, Agent platforms to scenario solutions. It covers capabilities from user intention understanding, complex task planning, to tool invocation, cross-application execution, multimodal creation, memory and active services, aiming to help smartphones move further from "being able to understand users" to "being able to complete tasks for users".
Qwen Intelligence has released three Agent solutions this time: Mobile Planner Agent, Mobile Use Agent and Creative Agent, corresponding to "planning, operation and creation" respectively. Among them, Mobile Planner Agent is responsible for understanding complex requirements, disassembling tasks and arranging tools; Mobile Use Agent is responsible for actually operating the mobile phone, adopting a hybrid mode of "API first, GUI as fallback"; Creative Agent is oriented to image and multimodal content generation, and the planning model understands requirements and automatically organizes the creation process.
Behind each of these Agents is the Qwen large model specially optimized for smartphones, and the Harness that cooperates and adapts with it for scheduling.
For mobile phone manufacturers, this solution adopts a modular delivery method: the model base, Harness, end-cloud collaboration, operation and maintenance and security capabilities can be accessed on demand, and manufacturers can also continue to add their own tools, Skills and system capabilities. In the AI smartphone industrial chain, Alibaba hopes to play the role of not a substitute for the mobile OS, but an Agent infrastructure layer between the OS and the basic model.
During the Yunqi Conference, Intelligent Emergence talked with Xu Zhuhong, Vice President of Alibaba ATH Business Group and Head of Token Foundry Qwen Mobile Intelligence, about this AI smartphone solution and the next-generation Agent phones in his eyes.
The following is the edited conversation between Intelligent Emergence and Xu Zhuhong.
01. Agent Phone is an Irresistible Trend
Intelligent Emergence: Why did you launch the Qwen Intelligence business, and what opportunities did you see?
Xu Zhuhong: The mobile Agent direction has attracted industry attention since last year, and it has really become popular this year. Up to now, almost every mobile phone manufacturer and large model manufacturer is making relevant layouts. The mobile phone industry is facing the biggest technological change since the release of iPhone in 2007, and we have seen huge market opportunities in the future.
Mobile Agent has the opportunity to achieve tenfold or even a hundredfold growth in terms of penetration, usage depth and commercial revenue in the future, but today this industry is still in its early stage, and it requires everyone to invest together. The core value that Alibaba can provide in this matter is our model and our full-stack AI capabilities. Therefore, we decided to incubate the Qwen Intelligence business line, hoping to build a general intelligent platform for the industry, so as to accelerate the development of the entire AI smartphone and accelerate the maturity of user experience.
Intelligent Emergence: There are so many smart devices now, why did you choose smartphones as the carrier of this business? What special advantages do smartphones have?
Xu Zhuhong: Smartphones are already the largest personal smart terminals in China, with a penetration rate of almost 100%, occupying 90% of the usage time of most people, and 90% of users' context is also precipitated on smartphones. If every user has a personal AI assistant in the future, most of the interactions of this assistant will most likely happen on the smartphone. So I believe that Agent Phone is the first entry for Personal Intelligence to enter the real world, and it is also the only way to move towards personal ASI.
Intelligent Emergence: Are there any special difficulties?
Xu Zhuhong: Tasks on mobile phones are more fragmented, and the challenge of intention understanding is greater — in the mobile scenario, intentions are often vague, and users' queries are usually very short, generally no more than 20 words.
Smartphones also have various sensors that can receive different modalities, including a series of perception signals such as sound, image, and location, which are not available on PCs. These signals can bring richer context to smartphones, and also give mobile Agent the opportunity to provide more active services. But this also puts forward higher requirements for the core capabilities of models and Agents, such as multimodal understanding and interaction, and how to make better reasoning decisions. These are all problems we need to solve in the mobile scenario.
In addition, the mobile scenario also faces stricter engineering constraints, such as latency, power consumption, and cost, which are much stricter than those of PCs. There is also the issue of privacy and security, which also needs to be treated with more stringent standards in the mobile scenario.
Intelligent Emergence: The form of "mobile assistant" has always existed, such as Siri, but in the past, such assistants did not become the real entry for users to use mobile phones. So which variables have changed today that allow smartphones to become such an entry?
Xu Zhuhong: The most core variable is the improvement of overall intelligence.
The capability boundary of mobile Agent will evolve from completing some simple instruction tasks to supporting more vague intentions, invocation of more tools, cross-application operations, and even complex tasks spanning hours and days in the future. With the further improvement of model capabilities, we feel that the critical point of qualitative change is not far away.
In fact, with the further improvement of Agent capabilities, users are also being continuously educated, and the overall user mindset is maturing rapidly. For example, in the past two years, we have seen that Coding Agent has been widely used; the explosion of OpenClaw at the beginning of the year also indicates that the next outbreak point of Agent will most likely occur on smartphones.
02. How Models and Agents Truly Understand Users in Smartphones
Intelligent Emergence: Please introduce Qwen Intelligence in detail. What kind of business is it?
Xu Zhuhong: This is a model and Agent solution that we have deeply optimized for the mobile scenario.
Our positioning is very clear: we want to be a toB solution provider, not to produce smartphones ourselves. We hope to empower mobile phone manufacturers by outputting solutions, and help them build the next-generation Agent phones together.
Therefore, we stepped forward in the early stage of industry development, built solutions around our Qwen large model, and drove the overall intelligence improvement of the industry. We hope to promote the accelerated development of Agent phones, so that personal intelligence can truly enter the real world.
Intelligent Emergence: This is actually a solution composed of three parts: "model + platform + Agent solution". Why is it designed in this way? What problems do these three parts solve respectively?
Xu Zhuhong: Among these three parts, the bottom layer is our Qwen large model, which is an intelligent base deeply optimized for the mobile scenario. It is not enough to directly migrate the traditional general model to the mobile scenario. The model needs to be deeply optimized for the mobile scenario — compared with the general model, it not only has stronger capabilities, but also higher cost performance.
The middle layer is the platform layer, which supports modular access. Moreover, we do not simply provide customers with a black-box API, but provide a customizable platform — for example, Harness and some customer business rules can be customized. In addition, we also provide a one-stop operation and maintenance platform to improve the overall operation and iteration efficiency, hoping to help customers reduce R&D and access costs while meeting their differentiated competition needs.
The top layer is our Agent solution. We provide high-value vertical scenario solutions, and are also willing to support manufacturers to polish some high-quality vertical Agents, so as to better meet user needs in real scenarios.
Finally, I would like to emphasize the aspect of security and privacy, which is a concern that both mobile phone manufacturers and users pay close attention to during the implementation of the solution. Therefore, we will design our product solutions with the highest security standards, and provide all-round protection to ensure that users' privacy is maximized. Whether it is model R&D, Agent design, or overall platform development, we always strictly abide by the security boundaries.
Intelligent Emergence: Why is the experience not good enough if a smartphone is directly connected to a general large model? Why do we need to make a customized model for smartphones?
Xu Zhuhong: The mobile scenario has very special challenges.
As we talked about earlier, the needs of mobile phone users are often fragmented and their intentions are vague. Therefore, to accurately understand user needs, the model needs to dig out user information from the shared context. The shared context may come from different parts of the mobile phone, such as different sensors and different applications, which is very different from PCs.
In the aspect of action space, the difference is also great. Smartphones have various system tools, and it is a highly complex Agent environment — how to invoke different applications, different tools, and different APIs is also extremely challenging.
Finally, smartphones are also very sensitive to latency, power consumption, and cost.
These characteristics of smartphones put forward very high engineering constraint requirements for the end-cloud collaboration, closed-loop execution and operation security of the model. These challenges cannot be solved by other means, and we still need to do in-depth optimization for the model — we need to integrate some scenario-specific requirements of smartphones, such as end-side perception, understanding of tools, requirements for reasoning experiments, thinking speed, memory usage, etc. into the model for optimization.
Intelligent Emergence: What specific optimizations have been made?
Xu Zhuhong: In real business scenarios, we not only evaluate the accuracy of the model itself, but also evaluate whether the completion rate of specific end-to-end tasks can achieve good results. Our mobile model is optimized for these real scenarios of smartphones to achieve SOTA as much as possible.
Today, the success rate of our end-to-end tasks exceeds 90%. In mobile-related evaluations, we also outperform the capabilities of the most cutting-edge general models in the industry.
Intelligent Emergence: You just mentioned "end-cloud collaboration". What kind of tasks are suitable for running on the end side, and what are suitable for processing on the cloud side?
Xu Zhuhong: In the long run, with the continuous improvement of the computing power of mobile phone chips, end-cloud collaboration is an inevitable trend. We also believe that if Agent Phone wants to provide a good user experience, the underlying architecture must adopt end-cloud collaboration — the end side is responsible for low-latency, high-privacy requirements, while taking cost into account; the cloud side mainly handles some complex requests, such as complex answers based on open knowledge, and planning and disassembly of some long-term tasks.
In the short term, the computing power on the end side is still relatively limited. Limited by the strict engineering constraints on the mobile end side, the end-side model will focus on handling relatively simple tasks, such as intention understanding and execution of simple instructions. However, with the enhancement of the overall end-side computing power and the improvement of model intelligence, part of the tasks that were previously completed on the cloud side will gradually be moved to the end side for closed-loop processing. The overall time cost of end-cloud collaboration will also be greatly improved in the future.
Intelligent Emergence: To enable Agent to complete complex tasks, it not only relies on the model, but Harness is also very important. In this solution, what problems do the model and Harness solve respectively?
Xu Zhuhong: To complete real complex tasks, in-depth adaptation between the model and Harness is indeed required.
The model is the intelligent base of Agent, which needs to achieve global understanding in open targets. For today's models, if you give them a very clear and complete target, they can already do a good job in disassembling steps. But in real scenarios, users rarely give very complete targets.
For example, if a user says "help me arrange a business trip to Beijing today", there are many different requirements and steps behind this expression, such as booking air tickets, booking hotels, modifying the schedule, notifying colleagues, etc. The model needs to infer the implicit requirements behind this simple sentence based on various information including messages, preferences, historical conversations, etc. What's more difficult is that users will change their minds in the middle. For example, if the original Agent is preparing to book a flight ticket, but the user says forget it, I will take the high-speed train this time, then the model needs to maintain the understanding and judgment of the final target in these partial adjustments — whether the user is modifying part of the original plan, or putting forward a new task request. This actually puts forward very high requirements for the model.
Then Harness is a key module that enables the overall task to advance. It needs to maintain the task status across applications and sessions. For example, which step the Agent has reached, what the intermediate results are, what is still missing, etc. More critically, what to do when an exception occurs during the execution of a certain task — whether to continue execution, retry, roll back, or confirm with the requester again. These are all things that Harness needs to handle.
In addition, to make Agent Phone understand users better, Agent also needs to maintain a set of active service capabilities, and use a fine multi-layer memory system to provide personalized services. At this time, long-term and short-term user identity recognition and richer context processing are required. Only in this way can we achieve the ability to provide accurate active services without disturbing users.
Therefore, we need to provide a deeply optimized overall solution, including the intelligent base of the model and a set of Harness that adapts and evolves with the model's capabilities. This set of Harness also supports users to customize some capabilities by themselves. When the model is upgraded, this set of Harness will also be adapted and optimized accordingly. For example, if the model adds support for a certain tool invocation, Harness also needs to incorporate this new tool in intelligent orchestration.
The collaborative optimization of the model and Harness improves the overall performance of Agent, and will continuously improve the understanding of users' personalized needs as users' experience and memory accumulate.
Intelligent Emergence: How to realize the collaborative optimization of model and Harness? Is collaborative training done in the training stage?
Xu Zhuhong: It is mainly post-training. We need to add the Agent environment in the post-training of the model, so that the model can learn various adaptation information of the environment through the reinforcement learning of Agent.
Intelligent Emergence: So the key is the task trajectory data from real scenarios?
Xu Zhuhong: Trajectory data is part of it, and the other part is to get some execution feedback in the real environment — such as when exceptions occur, under what conditions different tools and different skills need to be invoked, etc.
Many times, these things cannot be simulated in a simulation environment. Therefore, in addition to training in the simulation environment, we also need to do a good job in the collaboration between the model and Agent Harness in the real machine scenario.
Intelligent Emergence: Is this kind of collaborative training necessary for all Agents that help users complete complex tasks?
Xu Zhuhong: I think it is necessary. Complex tasks require Agents to cross multiple applications, multiple sessions, and even multiple scenarios. At this time, both the model's own planning ability and the stable execution of Agent are improved. If the model and Harness are not coordinated, it is difficult to get this done well.