Xu Zhuhong, Head of Qwen Intelligence: The competition for Agent smartphones is not only about the model.

Over the past two years, many smartphone manufacturers have integrated large language models into their devices.

"The smartphone is the smart terminal closest to the Personal Agent" has almost become a universal industry consensus at the current stage: the penetration rate of smartphones is close to 100%, they stay with users all the time, connect communications, payments, travel, imaging and various life services, take up 90% of the usage time of most people, and naturally precipitate the most complete personal context of an individual.

But in most cases, smartphones equipped with large language models still only provide users with a smarter "phone assistant": they can answer questions, summarize documents and edit images, but can hardly truly complete a complex task on behalf of users.

Nowadays, the emergence of Agent is changing this situation.

Agent enables smartphones to have task planning, tool invocation and long-term execution capabilities — users no longer need to tell the phone every step of how to do things, they only need to give the goal, and the rest of the problems will be understood, disassembled by the model, which will call system capabilities, applications and services to complete the task.

This also shifts the focus of competition in AI smartphones: what determines user experience is no longer just how smart the model is, but the entire set of systems beyond the model: how Context is organized, how Memory is invoked, how tools and Skills are orchestrated, how the GUI is operated, and how to recover after a task fails, etc.

Qwen Intelligence, the mobile smart business released by Alibaba at this year's Yunqi Conference, is exactly such a system.

Alibaba does not manufacture smartphones by itself, and Qwen Intelligence is not just a model or an Agent, but a full-stack AI smartphone solution for mobile phone manufacturers. Based on the Qwen large model, it provides terminal manufacturers with a complete set of "intelligent layer" covering underlying models, Agent platforms and scenario solutions. It covers capabilities ranging from user intention understanding, complex task planning, to tool invocation, cross-application execution, multimodal creation, memory and active services, with the goal of helping smartphones further evolve from "understanding users" to "completing tasks on behalf of users".

Qwen Intelligence has released three Agent solutions this time: Mobile Planner Agent, Mobile Use Agent and Creative Agent, corresponding to "planning, operation and creation" respectively. Among them, Mobile Planner Agent is responsible for understanding complex requirements, disassembling tasks and orchestrating tools; Mobile Use Agent is responsible for actually operating the mobile phone, adopting a hybrid mode of "API first, GUI as fallback"; Creative Agent is oriented to image and multimodal content generation, where the planning model understands requirements and automatically organizes the creation process.

Behind each of these Agents, there are Qwen large models specially optimized for smartphones, and the Harness that is collaboratively adapted with them for scheduling.

For mobile phone manufacturers, this solution adopts a modular delivery method: the model base, Harness, end-cloud collaboration, operation and maintenance and security capabilities can all be accessed on demand. Manufacturers can also continue to add their own tools, Skills and system capabilities. In the AI smartphone industry chain, Alibaba hopes to act not as a substitute for the mobile phone OS, but as an Agent infrastructure layer between the OS and the foundational model.

During the Yunqi Conference, Intelligence Emergence talked with Xu Zhuhong, Vice President of Alibaba ATH Business Group and Head of Token Foundry Qwen Mobile Intelligence, about this AI smartphone solution and the next-generation Agent phones in his eyes.

The following is the conversation between Intelligence Emergence and Xu Zhuhong, slightly edited.

01. Agent phones are an unstoppable trend

Intelligence Emergence: Why did you launch the Qwen Intelligence business, what kind of opportunities did you see?

Xu Zhuhong: The mobile Agent direction has attracted industry attention since last year, and it has really become popular this year. Up to now, almost every mobile phone manufacturer and large model manufacturer is making relevant layouts. The mobile phone industry is facing the biggest technological change since the release of the iPhone in 2007, and we have seen huge market opportunities in the future.

Mobile Agent has the potential to achieve tenfold or even a hundredfold growth in terms of penetration, usage depth and commercial revenue in the future, but today this industry is still in its early stage, which requires everyone to invest together. The core value that Alibaba can provide in this matter is our model and our full-stack AI capabilities. Therefore, we decided to incubate the Qwen Intelligence business line, hoping to build a general intelligent platform for the industry, so as to accelerate the development of the entire AI smartphone sector and make the experience mature faster.

Intelligence Emergence: There are so many smart devices now, why did you choose smartphones as the carrier for this business? What special advantages do smartphones have?

Xu Zhuhong: Smartphones are already the largest personal smart terminals in China, with a penetration rate of almost 100%, occupying 90% of the usage time of most people, and 90% of users' context is also precipitated on smartphones. If every user has a personal AI assistant in the future, most of the interactions with this assistant will most likely happen on smartphones. So I believe that Agent phones are the first entry point for Personal Intelligence to enter the real world, and also the only way to personal ASI.

Intelligence Emergence: Are there any special difficulties?

Xu Zhuhong: Tasks on smartphones are more fragmented, and the challenge of intention understanding is also greater — in the mobile phone scenario, intentions are often vague, and users' queries are usually very short, generally no more than 20 words.

Smartphones also have various sensors that can receive different modalities, including a series of perception signals such as sound, image, and location, which are not available on PCs. These signals can bring richer context to smartphones, and also give mobile Agents the opportunity to provide more active services. But this also puts forward higher requirements for the core capabilities of models and Agents, such as multimodal understanding and interaction, and how to make better reasoning decisions. These are all problems we need to solve in the mobile phone scenario.

In addition, the mobile phone scenario also faces stricter engineering constraints, such as latency, power consumption, and cost, which are much stricter than those of PCs. There is also the issue of privacy and security, which also needs to be treated with more stringent standards in the mobile phone scenario.

Intelligence Emergence: The form of "mobile assistant" has always existed, such as Siri, but in the past this kind of assistant did not become the real entry point for users to use mobile phones. So which variables do you think have changed today that allow smartphones to become such an entry point?

Xu Zhuhong: The most core variable is the improvement of overall intelligence.

The capability boundary of mobile Agents will evolve from completing simple instruction tasks to supporting more vague intentions, invocation of more tools, cross-application operations, and even complex tasks spanning hours and days in the future. With the further improvement of model capabilities in the future, we feel that we are not far from the critical point of qualitative change.

In fact, with the further improvement of Agent capabilities, users are also being continuously educated, and the overall user mindset is maturing rapidly. For example, in the past two years, we have seen that Coding Agent has been widely used; the popularity of OpenClaw at the beginning of this year also indicates that the next outbreak point of Agents will most likely occur on smartphones.

02. How models and Agents truly understand users in smartphones

Intelligence Emergence: Please introduce Qwen Intelligence in detail, what kind of business is it?

Xu Zhuhong: This is a model and Agent solution that we have deeply optimized exclusively for the mobile phone scenario.

Our positioning is very clear: we are to be a toB solution provider, not to manufacture smartphones ourselves. We hope to empower mobile phone manufacturers by outputting solutions, and help them build the next generation of Agent phones together.

Therefore, we stepped forward in the early stage of industry development, built solutions around our Qwen large model, and drove the improvement of overall industry intelligence. We hope to promote the accelerated development of Agent phones, so that personal intelligence can truly enter the real world.

Intelligence Emergence: This is actually a solution composed of three parts: "model + platform + Agent solution". Why is it designed in this way? What problems do these three parts solve respectively?

Xu Zhuhong: Among these three parts, the bottom layer is our Qwen large model, which is an intelligent base deeply optimized for the mobile phone scenario. It is not enough to directly migrate the traditional general model to the mobile phone scenario. The model needs to be deeply optimized for the mobile phone scenario — compared with the general model, it not only has stronger capabilities, but also has higher cost-effectiveness.

The middle layer is the platform layer, which supports modular access. Moreover, we do not simply provide customers with a black-box API, but a customizable platform — for example, Harness and some customer business rules can be customized. In addition, we also provide a one-stop operation and maintenance platform to improve overall operation and iteration efficiency, hoping to help customers reduce R&D and access costs while meeting their needs for differentiated competition.

The top layer is our Agent solution. We provide high-value vertical scenario solutions, and are also willing to support manufacturers to polish high-quality Agents in vertical categories, so as to better meet user needs in real scenarios.

Finally, I want to emphasize that security and privacy are issues that both mobile phone manufacturers and users pay great attention to during the implementation of the solution. Therefore, we will also design our product solutions with the highest security standards, and provide all-round protection to ensure that users' privacy is maximally protected. Whether it is model R&D, Agent design, or overall platform development, we always strictly abide by the security boundaries.

Intelligence Emergence: Why is the experience not good enough when a smartphone is directly connected to a general large model? Why do we need to make a customized model for smartphones?

Xu Zhuhong: The mobile phone scenario has very special challenges.

As we talked about earlier, the needs of mobile phone users are often fragmented and their intentions are vague. Therefore, to accurately understand user needs, the model needs to dig out user information from the shared context. The shared context may come from different parts of the smartphone, such as different sensors and different applications, which is very different from PCs.

In terms of action space, the differences are also huge. Smartphones have various system tools, which form a highly complex Agent environment — how to call different applications, different tools, and different APIs is also a huge challenge.

Finally, smartphones are also very sensitive to latency, power consumption, and cost.

These characteristics of smartphones put forward high engineering constraint requirements for the end-cloud collaboration, closed-loop execution and operation safety of the model. These challenges cannot be solved by other means, and we still have to make deep optimization for the model — we need to integrate some scenario requirements unique to smartphones, such as end-side perception, understanding of tools, requirements for reasoning experiments, thinking speed, memory usage, etc. into the model for optimization.

Intelligence Emergence: What specific optimizations have been made?

Xu Zhuhong: In real business scenarios, we not only evaluate the accuracy of the model itself, but also evaluate whether the completion rate of specific end-to-end tasks can be achieved. Our mobile model is optimized for these real scenarios of smartphones, striving to reach SOTA as much as possible.

Today, the success rate of our end-to-end tasks exceeds 90%. In mobile-related evaluations, we also surpass the capabilities of the most cutting-edge general models in the industry.

Intelligence Emergence: You mentioned "end-cloud collaboration" earlier. What kind of tasks are suitable for running on the end side, and what kind of tasks are suitable for being processed by the cloud side?

Xu Zhuhong: In the long run, with the continuous improvement of smartphone chip computing power, end-cloud collaboration is an inevitable trend. We also believe that to deliver a good user experience for Agent Phone, the underlying architecture must adopt end-cloud collaboration — the end side is responsible for low-latency, high-privacy requirements while taking cost into account; the cloud side mainly handles some complex requests, such as complex answers based on open knowledge, and planning and disassembly of some long-term tasks.

In the short term, the computing power on the end side is still relatively limited. Limited by the strict engineering constraints on the mobile end side, the end-side model will focus more on relatively simple tasks, such as intention understanding and execution of simple instructions. However, with the enhancement of the overall end-side computing power and the improvement of model intelligence, some tasks that were previously completed on the cloud side will gradually be closed on the end side. The overall time cost of end-cloud collaboration will also be greatly improved in the future.

Intelligence Emergence: To make Agents complete complex tasks, it not only depends on the model, but Harness is also very important. In this solution, what problems do the model and Harness solve respectively?

Xu Zhuhong: To complete real complex tasks, deep adaptation between the model and Harness is indeed required.

The model is the intelligent base of the Agent, which needs to achieve global understanding in open goals. For today's models, if you give them a very clear and complete goal, their step disassembly performance is already very good. But in real scenarios, users rarely give very complete goals.

For example, the user says "help me arrange a business trip to Beijing today" — this expression involves many different requirements and steps behind it, such as booking air tickets, booking hotels, adjusting schedules, notifying colleagues, etc. The model needs to infer the implicit requirements behind this simple sentence based on various information including messages, preferences, historical conversations, etc. What is more difficult is that users will change their minds halfway. For example, the Agent was originally preparing to book a flight ticket, but the user says never mind, I will take the high-speed train this time. Then the model needs to maintain the understanding and judgment of the final goal in these partial adjustments — whether the user wants to modify part of the original plan, or put forward a new task request. This actually puts very high requirements on the model.

Harness is a key module that enables the overall task to move forward. It needs to maintain the task status across applications and sessions. For example, which step the Agent has completed, what the intermediate result is, what is still missing, etc. More critically, when an exception occurs during the execution of a task, what to do — whether to continue execution, retry, roll back, or confirm with the requester. These are all issues that Harness needs to handle.

In addition, to make the Agent phone understand users better, the Agent also needs to maintain a set of active service capabilities, and use a fine-grained multi-layer memory system to provide personalized services. At this time, we need some user long-term and short-term identity recognition, and richer context processing. Only in this way can we achieve the capability of active service that is accurate and does not disturb users.

Therefore, we need to provide a deeply optimized overall solution, including the intelligent base of the model, and a set of Harness that adapts and evolves collaboratively with the model capabilities. This set of Harness also supports users to customize some capabilities. When the model is upgraded, this set of Harness will also be adapted and optimized accordingly. For example, if the model adds support for a certain tool invocation, Harness will also incorporate this new tool in intelligent orchestration.

The collaborative optimization of the model and Harness improves the overall performance of the Agent, and will continuously improve the understanding of users' personalized needs as users' experience and memory continue to accumulate.

Intelligence Emergence: How to realize the collaborative optimization of the model and Harness? Is collaborative training done in the training phase?

Xu Zhuhong: Post-training is the main approach. We will add the Agent environment in the post-training of the model, so that the model can learn various adaptation information of the environment through Agent reinforcement learning.

Intelligence Emergence: So the key is the task trajectory data from real scenarios?

Xu Zhuhong: Trajectory data is part of it, and the other part is to get some execution feedback in the real environment — such as when exceptions occur, under what conditions different tools and different skills need to be called, etc.

Many times, these things cannot be simulated in a simulation environment. Therefore, in addition to training in the simulation environment, we also need to do a good job in the collaboration between the model and Agent Harness in the real device scenario.

Intelligence Emergence: Is this kind of collaborative training very necessary for all Agents that help users complete complex tasks?

Xu Zhuhong: I think it is necessary. Complex tasks require Agents to cross multiple applications, multiple sessions, and even multiple scenarios. At this time, both the planning capability of the model itself and the stable execution of the Agent have higher requirements. If the model and Harness are not coordinated, it is difficult