Doubao smartphones will go on sale tomorrow. Can AI help you operate apps now?

Text by Li Zhaofeng

Edited by Zhang Yuxin

Yesterday, the Doubao Mobile Assistant released its consumer version. This AI assistant, built on the Doubao App and developed through cooperation with mobile phone manufacturers at the operating system level, is first launched on the nubia NaviX Ultra under ZTE Corporation, and this mobile phone will be officially released for public sale on September 16.

At the end of last year, the technical preview version of Doubao Mobile Assistant was deployed on the nubia M153 engineering prototype, focusing on demonstrating how AI can "operate the mobile phone" across applications. In less than a year, the newly launched consumer version puts more emphasis on daily usability: in addition to on-screen Q&A, local memory and retrieval, the "operate mobile phone" function is open in Beta form; Doubao has also launched SAEP (Screen Automation Execution Protocol) with a 30-day public notice period.

According to the protocol, third-party applications can independently declare whether to allow AI assistants to perform screen automation, and can also restrict specific operations. During the 30-day public notice period, third-party applications that do not explicitly agree to access will not be operated by default. The consumer version of Doubao Mobile Assistant further adds the right of choice on the application side: third-party applications can reject GUI as a whole, or separately restrict operations such as publishing, deletion, and benefit collection.

People engaged in GUI (Graphical User Interface) Agent research believe: "There must be a native AI assistant in mobile phones in the future. This does not necessarily have to be done by mobile phone manufacturers themselves, but it must finally reach the system level." With the NaviX Ultra and a number of system-level mobile assistants entering the consumer market, system-level AI assistants are moving into the daily usage scenarios of ordinary users.

01 Step out of the dialog box, mobile AI starts to complete tasks across Apps

In July this year, Ni Fei, Senior Vice President of ZTE Corporation and President of nubia, said in an interview with the media: "What users really want is 'peace of mind', but today's mobile phones have more and more functions and more complex operations. This contradiction has accumulated to a certain extent, and a brand-new product form is needed to respond to it."

This is exactly the problem that Doubao Mobile Assistant is trying to solve: it is embedded in the mobile phone system, can search the current screen and local device information after user authorization, call system tools such as calendar, or enter other Apps to complete operations in the Apps. Users can initiate tasks through the AI key, and multiple tasks can also be queued or temporarily prioritized.

For example, in the official demonstration of Doubao Mobile Assistant, after the user takes a photo of the new semester schedule, the assistant can be asked to add the schedule to the personal calendar; or ask it to take a taxi to the address displayed on the screen, add Yindu Building as a passing point, and send the license plate number to Feishu colleagues after getting in the car, and remind the other party to go downstairs in 10 minutes.

This usage has taken shape on the technical preview version nubia M153 last year. In a public demonstration, the user asked Doubao to "pick a cheap identical product" on Taobao based on a photo of a blood pressure monitor in the photo album, and then "send the link to Zhang San". Doubao recognized the product and also recognized that "Zhang San" is a QQ friend, and then continuously completed the operations between the photo album, e-commerce and social applications.

It is reported that M153 called the INJECT_EVENTS system-level permission of Android. After obtaining user authorization, even if the third-party App does not provide an interface for AI, Doubao can simulate tapping, swiping and inputting; when encountering sensitive links such as passwords and face recognition, it will return the mobile phone to the user.

However, in actual use, cross-App operations involving login verification and platform risk control may interrupt tasks. Subsequently, Doubao actively restricted high-risk scenarios such as financial payment and incentive brushing.

In February this year, Google tested Gemini multi-step tasks on the Galaxy S26 series and some Pixel 10 models: it can arrange a ride-hailing with multiple stops for colleagues. After the task is transferred to the background, users can still check the progress through notifications, take over at any time, and confirm before purchasing. Matthew McCullough, Google's Vice President of Product Management for Android Developers, wrote in the official blog: "The measure of an app's success is shifting from 'getting users to open the app' to 'actually getting users' tasks done'."

Since 2026, a number of system-level mobile AI assistants have emerged in batches.

In the HarmonyOS 7 Developer Beta released in June, Huawei Xiaoyi accessed more than 200 system-level sensing data, which can identify the time, location and precautions from the interview invitation and write it into the memo; you can also ask Xiaoyi on the mobile phone to find the files in the computer and send them to the mobile phone.

At the 2026 World Artificial Intelligence Conference (WAIC) in July, the nubia NaviX Ultra, StepX Neo and Honor Robot Phone made their collective debut. After receiving the instruction "Help me deal with the things for tomorrow's report", the Amoo assistant carried by StepX Neo will read the Feishu calendar and group messages, disassemble the tasks, generate a PPT (presentation), and then send the file back to Feishu; it will apply for permission before modifying the document, and the payment-related operations will be confirmed by the user.

At almost the same time, the relationship between super Apps and mobile AI assistants is also changing.

In June this year, WeChat confirmed cooperation with Huawei, Honor, Xiaomi, OPPO and vivo on A2A (Agent to Agent) communication capability. Mobile assistants can submit instructions for sending messages or audio and video calls, which are executed by WeChat and return results, with dual authorization throughout the process. In August, vivo Blueheart Xiao V cooperated with Alipay's service agent "Abao". When users say "Help me call a car to the high-speed railway station" or "Help me issue a taxi invoice", related services can be completed in the conversation.

It can be seen that in 2026, mobile AI assistants have entered a period of intensive release. In addition to nubia equipped with Doubao Mobile Assistant, Google, Huawei, StepX and Honor have successively brought Agent to the mobile phone system layer, and super Apps such as WeChat and Alipay have also begun to receive instructions from system assistants.

02 AI taps the screen for you, how to avoid crossing the line?

As mobile AI assistants adopting the GUI route become more and more popular, they urgently need to handle three permission issues: whether the user authorizes, whether the application allows access, and who confirms before sending, placing an order and paying. From the SAEP (Screen Automation Execution Protocol) launched by Doubao this time, we may get a glimpse of the security measures related to this permission boundary.

At present, CUA (Computer Operation Agent) has taken the GUI technical route as the mainstream. The problem it is solving is: how AI can directly operate software and how to limit the scope of accessible content. Some researchers describe CUA as "equipping your computer with a very versatile programmer who knows everything, can write code, and also understands network security".

The current operation mode is: for tasks that can be processed structurally, the mobile phone will preferentially call codes, commands and tools; only when encountering software without interfaces, it will turn to identify the screen and click buttons. GUI is more like a fallback capability, allowing Agent to continue to operate those software that have not been opened to AI — there are a huge number of mobile Apps, and various temporary pages are constantly changing, so it is impossible to interface all of them, and GUI is needed to cover the unconnected parts.

A GUI researcher told Intelligent Emergence: "We can't expect all App developers to complete adaptation for AI, and GUI must be a necessary supplement." Another practitioner also mentioned, "In a short period of time, there will still be a lot of software on the market that does not have AI capabilities, especially large-scale software. Even if there are interfaces, they are very complex, and it may not be possible to connect all of them in a short time."

Mobile Agent relies more on the application ecosystem than computer Agent. On computers, the operating system can directly decide which software can be connected with Agent; but on the mobile phone side, the operating system does not have such permissions. Not only are different services encapsulated in their own Apps, but permissions such as positioning, camera, and address book are also jointly controlled by the system and applications. If third-party Apps neither open interfaces nor allow AI to operate the interface, mobile assistants will not be able to continue to execute tasks.

At present, GUI will display every step of operation on the screen. A researcher said, "Everyone can understand what this AI assistant is doing." Users can judge whether to pause or take over accordingly.

But "visible" does not mean "well done". The mobile Agent evaluation benchmark iOSWorld launched by the research team of Carnegie Mellon University this year uses 26 self-developed iOS applications to simulate the digital life of the same user, and designs 133 tasks. The best-performing model only has a success rate of 37% in cross-application tasks, and the real consumer scenario also faces constantly changing pages, identity verification and platform risk control.

Compared with the App scenario, the mobile phone system layer also aggregates users' personal information such as photo albums, address books, locations, and calendars. The more complete the information the mobile AI assistant masters, the more tasks it can complete. Therefore, the next step is to compete for two capabilities: one is "how to create more and better data", and the other is "how to orchestrate the collaboration between various steps and various Agents".

But at the same time, more capabilities also mean higher requirements for the ecosystem.

In response to this, Doubao Mobile Assistant chose to launch SAEP (Screen Automation Execution Protocol), allowing third-party applications to declare in advance whether to allow access and which operations cannot be executed before the GUI Agent executes.

"SAEP is a bit like the Robots Protocol (web crawler access rules) in the AI era. Through this protocol, websites tell automatic programs such as search engines which pages are allowed to be crawled and which are not; SAEP allows applications to declare whether Agent can enter the application and which operations must be stopped." A researcher explained.

After accessing SAEP, third-party applications can either reject all screen automation, or only restrict specific operations such as content publishing, deletion, check-in lottery and benefit collection; the rejection declaration takes effect at any time. During the 30-day public notice period, except for system-built applications, ByteDance's applications and third-party applications that explicitly agree to access, the rest of the applications will not be operated by default. After the public notice period, unresponsive applications will be gradually opened according to the risk level, and applications that explicitly reject will never be operated.

Applications express their willingness through SAEP, and the mobile phone system is responsible for reading the declaration, restricting GUI permissions, and returning control to users at sensitive steps. This requires model companies and terminal manufacturers to complete it together.

Previously, Ni Fei, President of nubia, summarized the cooperation between Doubao and nubia as: "Model companies solve the problem of whether AI can achieve the goal, and terminal manufacturers solve the problem of whether users can really use it." According to the division of labor disclosed by Ni Fei, Doubao is responsible for the large model and Agent capabilities, and nubia is responsible for systems, hardware, product engineering, quality, channels and user services. The scope of cooperation covers models, operating systems and complete machine engineering.

In this division of labor, permissions and trust are ultimately undertaken by the operating system, which means that the relationship between mobile operating systems and Agent will become more and more integrated.

In August this year, at the Alipay AI Ecological Partner Conference held in Hangzhou, Zhou Wei, Vice President of vivo and Dean of the Global AI Research Institute, once said: "Hardware forms will not disappear, but the operating system will be reconstructed — from managing Apps to managing Agents, managing permissions, and managing trust."

Earlier in July, the Cyberspace Administration of China announced for the first time the filing of 7 mobile-side generative AI services separately, involving Apple Intelligence, Huawei Xiaoyi AI Large Model, OPPO AndesGPT Large Model, vivo Blueheart Side Large Model, Xiaomi Surge AI, Samsung Galaxy AI and nubia Doubao Mobile Large Model. System-level mobile AI assistants are entering a period of industry blowout.

A GUI researcher told Intelligent Emergence that compared with independent AI applications, mobile phones have a more stable user relationship: "If you only make AI software, users may use whichever is free; but mobile phones are different, a mobile phone is often used for several years and will accompany users for a long time."

On September 16, NaviX Ultra will be officially on sale. At that time, Doubao Mobile Assistant will officially enter the consumer market, and SAEP will also be tested by real users and application developers.