The next game point for AI agent smartphones does not lie in large models.

In the past month, the most bustling track in the smart hardware sector has undoubtedly been AI agent smartphones.

First, on July 15, the cyberspace administration authorities released for the first time an announcement covering the filing information of seven mobile-side generative AI services; later, during the World Artificial Intelligence Conference (WAIC), the booths of Stepfun and ZTE Nubia showcasing their AI smartphones were packed with visitors, all eager to catch a glimpse of what the future of smartphones will look like.

Image Source: Qoo Business

Behind AI smartphones lies a critical technology — on-device AI. On-device AI enables large language models to run directly locally on the smartphone, instead of uploading data to cloud servers for processing; chat records, photos, and documents never need to leave the device to be understood and accessed by the AI, which on one hand reduces latency, and on the other hand protects user privacy.

"Qoo Business" has observed that while Stepfun and Nubia have chosen different technical paths for their AI smartphones, on-device AI remains the key breakthrough direction for both brands and all smartphone manufacturers going forward.

01. AI smartphones can get things done, but not that much

In terms of appearance, the two AI agent smartphones from Stepfun and Nubia are almost indistinguishable from ordinary smartphones.

Stepfun's "world's first large-model native agent smartphone" STEPX Neo features an orange-black color scheme and dual rear cameras. Nubia's "world's first mass-produced AI agent smartphone" NaviX Ultra, also known as the Doubao 2nd Generation phone, comes in four color options: blue, pink, silver, and black, with a single punch-hole flat screen.

Honor has unveiled the "world's first robot phone" — Robot Phone — which houses a movable motorized gimbal inside the device, supporting gesture wake-up, automatic tracking of shooting subjects, and even moving in sync with music; the agent solution co-developed by Alibaba and Honor based on the "Tongyi Large Model" is implemented on this device.

Interestingly, in their promotional messaging, all three brands claim their product is the "world's first".

Stepfun also states that it has independently developed the world's first agent-native system (Step AOS), which is not a deeply customized version of Android; however, its UI (User Interface) looks largely similar to the Android system, still featuring a grid of app icons, with apps adopting orange-yellow rounded rectangle icons, and only the icons of some third-party apps showing slight differences.

Image Source: Qoo Business

The most highly anticipated aspect of AI agent smartphones is whether they can break away from the graphical tap interaction method, but neither of the two current models from Stepfun and Nubia delivers such a new experience. According to on-site staff, the selling points of the two phones are the same as the Doubao 1st Generation phone that previously caused a stir in the market: users send text or voice commands through the system assistant, which then schedules different apps to complete tasks.

Stepfun's staff stated that the STEPX Neo currently allows the AI assistant to book tickets, order food delivery, hail a ride, as well as edit documents and reply to messages on Feishu. Except for steps involving personal information and payment, which require manual user authorization, all other procedures are completed autonomously by the AI assistant.

However, the back of the Stepfun phone is equipped with an interactive secondary screen (using dot-matrix LEDs); the combination of the main and secondary screens may support more diverse interaction methods such as voice and vision in the future.

Staff from both phone brands stated that apart from the apps that have already established cooperation, all other apps on the market are being gradually integrated. The launch dates of these two phones have not yet been confirmed, and many details are still being refined.

Image Source: Qoo Business

This means that the future user experience of AI smartphones will depend on the openness of major internet applications.

The Doubao 1st Generation phone was "blocked" by apps like WeChat and Taobao due to security issues, as it adopted the "GUI Agent" technical path (simulating user tap operations) that allowed it to call and execute actions in any app without requiring app permission grants. Perhaps due to security and privacy controversies, neither the Stepfun STEPX Neo nor Nubia has chosen the "GUI Agent" path this time. According to staff introductions, the Stepfun STEPX Neo uses a hybrid approach combining GUI Agent and A2A; the Nubia NaviX Ultra follows a pure A2A path.

The feature of A2A is that it does not need to simulate user taps, but instead calls app capabilities through standard interfaces, with the app executing the actions itself. This approach is also the technical path chosen by major hardware manufacturers such as Huawei, OPPO, and Xiaomi, which is more secure and stable, but highly dependent on the permission openness of the apps.

Judging from the apps that the phones have partnered with so far, utility apps like Meituan and Ctrip are relatively proactive in integrating with mobile AI, as their main revenue comes from order commissions rather than advertising. However, high-frequency social apps such as WeChat, Douyin, and Xiaohongshu, which take advertising as their core revenue source, ceding usage permissions is almost equivalent to giving up their lifeline (WeChat is even developing its own agent assistant).

Whether a cooperation agreement can be reached with these high-frequency apps will directly determine whether agent smartphones become "useful tools" or "expensive toys", and is also the most noteworthy variable going forward.

02. There is still a "hardware barrier" to cross

"Qoo Business" observed that at the booths of these two phone brands, apart from questions about "how to operate", the most frequently asked questions from visitors concern task execution efficiency and data security.

On-device AI is seen as the solution to these issues. The large models such as Doubao and DeepSeek that we normally use on web pages and apps all follow the cloud AI path: the operational logic of cloud AI is that users send commands → data is uploaded to cloud servers → large models process the data → results are returned to the smartphone. During this process, users' chat records, photos, location information, and consumption habits all pass through third-party servers, which on one hand increases latency and slows down the execution speed of agents; on the other hand, it makes many users worry that cloud service providers may misuse, leak, or retain their data for long periods.

The logic of on-device AI is completely different: large models are deployed directly locally on the smartphone, so users' data never leaves the device from generation to processing, directly reducing latency and privacy risks.

However, due to hardware limitations, there is currently no smartphone that can run large models entirely on the device. All smartphone manufacturers currently choose the "cloud-device collaboration" model, where simple conversations are processed locally, and complex long-chain tasks still need to be uploaded to the cloud.

Recently, the cyberspace administration authorities announced the filing information of seven "mobile-side generative AI services", namely Apple, Huawei, OPPO, vivo, Xiaomi, Samsung, and Nubia (ZTE). This is the first time the regulatory authority has disclosed "mobile-side AI" as an independent category, which is seen by the public as a signal that the smartphone industry is starting to sprint toward "on-device AI".

Image Source: Cyberspace Administration of China

Smartphone manufacturers began their R&D on on-device models in 2023. Three years later, the aforementioned companies that completed the filings have successively released their latest achievements.

In June this year, Apple released two on-device models at WWDC, among which the AFM 3 Core Advanced has a total parameter count of 20B but only activates about 4B per inference, balancing performance and efficiency, mainly for basic text generation and instruction following. Also in the first half of the year, Xiaomi released the MiMo-V2 series, with a total parameter count of up to 1T and 42B activated parameters, which can be used for more complex inference, copywriting generation, and in-depth conversations.

Different manufacturers deploy models of varying sizes on-device, and the problems they hope to solve through on-device AI are also different. Li Dahai, CEO of FaceMind, a domestic company focusing on on-device large models, said that on-device models should not be judged solely by parameter size or a single benchmark score, but must simultaneously consider capability, speed, power consumption, and memory usage.

Li Dahai also mentioned that when smartphone manufacturers evaluate on-device models, they usually focus on the adaptation capability between the on-device model and the chip, as well as the inference efficiency of the model. "Users will not accept an AI feature that seems very capable, but causes obvious power drain, overheating, or unstable response as soon as it is used. So when performance levels are similar, the manufacturer that can deliver the experience with lower power consumption and lower latency will have a greater advantage."

Hardware is a major hurdle for on-device AI. Running on-device AI requires a large amount of parallel computing, but the CPU and GPU in a smartphone's SoC chip can hardly meet the computing power demand of on-device AI, which usually requires an NPU specifically designed for AI inference.

An investor focused on consumer electronics said that the current mainstream solution for on-device AI chips is to integrate the NPU inside the SoC, which is the path adopted by both Apple and Qualcomm. There are already two to three dozen domestic companies developing on-device AI chips, each with a different technical path. In the future, the design of smartphone SoC chips may undergo a huge revolution, no longer centered on the CPU, but on the NPU, with the CPU, GPU, and different acceleration units working in coordination.

In addition, on-device AI also imposes higher requirements on the memory bandwidth and storage of smartphones. Taking Apple as an example, only iPhone 15 and later models equipped with A17 and above chips can run large models on the mobile device. Against the backdrop of continuously rising memory prices, the overall manufacturing cost of smartphones is likely to keep increasing going forward.

Overall, AI smartphones are still in the transition phase from "chatting" to "getting things done". Before they can be frequently used by users, AI smartphones still have many problems to solve; the walls of app permissions, model capabilities, and chip computing power are all bottlenecks that must be overcome step by step.

However, in the long run, the smartphone industry, which has been silent for many years, is ushering in a new wave of device upgrades, and the winner this time may not be the winner of the last era. When smartphones start to handle tasks on their own, it is not just the interaction methods that are being rewritten, but also the power landscape of the AI era.

This article is from the WeChat official account "Qoo Business", written by ZHANG Yuge, and published by 36Kr with authorization.