Edge-side large models have become a rigid demand. Are the chips ready?

At the 2026 WAIC venue, we observed that the capabilities of large models are breaking out of the confines of the internet and chat windows.

Two years ago, almost all industry visions for large models were centered on the cloud: larger parameters were better, more powerful clusters were superior, and whoever stacked more GPUs in their data centers would emerge as the winner. In fact, this logic still holds true today, but it only tells half the story.

As for the other half, it is unfolding quietly beyond the cloud.

With the advent of Agents, people have discovered that inference tasks are beginning to migrate away from data centers, settling onto local PCs, mobile phones, the "brains" of robots, and countless other unnamed end devices. These terminals come in diverse forms, yet they all face the same "impossible triangle": power consumption of only a few watts, cost capped at tens of dollars, while customers demand they can run models with billions or even tens of billions of parameters.

The semiconductor industry has not solved such problems in the past decade, because cloud rules are about pushing the upper limit, where the most powerful computing dominates everything. For edge-side scenarios, the goal is to maximize the efficiency of large model inference within the rigid constraints of physical laws.

This problem has rewritten the competitive rules for edge-side AI chips.

01. Large Models Are Moving Toward the Edge

Why have edge-side large models become an irreversible trend "overnight"?

Recently, Jensen Huang has repeatedly emphasized a judgment: the demand for inference computing power will increase by approximately 10,000 times, and Tokens will become a new unit of measurement.

The number itself is not the point — the change in magnitude is. Once inference becomes such a high-frequency, continuous activity directly tied to billing, relying entirely on the cloud will become increasingly cost-prohibitive. A single inference call may cost very little, but hundreds of millions of calls per day will add up to a staggering expense.

The cost pressure created by this new scale of inference will push more and more computing tasks to the edge.

The Chinese market is responding faster than overseas, with a large number of application solution providers already approaching chip vendors to request hybrid local-cloud solutions. Vertical industries such as law and finance have particularly specific demands, with higher requirements for data privacy and computing latency, as well as a sharper sense for productivity tool iterations.

Compared with verifying concepts, the market needs delivery-ready products that can operate stably and be mass-produced. From the perspective of chip companies, this is where the market shift lies: AI chips optimized for edge-side large models will soon translate into batches of real orders.

A Shenzhen-based company provides a case study: they built a hybrid edge-cloud Agent platform running industry-specific agents for daily operations, using an algorithm to dynamically allocate tasks and determine in real time which tasks run locally and which go to the cloud. After operating for some time, the conclusion was: approximately 80% of tasks are completed locally, with only 20% going to the cloud. The platform also automatically calculates how much Token cost customers have saved.

This figure illustrates one key point: the workflows of numerous agents can be fully supported by local computing power. Moreover, whether for end devices or better AI chips, customers only need a one-time investment to acquire local computing power, whereas cloud token bills are charged based on usage on a monthly basis.

Once the economic math adds up, this migration becomes irreversible.

Cost is only the first layer; latency is the second. Edge-side inference can achieve millisecond-level response, while cloud inference is constrained by network conditions — for Agents requiring real-time interaction, latency is a critical flaw that can disrupt tasks.

Privacy is the third factor bringing edge-side large models to center stage. The recognition that data is an asset is becoming widespread, and local processing means photos, files, and video data always remain on the end device, ensuring security and enabling offline functionality.

In other words, edge-side large models can connect to the internet to retrieve external information, while local data and processing capabilities stay on the device, allowing operation even when disconnected. For many industrial equipment and mobile terminals that need to operate in network dead zones, secure offline functionality is a lifeline for digital transformation.

The combination of these factors reveals the true reason for the value escalation of edge-side large models: the value anchor of large models is shifting from knowledge density to action density.

A model that can chat only requires cloud computing power and data, but a model that can perform tasks — due to the significant increase in inference frequency and continuity — brings more prominent cost and security issues. It must move toward the secure, flexible, and controllable edge, so the proportion of edge-side operations will only continue to grow.

In the future, the specific edge-cloud ratio in mainstream solutions will not matter, but the underlying logic is clear: scenarios that can be satisfied by local inference will prioritize local execution; only tasks that local devices cannot handle will go to the cloud.

At the same time, the trend toward edge-side deployment poses a new challenge for hardware: cloud chips compete with raw power, where computing power equals supremacy, but the edge-side game follows different competitive rules, testing a different set of capabilities.

AI chips moving toward edge-side large models cannot avoid a critical entry threshold: how to solve that "impossible triangle" — where performance requirements are increasingly high, demanding the ability to run larger models and process tasks faster, while power consumption remains only a few watts and cost is capped at tens of dollars.

Performance, cost, and power consumption are three interlocking constraints. Compromising any one of them will strip edge-side large models of their practical value.

02. New Terminals, New Solutions

A cohort of entirely new terminals has brought this "impossible triangle" to the forefront.

In past years, AI played a minor role on end devices — for example, adding photo editing optimization to mobile phones or a voice assistant to PCs, which led to the early emergence of the AI PC category. Chip vendors' responses were also lightweight: Intel integrated a small GPU or NPU in Panther Lake, while Qualcomm embedded an NPU in Snapdragon, serving as incremental enhancements to the PC category.

But starting in 2025, new device forms began to emerge.

"They don't look like any computer you've seen before — they might resemble eggs, vases, or routers," a chip entrepreneur described customer prototypes using these metaphors.

Their common trait is: they are not designed "for humans," but are native hardware built for AI and Agents. As a result, they do not require touchscreens, keyboards, or human operators sitting in front of them, and may not even have displays. However, they support voice and multimodal interaction and operate 24/7 without interruption. Their sole purpose is to give AI agents a physical, continuously connected body.

These devices have a newly popularized name: Agent Computer.

Earlier this year, Lenovo launched the world's first personal super agent Qira at the Sphere in Las Vegas, while NVIDIA advanced from DGX Spark to RTX Spark. The actions of these two giants point to the same direction: turning Agent Computer from a concept into a real hardware category.

WAIC Venue: Lenovo AI Host P7

But ultimately, the application scenarios, rather than PC vendors or chip manufacturers, will determine the mainstream form of this category.

For the relatively general-purpose home and enterprise scenarios, two distinct groups are currently developing two different sets of devices. Home users want a central brain, placed in the living room, to control lighting, remember conversations, and monitor the care of the elderly and children; enterprises need productivity tools to run legal document processing, data analysis, and code reviews.

A single hardware manufacturer can hardly master both paradigms simultaneously, and a single chip architecture that covers both needs is far from trivial. This means the edge-side terminal market is inherently fragmented.

The diversification and fragmentation of terminal forms, in turn, are reshaping the definition of AI chips.

A device that continuously supports Agent tasks has completely different local computing requirements compared to devices running a single local model in the past. It needs an independent, sufficiently powerful NPU with high bandwidth, large storage capacity, and the ability to process complex long-context tasks. A single task may run for half an hour or longer without interruption.

New terminals have sharpened the impossible triangle further. All AI chip players face the same problem: maximizing efficiency, a point that is undisputed. However, the solutions are clearly diverging.

Qualcomm and Apple follow the SoC extension path, adding NPUs to their existing ARM architectures. The advantage is a mature ecosystem already running on billions of devices, but the ceiling for improvements is right overhead. Even with advanced processes like 3nm or 2nm, the diminishing returns of process technology cannot bring qualitative leaps.

NVIDIA is extending downward from GPGPU, with no shortage of computing power reserves. However, its 30-year-old CUDA ecosystem is both a moat and a historical burden. Moving an entire architecture optimized for cloud training down to the edge is comparable to carrying a mountain downhill.

Between the giants' divergent paths, a third approach has emerged: Processing-in-Memory (PIM).

The essence of PIM is redesigning at the architectural level, breaking the physical boundary between computing and storage. The bottleneck of traditional chips lies in the constant movement of data between computing units and storage units, wasting power and time. The PIM solution performs calculations directly within the storage units, minimizing the overhead of data movement.

Compared with ecosystem extension paths, this new route will be slower and more difficult in the early stages, requiring repeated test chip iterations to solve unprecedented mass production challenges. However, its advantage lies in the structural efficiency gains of the new architecture, which comes with a much higher upper limit.

Few chip players are betting on this path, precisely because overseas giants cannot afford to abandon their ecosystem games. Among manufacturers daring to pursue architectural innovation, domestic players are in a prominent position. Founded at the end of 2020, Motian Intelligence is one of the earlier movers and a more resolute advocate of the PIM path.

At this inflection point of edge-side large models and Agent Computers, chip manufacturers are essentially starting from the same baseline — unlike in mobile basebands or cloud training chips, where overseas players have a decade-long lead. In the competition for edge-side chips, domestic players do not lack potential first-mover advantages; instead, they have the opportunity to gain an early lead in mass production speed and industry implementation.

03. The Competition for Edge-Side Chips Has Only Just Begun

Edge-side AI chips represent a newly opened track, where players are accelerating their entry.

The current landscape can be roughly divided into three categories: SoC giants like Qualcomm and Apple, with mature ecosystems and massive shipments, for whom edge-side large models remain incremental demands — their improvement-oriented DNA limits their performance ceiling. Qualcomm is currently developing an independent NPU, indicating it acknowledges that integrated solutions are no longer sufficient.

NVIDIA is moving downward from the cloud with the computing power momentum of its GPGPUs, delivering 1 PetaFLOP of AI computing power into consumer-grade devices, which is more than sufficient. However, the power consumption and cost constraints of the edge, which NVIDIA has barely needed to take seriously in the past 30 years, are now tangible market demands.

The third category consists of new players like Motian Intelligence that focus exclusively on the edge. Their disadvantages include limited mass production experience and immature ecosystems, but they also carry no historical architectural baggage, allowing them to redesign chips directly based on the real needs of terminals.

Evidently, the third path, where startups are most concentrated, offers strategic decisions worth analyzing.

Motian Intelligence, which entered the edge-side large model space via the PIM route, is a typical example of a company where "strategy determines position." Back in the second half of 2023, its founder Wu Qiang made a judgment: large models would migrate from the cloud to the edge.

At that time, Agents had not yet emerged, and there was little evidence to support this prediction. The main battlefield for large models was still in the cloud, with domestic "Six Dragons" competing over parameter sizes and cluster scales. As an architectural path, PIM for edge-side large models was akin to making a "horizontal cut" across the industry.

But Wu Qiang identified two key signals: Zhipu AI released its 6B model, and Meta launched Llama 7B. Large models were no longer the exclusive domain of hundreds-of-billions-parameter behemoths. Small models could also deliver practically usable results, avoiding the "using a cannon to shoot a mosquito" scenario in many use cases.

The team tested running local large models on their PIM architecture and achieved excellent performance. As a result, proactively laying out edge-side large models became a logical step for Motian, a company founded on PIM technology.

This prediction was intensively validated two years later. In 2025, DeepSeek R1 made its debut, drastically lowering the technical threshold for deploying large models on the edge. In 2026, Agents saw explosive implementation, with numerous hardware companies flooding in to develop Agent Boxes and Agent Computers for consumer, enterprise, and home use. All these hardware vendors needed to solve one problem: finding cost-effective, mass-producible chips.

Having laid out their strategy two years in advance, Motian reaped the first wave of dividends: with few mature solutions available in the market at that time, their first-mover advantage in strategy and technology translated directly into customers and orders.

In terms of product performance, Motian's Manjie M50 achieves 160 TOPS using PIM, with power consumption of approximately 10 watts. A single chip can run models with 35B to 120B parameters, covering scenarios such as AI PCs, robots, and edge inference. Six months to a year ago, edge-side chips could only run 7B models, but now they can support up to 120B parameters, redefining the performance boundary of the edge.

WAIC Venue: Motian Manjie M50

Another piece of supporting evidence is that Motian Manjie M50 has reached mass production readiness. In the AI PC direction, Motian has entered the supply chains of Lenovo and Great Wall: Lenovo's AI Host P7 made its global debut equipped with the M50, and Great Wall's N90 Pro also leverages the M50 to highlight its edge-side computing power. At the operating system level, Motian Manjie M50 has completed deep adaptation with the Kylin V11 operating system. On the edge-side large model track, Motian has also reached a chain-leader ecosystem partnership with China Mobile, with customer scenarios expanding from Agent Computers to embodied robots, edge-side inference, and smart mobile terminals, and mass production orders are being converted.

WAIC Venue: Great Wall N90 Pro Laptop

However, while both chip giants and startups accelerating entry with new architectures have proposed their own solutions, zooming out reveals that the competition for edge-side AI chips is still in its early stages. It can be understood as two distinct phases moving forward.

The first implementation phase prioritizes efficiency — that is, who can maximize the energy efficiency of large model local inference within the physical constraints of power consumption, cost, and area. This is a head-on competition of engineering capabilities, where superficial gimmicks and long-term narratives hold no value.

Looking further ahead, after efficiency gradually converges, the second phase will focus on ecosystem and scenario adaptation. In other words, it will enter NVIDIA's comfort zone, testing whose toolchain is more complete, whose adaptation with mainstream model frameworks is smoother, and who can help customers get their solutions up and running faster — factors that will continue to widen the gap over the long term.

In the first phase, PIM, with its architectural redesign, inherently has structural advantages in efficiency and power consumption, but this window of opportunity will not last forever. Fortunately, domestic manufacturers start on equal footing with overseas players on this track. For example, Motian, with its