When AI penetrates deep into GPUs, edge computing power is moving toward convergence.
Recently, Arm announced updates to its new-generation Mali GPU architecture, upgrading the matrix arithmetic units within the graphics pipeline to enhance the collaboration between edge-side large model inference and neural rendering, and launched a unified software scheduling solution for in-vehicle, mobile and embedded terminals.
This approach of integrating AI computing power into GPUs and carrying multiple types of workloads on unified hardware is becoming a common choice across the industry. Similar integration attempts are also unfolding at different market levels, from mobile terminals to desktops, and then to the IP licensing market. Vendors follow different paths, but they are heading in the same direction.
01 Why Local AI Needs GPUs
As edge-side application scenarios continue to enrich, devices need to carry multiple types of tasks such as graphics rendering, machine vision, signal processing and large model inference at the same time.
Taking edge-side large models as an example, from single-round dialogue to multi-modal collaboration, from short context to long context, the workload characteristics are constantly changing, putting forward higher requirements for hardware flexibility. Over the past period, the mainstream idea of chip design is to configure dedicated hardware modules separately for different workloads, and the graphics unit, AI accelerator, and video codec unit each have independent data paths and software stacks. This solution can achieve optimal performance for a single indicator, but its shortcomings are also prominent. The coexistence of multiple accelerators will significantly increase the difficulty of system integration, making it difficult to schedule hardware resources across tasks. Once the terminal requirements are iterated, some dedicated fixed-function hardware will face the problem of insufficient utilization, pushing up the overall cost and power consumption of the chip.
In the early stage of the edge AI boom, a large number of independent NPU products emerged in the market, with specifications designed around fixed operators and model structures. However, the large model architecture, quantization scheme, and operator set are still in a rapid iteration cycle, and the fixed hardware architecture is very likely to face adaptation bottlenecks after the model is updated. The industry has gradually realized that a single dedicated accelerator cannot adapt to the ever-changing AI workloads in the long run. GPUs with graphics capabilities, general-purpose parallel computing and programmable features have begun to become the core carrier of edge integrated computing power. The competitive focus of the computing power architecture has also shifted from simply competing for peak computing power to hardware programmability, software stack versatility and actual energy efficiency performance under multi-task scenarios.
Against this background, many chip manufacturers have long begun to practice this integration idea. NVIDIA relies on Tensor Core to enable GPU hardware to reuse computing power between graphics rendering, machine vision and AI inference; Qualcomm, on the one hand, opens up the scheduling between Adreno GPU and Hexagon NPU, and on the other hand, migrates AI tasks such as super-resolution and frame generation to the graphics pipeline; Imagination's E-series products choose to directly integrate the programmable matrix accelerator into the rendering pipeline, using one set of cores to support graphics, general computing and AI inference at the same time. Therefore, if the edge side wants to run AI smoothly, graphics capability has become an indispensable underlying foundation.
02 Hardware and Software Collaboration Under Computing Power Integration
Basic matrix and convolution computing capabilities have become the core of integrated GPUs supporting AI and visual tasks. AMD's embedded product line integrates dedicated AI units in GPU cores. For industrial robot scenarios, it processes real-time visualized images and computer vision tasks such as target detection and segmentation simultaneously, focusing on multi-task concurrency stability. Intel's Arc integrated graphics continues to iterate XeSS neural super-resolution, deeply binding the AI upsampling capability to the graphics rendering process, reducing the data movement overhead brought by a separate NPU. Behind these trade-offs is the common consideration of edge-side multi-task concurrency efficiency and data movement bandwidth overhead. Although each manufacturer has different implementation paths, they all recognize that a single hardware indicator cannot support the continuously evolving AI workload, and hardware programmability is replacing fixed operators to become the focus of underlying competition.
Neural rendering is the most direct entry point for introducing AI into the graphics pipeline. NVIDIA's DLSS reconstructs low-resolution frames with neural networks, and DLSS 4 further introduces multi-frame generation, which can generate multiple frames for each traditional rendered frame, increasing the frame rate to several times that of traditional rendering at most; it then iterates to 3D-guided neural rendering that leaves lighting and materials to AI. AMD's FSR 4 uses a trained neural network to reconstruct image quality from low-resolution frames, and Intel's XeSS 2 packages AI frame generation and low-latency technology, with the total number of supported games continuing to expand. Super-resolution has become a common entry point for all manufacturers, because it can use neural networks to supplement image details without changing the rendering resolution, taking into account both image quality and performance, avoiding laying dedicated hardware solely for AI functions. Imagination's E-series has launched the NSR neural super-resolution scheme in this direction, which is deeply adapted to its own GPU, adopts single-pass temporal super-resolution processing, and cooperates with network self-compression technology to reduce redundant weights to lower inference power consumption. On a 1GHz single-core configuration, the super-resolution latency from 540p to 1080p is about 2.3 milliseconds, and it plans to provide support through PowerVR SDK and engines such as Unreal Engine and Godot. Putting tasks like super-resolution into the GPU instead of a discrete NPU brings the common benefit of reducing bandwidth loss caused by frame data moving back and forth between different hardware modules.
In terms of basic computing power, a set of comparison data provided by Imagination shows that compared with the previous generation D-series, the E-series has a 4.4-fold increase in Conv2D convolution operation performance, a 4.8-fold increase in GEMM matrix multiplication, and the maximum GPU utilization in matrix operation scenarios can reach 89%. On typical edge language models, its prefill performance is improved by 4.7 times. Taking the Qwen 3.5 4B model as an example, the 1.5GHz quad-core configuration can achieve the first token generation in 0.2 seconds, and the decoding throughput is about 150 tokens per second. To reduce the cost of model migration, the team is submitting the optimized Llama.cpp backend to the upstream open source community, and plans to natively support PyTorch and ONNX Runtime in the future. Graphics remains the core foundation of this IP. The E-series adds DirectX 12 FL11_0 support, with a maximum 54% performance improvement under game workloads compared to the previous generation, and a maximum 39% improvement in performance per watt. It provides multiple configurable specifications covering from smartphones to embedded systems, and the first batch of chips is expected to be taped out within the year.
03 The Whole Industry is Converging to GPUs
NVIDIA continues to polish the unified computing power system based on the Blackwell architecture, relying on the CUDA software stack to connect rendering, general computing and local agent inference. The new version of 3D-guided neural rendering directly injects AI generation capabilities into the real-time rendering process. Consumer-grade hardware like RTX Spark converges graphics, CPU and AI computing power into a single chip, supporting offline operation of large-parameter local agents, which is the terminal implementation of this idea.
At the GTC conference in Berlin on October 21, Jensen Huang's keynote speech also confirmed this direction from the side. He proposed that "Physics AI" will be the main line of the next wave of development, and real-world interaction scenarios such as humanoid robots and autonomous driving rely heavily on devices to complete simulation rendering and AI inference locally at the same time. In the on-site demonstration, RTX Spark ran on the same set of GPU resources to undertake 3D simulation and robot agent inference in parallel.
On the other hand, Arm's new-generation Mali architecture chooses to increase matrix computing resources within the graphics pipeline to achieve unified scheduling of graphics AI tasks, reducing the back-and-forth data movement between multiple accelerators; Qualcomm also adds AI matrix cores to the GPU, bringing neural processing, super-resolution and frame generation into the graphics pipeline, relying on on-chip high-bandwidth memory to keep AI computing local.
From IP vendors to terminal chips, the boundaries between hardware are dissolving: CPUs use vector extensions to reinforce AI capabilities, NPUs are constantly enhancing programmability, and GPUs have outstanding advantages in mixed workload scenarios due to their parallel computing power, mature software ecosystem and native graphics capabilities. With the popularization of mobile multi-modal large models, the edge side is no longer only for running dialogues, but also needs to take into account the collaborative processing of vision and voice, and the demand for unified hardware scheduling is getting stronger and stronger. The industry's criteria for evaluating hardware are also changing. The nominal TOPS peak computing power is no longer the only indicator. End-to-end token throughput, rendering latency, bandwidth overhead, real energy efficiency and software scalability have become more important.
04 Shifting from Selling Hardware to Selling Platforms
In the era of computing power integration, the competitive focus of chip IP and processor manufacturers has gradually shifted from competing for hardware unit performance to building complete platform capabilities. Hardware programmability, open source ecosystem adaptation, and upper-layer engine integration jointly determine the value of the product during the device lifecycle. The advantage of single hardware parameters is difficult to maintain for a long time. A unified and iterable software stack has become the core moat of integrated GPU solutions. This also means that the development toolchain, open source contributions and engine adaptation built around GPUs are replacing pure computing power figures, becoming an important dimension to measure the competitiveness of a chip enterprise. For downstream developers, familiar programming models and reusable toolchains directly determine whether a piece of hardware can quickly enter the mainstream ecosystem.
Jon Peddie, President and Founder of Jon Peddie Research, said: "The boundaries between traditional processor categories are gradually blurring. CPUs are using vector extensions to improve AI performance, and NPUs are becoming more general-purpose. With the continuous integration of functions, GPUs are most likely to stand out in this trend; after all, GPUs not only have parallel computing capabilities and a mature software ecosystem, but unlike other processors, they can process graphics tasks while running AI and computing workloads."
Looking into the future, with the continuous implementation of edge applications such as generative AI, digital humans, robots, and in-vehicle interaction, the mixed workload of graphics rendering and AI inference will become more and more common. Chip design enterprises will continue to weigh the trade-off between dedicated hardware and general-purpose programmable architecture. Integrated GPUs provide the industry with a feasible path to reduce system fragmentation and reuse computing resources. Subsequent industrial competition will also focus more on the perfection of the software ecosystem, cross-workload scheduling efficiency, model migration cost, and continuous iteration capabilities in different terminal scenarios.
This article is from the WeChat official account "Semiconductor Industry Insight" (ID: ICViews), written by Zi Hao, authorized to release by 36Kr.