Qualcomm is going all out for edge-side AI by integrating PC chips into mobile phones. | Zhiyong Analysis
The King of Mobile Phone Chips Finally Plays Its Card at the End of 2024.
During the 2024 Snapdragon Summit, Qualcomm officially launched the Snapdragon 8 Extreme Edition mobile platform (Snapdragon X Elite), as well as two new automotive chip products: the Snapdragon Cockpit Extreme Edition platform and the Snapdragon Ride Extreme Edition platform.
From the name, one can see Qualcomm's ambition. The latest generation of chip X Elite does not continue to use the name of the Gen series (8 Gen 4) as usual, but has changed its name again - using the name "Elite" (Extreme Edition) of Qualcomm's own laptop processor series.
The Snapdragon 8 Elite adopts TSMC's 3nm process and is the most powerful and fastest mobile system-on-chip in the world for Qualcomm to date. Compared to the previous generation, the CPU's single-core and multi-core performance have both increased by 45%.
Source of Qualcomm Snapdragon 8 Elite Chip: Taken by the Author
Qualcomm has also introduced the Hexagon NPU, the Adreno GPU with a new slicing architecture, and the Spectra ISP architecture to this chip. For example, the Hexagon NPU can improve AI performance by 45%.
At the launch event, Qualcomm did not shy away from showing the comparison results with its competitors Intel and AMD.
Source of Qualcomm CEO Cristiano Amon: Taken by the Author
Qualcomm even made a big move: it showed the performance comparison results of the Qualcomm 8 Elite and its competitors without being plugged in. In the end, the single-core performance of the Snapdragon 8 Elite did not decrease significantly, while the chips of Intel and AMD decreased by 45% and 30% respectively when not plugged in.
"I suggest that you should achieve the performance you have promised," CEO Cristiano Amon commented.
The sharp comments on competitors are a side of the fierce competition in the chip industry nowadays.
2024 is called the year of AI mobile phones. Almost all mainstream mobile phone manufacturers have launched their own AI mobile phones, and chip manufacturers are also competing for these orders. As a chip giant, Qualcomm certainly does not want to miss this historical opportunity.
No More Holding Back, Aiming at Edge-side AI
If only one keyword could be chosen for the 2024 Snapdragon Summit, it must be AI.
In the three-day Snapdragon Summit, almost no speech in the AI roundtables on the first and third days deviated from the theme of AI. The opening ceremony even invited several of the hottest big names in the current AI field - Microsoft CEO Satya Nadella, Meta CEO Mark Zuckerberg, and OpenAI's Sam Altman - to talk about how AI is changing our daily lives.
Source of OpenAI CEO Sam Altman: Taken by the Author
More importantly, Qualcomm also took the opportunity to show its ecological position in the AI circle. In addition to these several, Qualcomm has also reached cooperation with domestic mainstream large model manufacturers, such as Zhipu GLM and Tencent Hunyuan.
In 2024, almost all mobile phone manufacturers have released their own large model bases, and chip manufacturers have also accelerated the competition for these orders in order to stimulate the growth of a new generation of hardware shipments with AI. They have introduced new chip architectures, such as increasing the R & D proportion of AI chip architectures such as the NPU.
Because of this, Qualcomm will not "hold back" on this new generation of products, but has maximized its performance.
Earlier this year, when the Snapdragon X series commercial terminals were launched, they were equipped with an NPU with a computing power more than 4 times higher than other products on the market.
And the Snapdragon 8 Elite has directly moved the second-generation Oryon CPU on the PC to the mobile phone.
Source of Qualcomm Shows AI Hub Ecology: Taken by the Author
This move is quite historically significant, meaning that Qualcomm has abandoned the public version of the ARM architecture and completed the last piece of the puzzle of its self-developed SoC.
"The CPU is crucial for managing system-level processing. It can execute instructions from applications, operating systems, and user inputs to ensure smooth and fast operation. It can also run complex applications such as high-end games, video editing software, AR experiences, AI and machine learning, and security," said Alex Katouzian, General Manager of Qualcomm's Mobile, Computing and XR Business Group.
Compared to the previous generation Snapdragon 8 Gen 3, the Snapdragon 8 Elite has a cross-generational improvement in performance. For example, the performance improvement of the Oryon CPU on the Chrome browser is more than 62%; its energy efficiency has also increased by 44%.
It is worth noting that the 8 Elite adopts a 2 + 6 core architecture, that is, two super cores for intensive applications that require a faster response speed; and six performance cores, each of which is tuned to run the most intensive applications.
Qualcomm has even equipped these two levels of cores with a 12M cache, which many people comment is a configuration that can compete with Apple's A18 Pro.
The configuration of the Snapdragon 8 Elite is similar to the "all big core" architecture of the first-generation Oryon CPU, which shows Qualcomm's sincerity. The main frequency of the 8 Elite is as high as 4.32GHz. To put it simply: even if such performance is placed on a PC, it is not inferior.
It can be seen that Qualcomm is trying to blur the boundary between PC and mobile phone devices with an aggressive strategy of stacking parameters to serve more AI needs.
In the past, the large models that could be run on mobile phones, PCs and other terminals were limited by chip performance, and their parameters remained within 10B (10 billion). Compared with products based on 100-billion-level large models such as ChatGPT, the AI capabilities on the edge side are still quite limited.
The innovation in architecture and the collaborative work of multiple cores, including the performance improvement of the Hexagon NPU, have opened up the possibility of more scenarios on the edge side. AI video editing, AI dynamic shooting, and AI assistants can all be truly implemented due to the improvement of chip performance.
Take video editing as an example. Previously, the AI model on the edge side could only process images, such as erasing passers-by and objects in the background.
But on the 8 Elite, a rather remarkable progress is that - AI can already edit videos in real time, and the entire process is calculated locally without the need to go to the cloud. This puts extremely high requirements on the performance of the chip and the collaboration of the model on the mobile phone side.
Source of Comparison of Qualcomm Snapdragon 8 Gen 3 and Qualcomm Snapdragon 8 Gen Elite: Taken by the Author
In the photography scene, smart phones can use autofocus to track the eyes of pets, so that no matter how fast they move, the photos can be clearly focused.
And users can also observe their pets through their phones and press the shutter at the moment the pet looks at the camera. Even if the reaction is not fast enough, the lens can clearly capture the pet in fast motion.
As usual, Qualcomm's summit is also a stage for various manufacturers to "show their first releases". This year, many mobile phone manufacturers including Xiaomi, Honor, Oppo and others appeared on the scene to show their cooperation with Qualcomm. Among them, Xiaomi's upcoming 15 series will be the first to launch the Qualcomm 8 Elite. In addition, terminal manufacturers such as Honor, ZTE, and vivo will successively launch products equipped with the Snapdragon 8 Elite chip.
However, the market response was not as enthusiastic as expected. After the new product was released, Qualcomm's stock price briefly rose to $172, but due to the impact of ARM's patent lawsuit news, as of the close of October 23, Qualcomm's stock price fell by 6% to $166.6.
"AI Brings More Than Just a Killer Application"
In the opening speech of this summit, CEO Amon also put forward a distinct viewpoint that has been discussed for a long time: "AI brings more than just a killer application, but rather, it will change all APPs.
"People always ask me, what is the killer application on AI? What is the killer AI application on mobile phones? This is a way of thinking, but I don't think it is the only way of thinking," Amon said. He believes that the moment when AI changes all Apps will definitely come, "but I'm just not sure when."
The historical opportunity brought by AI is definitely not as simple as adding a new AI chip product. As a mobile phone chip giant, Qualcomm needs greater ambition and dreams.
"Within the next five years, we may all have an artificial intelligence smart phone," Qualcomm CEO Amon said in an interview with Bloomberg Television on Tuesday. He compared the situation to when smart phones first appeared, indicating that a similar situation has occurred in the generative AI era: waiting for application scenarios to emerge.
However, manufacturers' efforts to launch mobile phones and PCs with AI have not led to the expected sharp increase in sales.
According to IDC, in 2022 and 2023, the global shipments of smart phones, computers and tablets have declined year-on-year - hardware manufacturers in the upstream and downstream of the industrial chain are under great pressure to ship products.
Although Qualcomm is still the king in the mobile phone market, especially in the high-end mobile phone chip market, with its mobile phone business accounting for more than 60%, its competitors are pressing hard.
According to Statista, as early as the third quarter of 2020, Qualcomm's market share began to be lower than that of MediaTek. By the fourth quarter of 2022, Qualcomm's market shrank to 19%. It was not until the second quarter that Qualcomm's share in the global mobile phone market recovered to 31%, but there is still a slight gap with MediaTek's 32%.
Source: Statista
In the AI battlefield, in the face of Nvidia, which is far ahead, and the rapidly advancing Apple M series chips, the chip industry is experiencing a new round of reshuffling. Recently, Qualcomm has been reported by the media to want to "merge small with big" and acquire the veteran chip giant Intel, which are all strategies to deal with these impacts.
In the software layer, Qualcomm quite emphasizes the ability of AI Agents. CEO Amon enthusiastically showed the demo of the AI intelligent assistant built by Qualcomm to demonstrate its own AI capabilities.
Qualcomm's AI assistant can bypass the App interaction interface and directly call the application api to perform tasks. For example, users do not need to learn how to use a complex bank App, but can directly interact with the AI Agent - tell the AI assistant in text or voice to perform tasks such as transferring money, and the interaction has almost no delay.
Source of CEO Amon Shows the Bank App AI Agent: Taken by the Author
However, Qualcomm emphasizes that as a chip manufacturer, they will still choose to "stay behind the scenes" rather than replace Apps. "This is also the case with the technology demonstration we are doing today. We did not delete all the applications, but just connected the applications through plugins to let the applications work behind the scenes," Durga Malladi, Senior Vice President of Qualcomm and General Manager of Technology Planning and Edge Solutions Business, told "Intelligent Emergence".
This also benefits from the improved processing efficiency of the edge-side chip for the model. Token throughput, which can be understood as the text input and input unit of the model, is an important indicator to measure the performance of the model.
In terms of this indicator, compared to the previous generation, the Token generation rate of the 8 Elite has increased by up to 100%. The previous 8 Gen 3 only supported a context length of about 1k, while the 8 Elite has reached 4k.
"It's like changing from snail mail to instant messaging," Siddhika Nevrekar, Senior Director of Product Management at Qualcomm, described.
But the future that Qualcomm envisions is actually quite distant.
A reality that must be faced squarely is that most AI applications are still in the fog nowadays. The technological evolution of large models is slowing down, and there are many discussions that generative AI is difficult to prove that it can overthrow the existing "super applications", but is more absorbed by the existing super applications.
Large factories with "super applications", such as search and short videos, make it even more difficult for AI to get the opportunity to "revolutionize itself". For example, the huge advertising revenue in traditional search engines will certainly not allow AI search to directly become a new interactive entry.
Change is bound to be slow. Before the AI applications and OS manufacturers cooperating with Qualcomm can prove that generative AI can have sufficient commercial value, Qualcomm is expected to go through a longer investment period before seeing returns.
In addition to AI, Qualcomm also focuses heavily on the automotive business. The 2024 Snapdragon Summit also released two new automotive chip products: the Snapdragon Cockpit Extreme Edition platform and the Snapdragon Ride Extreme Edition platform.