Surprisingly, the GPU utilization rate of 79% of enterprises is less than half.

The AI arms race has reached a point where the situation seems to have turned surreal...

Recently, continuous survey data for Q3 2026 collected by VentureBeat Intelligence, the research arm of tech media outlet VentureBeat, shows that 79% of enterprise respondents said their GPU computing power is completely idle for at least half of the time.

This figure has remained steadily at a high level between 77% and 81% over the past three months. In other words, underutilization of GPUs has become a persistent phenomenon among the surveyed enterprises.

But what is even more bizarre is that among these enterprises where the average GPU utilization rate does not exceed 50%, 70% still plan to introduce, expand or replace new AI computing platforms in the next 12 months.

VentureBeat Intelligence, the research arm of VentureBeat, aggregated the results of three rounds of VB Pulse surveys on artificial intelligence infrastructure and computing power conducted in July, August and September 2026. The three rounds of surveys attracted a total of 385 qualified respondents from organizations with no less than 100 employees, including consultants who answered on behalf of their major clients.

On one hand, the hardware is gathering dust in the computer room, and on the other hand, the AI budget bill is still accelerating upward, and the question has emerged: with existing resources not yet fully utilized, why do enterprises continue to look for new AI platforms?

Looking at the return on investment, VentureBeat's survey in September this year shows that among 64 respondents who plan to adopt new AI platforms, 58 do not have return on investment (ROI) data reviewed by the finance department, accounting for as high as 91%.

This means that a considerable number of surveyed enterprises have not formed a financially reviewed revenue measurement when preparing for the next round of AI investment.

Of course, an unreviewed ROI does not mean the project has no revenue. In the same survey, respondents who had no plans to adopt new platforms also generally lacked such data, but it at least indicates that the financial assessment of enterprise AI investment may not have kept up with the speed of infrastructure investment. That is to say, AI capabilities are rapidly entering production environments, but how much has been spent and how much value has been created are still questions that many enterprises have not fully answered.

These figures are important because Wall Street is fiercely debating whether AI has formed a bubble, and VentureBeat has put forward a controversial judgment based on the survey: The risk of the AI bubble may have already appeared inside enterprises...

Misaligned Two Sets of Bills: Idle "Computing Power" and Money-burning "APIs"

On the surface, this phenomenon of "idling on one side and increasing investment on the other" is extremely contrary to business common sense, especially at a time when most technology leaders are facing pressure to cut AI budgets: some technology leaders told VentureBeat that during this fall's budget season, they are under enormous pressure to reduce AI bills.

But a deep dive into the internal architecture of enterprises reveals that the real AI demand of enterprises has not disappeared, but has precisely fallen into areas that existing hardware cannot support at all.

Professionals and consumers are using more and more AI, and a large number of these tasks rely on cutting-edge models provided by Anthropic and OpenAI, and this demand will continue to support these companies' investment in computing power for some time to come.

The real mismatch occurs further downstream inside enterprises: enterprises are currently paying for two completely different types of AI expenditures at the same time, and there is an insurmountable gap between the two:

Panic-driven computing power reserve: In the past two years, amid the anxiety of global computing power shortage, a large number of enterprises pre-locked and purchased GPU clusters out of the psychology of "avoiding missing the trend". However, after the hardware is in place, enterprises lack mature internal scenarios and engineering capabilities to run it at full capacity.

Path dependence on closed-source cutting-edge models: Business teams and engineers prefer to directly call the top closed-source cutting-edge models under OpenAI, Anthropic and Google. Enterprises pay per Token through the APIs of these companies, or purchase model capabilities through monthly subscriptions and large cloud service providers, and cannot deploy them directly on their previously controlled local or hosted GPUs.

This has created a highly ironic situation: self-built computing power has become a sunk asset, while the cost of calling closed-source APIs is eating into the budget at an alarming rate.

Back in August this year, Fortune magazine reported that Uber's chief technology officer revealed that since the company encouraged employees to use AI tools as much as possible, it even set up a "leaderboard" to rank software engineers based on their usage of these tools.

The result is that the full-year 2026 AI budget was exhausted within a few months, forcing enterprises to announce the end of the "Tokenmaxxing" era.

Survey data further corroborates this structural fragmentation: among enterprises that plan to purchase new computing platforms and have idle GPUs, 90% are turning to public cloud and model API service providers, and only 8% are considering expanding local private clusters. At the same time, 91% of enterprises planning to introduce new platforms cannot come up with any actual ROI assessment reviewed by the finance department at all.

Of course, part of the idle computing power may be intentional on the part of enterprises, such as reserving GPU resources to cope with peak demand, and the new platforms planned to be introduced may also be used to replace old platforms.

From "Computing Power Arms Race" to "Inference Economics"

If GPU idleness exposes the problem of infrastructure utilization efficiency, then the popularization of Agents may make enterprises face more complex cost challenges.

VentureBeat predicts that the proportion of enterprise AI expenditure used for model calls will become higher and higher in the future. Especially as businesses move from simple generative conversations to multi-step autonomous AI Agents, the computing power contradiction of enterprises is being further amplified.

Gartner predicts that by 2028, the inference cost of a single Agent workflow will rise by more than 5 times.

This does not mean that the unit Token price of the model will become more and more expensive. In fact, with the continuous optimization of inference infrastructure and model efficiency, the unit Token price may continue to decline, but the real problem is that the number of Tokens consumed by Agents to complete tasks may grow faster than the decline in unit price.

One easily overlooked source of cost is context. Taking AI programming as an example, the model not only needs to understand the current problem, but also continuously read project code, system instructions and historical execution records. As the task progresses, these contexts may appear repeatedly in multiple requests. If they are reprocessed every time, the cost will accumulate rapidly, which is why context caching is getting more and more attention.

NVIDIA engineers mentioned in a technical guide that in the Claude Code session scenarios they studied, about 85% to 97% of the context of subsequent requests after the first call can be reused through caching, and Anthropic's cache read price can be as low as 1/40 to 1/10 of the price of ordinary input Tokens, depending on the model and pricing conditions.

For Agents that need to perform tasks continuously, this optimization can bring considerable savings.

Another variable is the number of Agents. A programming task test by industry evaluation agency Vals AI found that the solution of using a master Agent to schedule multiple sub-Agents costs 1.8 to 5.1 times that of a single Agent.

However, among the four test configurations, only one achieved a statistically significant performance improvement. In other words, adding Agents may bring stronger capabilities, or it may just make more models consume Tokens at the same time.

Say Goodbye to Extensive Expansion: Pragmatic Self-rescue Path for Enterprises

Faced with an extremely high idle rate and out-of-control model call bills, for enterprises, perhaps the extensive "blind procurement" is becoming a thing of the past. The industry generally believes that before enterprises sign the next computing power contract, they should prioritize squeezing efficiency out of existing assets: find ways to use the already paid-for computing power and Tokens to complete more actual work.

Specifically, the following things need to be done well:

Measure the cost of each completed task, taking into account both GPU computing power resources and API call expenditures;

Test different models and inference intensities to determine the enterprise's default configuration: the evaluation criteria should be the cost of each completed task, not the price of a single Token. Because a cheaper model may consume more Tokens, resulting in a higher cost per task in the end;

Only provide the Agent with the context it actually needs, and cache reusable content: this should be done whether through the model vendor's API or running on the enterprise's own infrastructure;

Add a second Agent only when the task can be split into parts that can be executed in parallel;

Migrate regular, high-frequency tasks from cutting-edge model APIs to open models running on idle computing power, such as classification, information extraction and text summarization;

Use the remaining GPU idle time to process tasks that can be postponed, such as batch inference, model evaluation and fine-tuning.

The ultimate goal is to make enterprises pay a more reasonable cost for every actual work completed.

This also gives the surprising set of data at the beginning a new meaning. The low GPU utilization rate of 79% and the 70% willingness to adopt new platforms cannot directly prove that the AI bubble has formed, let alone indicate that the entire AI industry is moving towards overcapacity. But it reminds enterprises: there does exist a gap between resource input and actual demand that needs to be re-examined.

In the past few years, the AI industry has been accustomed to measuring progress by parameter scale, training computing power and capital expenditure. As technology gradually enters real businesses, beyond these indicators, a more pragmatic evaluation system is needed: how much work has enterprises actually completed with AI? What is the success rate? How much time has been saved? And how much cost has been paid?

For model vendors, this means that competition will not only revolve around model capabilities, but also extend to inference efficiency, deployment flexibility and overall task cost. For enterprises, what is more important than continuing to hoard computing power may be to find ways to make good use of the GPUs and Tokens they already own first.

After all, GPUs are bought to do work, and what is really worth paying attention to is how much value they ultimately create...

References:

https://venturebeat.com/infrastructure/ai-bubble-update-79-of-companies-say-their-gpus-sit-idle-at-least-half-the-time-yet-most-plan-new-ai-platforms

https://fortune.com/2026/08/07/uber-ai-spending-tokenmaxxing-is-over-cto/

This article is from the WeChat Official Account "Synced" (ID: almosthuman2014), written by AI-focused contributors, and published with authorization by 36Kr.