Token eats 30% of the salary, Silicon Valley's AI bills are out of control

It only costs $0.99 per million tokens.

This is the real - world cost on the own bill of SemiAnalysis, the most in - depth semiconductor research institution in Silicon Valley.

But what's even more astonishing is this figure: Internal large - model token expenses already account for 30% of the total employee salaries.

It may sound like a lot. But if you think about it the other way around, the output bought with this money used to require several times the labor cost in the past. Each person consumes nearly 5 billion tokens per month on average, more than five times the per - capita level at Meta. Core contributors consume over 100 billion tokens per month.

What used to take junior analysts several hours to complete, such as converting Excel models and creating financial report charts, can now be done within minutes at a cost of just a few dollars.

SemiAnalysis' own evaluation hits the nail on the head: This isn't just a 10% increase in efficiency; the unit economics of professional services are being rewritten.

Research firms, hedge funds, law firms - For all industries that rely on human brains, it's just a matter of time before token expenses account for 20% - 30% of salaries.

NVIDIA CEO Jensen Huang is more anxious than anyone else.

At this year's GTC conference, he directly stated: For an engineer with an annual salary of $500,000, if their token consumption by the end of the year is less than $250,000?

“I'd go completely crazy.”

He plans to give each NVIDIA engineer a token budget equivalent to half a year's salary and have 750,000 AI agents work alongside 75,000 employees.

Not using AI? Jensen Huang says it's no different from a chip designer insisting on using paper and pencil.

Tokens are no longer just tools; they're becoming the “means of production” in the new era.

But the other half of Silicon Valley is going crazy over AI bills.

Interestingly, while SemiAnalysis is saving real money with tokens, Silicon Valley giants are struggling with AI bills.

Uber is a classic example.

At the end of last year, the company promoted Claude Code to 5,000 engineers and even created a leaderboard - the more you use it, the higher your ranking, intensifying internal competition.

The result was too successful: In February, the engineer usage rate was 32%. In March, it soared to 84%. By April, 95% of engineers were using AI every month, and 70% of the submitted code was AI - generated. And the annual budget - was already spent.

The CTO said, “We have to redo the budget from scratch.” Later, it got even stricter - Bloomberg reported that Uber set a monthly token limit of $1,500 for each employee, requiring special approval if exceeded.

But COO Andrew Macdonald told the truth in a podcast: The usage of AI is indeed increasing, but the connection between it and consumer - facing feature innovation... is not yet visible.

Microsoft's situation is even more surreal. Last month, The Verge reported that Microsoft is canceling most of its Claude Code licenses and switching to its own GitHub Copilot CLI.

The reason is simple: The rate of spending money is faster than the rate of generating output.

NVIDIA's vice - president of applied deep learning, Bryan Catanzaro, put it more directly in April this year: “For my team, the computing cost far exceeds the employee cost.”

A 2024 MIT study shows that in positions mainly focused on visual work, AI automation is only economically viable in 23% of scenarios.

In the remaining 77% of cases, hiring human workers is cheaper than using AI.

Some engineers even complained that AI agents “ruined his database and network” during use - what he called the cost of “overuse.”

With sky - high budgets, out - of - control usage, and constant setbacks, Silicon Valley is in the most divided stage of AI economics.

On one hand, technology brings unprecedented productivity. On the other hand, the bills are swelling at an equally unprecedented rate.

The cost collapse is just beginning

But the core argument of SemiAnalysis is: Don't focus on today's prices; the cost collapse has just started.

First, look at the software side.

Running DeepSeek R1 on B300, through three - layer pure - software optimization of wideEP, disagg, and MTP, the single - GPU throughput can soar from the baseline of 1,000 tokens/second to 14,000 tokens/second - a 14 - fold increase, purely through code.

Then, look at the hardware side.

The throughput of the optimally configured GB300 NVL72 is 17 times that of the H100, and it can reach 32 times when switched to FP4 precision.

Opus 4.7 is priced at $5 per million for input and $25 per million for output, which doesn't seem cheap.

However, due to the input - to - output ratio of the agent workload being as high as 300:1 and a cache hit rate of over 90%, the actual mixed cost is pushed down to $0.99.

It's less than one - fifth of the listed price.

Combining software and hardware, one conclusion is hard to avoid: The gross - margin expansion of large models is not a one - time pricing coincidence but a structural trend.

Anthropic's ARR this year has soared from $9 billion to over $44 billion, and its gross margin has jumped from 38% to over 70% - Tokens have become cheaper, but those selling tokens are making more money.

A Gartner report in March this year confirms this: By 2030, the inference cost of trillion - parameter large models will decrease by over 90% compared to 2025.

SemiAnalysis' judgment is clear: If you want to predict the token price in 2027, the answer is a single word - down.

Spent the money, then what?

This is exactly the most divided aspect of AI at present: Global technology companies have announced a total of $740 billion in AI capital expenditures this year, a staggering 69% increase from last year. At the same time, the rate of layoffs in the technology industry has exceeded that of the entire last year.

Money is being burned wildly, and people are being laid off. But the chief economist at Goldman Sachs told the truth - The actual impact of AI on the economy has been basically zero so far.

This isn't because AI doesn't work; it's the growing pain that every round of infrastructure revolution has to go through: First, burn money to build the pipeline, then wait for the water to flow.

This was the case with the power grid, the Internet, and AI is no exception.

The only difference is that this time, the speed of pipeline construction and the speed of water flow are both on a scale that the previous generation has never seen.

SemiAnalysis is already on the side where the water is flowing - 30% of the salary has brought several times the output leverage, and the cost curve is still dropping sharply.

As for other companies: Do they wade through the water now, or do they wait until the people on the other side have built a city and then catch up?

Reference materials:

https://x.com/SemiAnalysis_/status/2070915305858007345

This article is from the WeChat official account “New Intelligence Yuan”, author: ASI Apocalypse, editor: Solomon. It is published by 36Kr with authorization.