Two PhDs work as instructors for AI, with an annual revenue of 670 million yuan and a valuation of 20.7 billion yuan.
Another AI business is booming.
On October 8, AI evaluation platform Arena announced the completion of a $200 million Series B financing (equivalent to approximately 13.4 billion RMB), jointly led by Lightspeed Venture Partners and Khosla Ventures, pushing its valuation to $3.1 billion (equivalent to approximately 20.7 billion RMB).
Back in January this year, Arena's valuation stood at only $1.7 billion. Its value has jumped by 80% in just nine months.
What is even more striking is its revenue growth speed. Arena disclosed in June this year that only eight months after the launch of its commercial services, its annualized revenue had exceeded $100 million (equivalent to approximately 670 million RMB).
The business of this company is very special: designing questions, administering exams and scoring for AI models including ChatGPT, Claude and others.
Similar stories keep emerging. In September, Stanford-affiliated startup Snorkel AI raised $350 million at a valuation of $3.5 billion. Domestic news also came out that Alibaba plans to lead a roughly $300 million financing round for AI evaluation company UniPat at a post-money valuation of $2.5 billion, with institutions including Tencent and HSG likely to participate.
AI evaluation, which originally belonged to university laboratories, has suddenly become a hot business in the capital market.
- 01 - Berkeley Lab spawns a $3.1 billion unicorn
The story of Arena dates back to the University of California, Berkeley.
In 2023, Jiang Weilin and Anastasios Angelopoulos, two PhD candidates at UC Berkeley, launched Chatbot Arena to solve a problem: which model is actually smarter among ChatGPT, Claude, Gemini and other similar models?
At that time, the large model industry was keen to show off its report cards. Every time a new model was released, manufacturers would present test results covering mathematics, programming, knowledge Q&A and other scenarios to prove that they were ahead of their peers.
However, there was a problem with these exams: a high score in the test does not mean that users find the model easy to use.
Therefore, the Berkeley team came up with an idea: let ordinary people act as examiners.
On the Chatbot Arena website, users input a question, and the system will randomly let two anonymous models answer it. Users do not know which answer comes from OpenAI and which comes from Anthropic, they only need to select the answer they think is better.
As the number of votes increased, the platform gradually formed a ranking list of AI models.
This gameplay quickly became popular in Silicon Valley. Whenever a large model company releases a new product, in addition to showing its own test results, it also hopes to rank high on the Arena leaderboard.
What was originally just a research project has gradually become an important evaluation platform in the AI industry.
In 2025, the team established the commercial company Arena Intelligence and began to accept venture capital. By January 2026, Arena completed a $150 million Series A financing at a valuation of $1.7 billion.
Only nine months later, Lightspeed Venture Partners and Khosla Ventures jointly led its $200 million Series B financing, with participation from Salesforce Ventures, Dell's investment arm, a16z and other institutions.
What does the capital see in this business? The answer lies in the user data accumulated by Arena.
As of June this year, Arena has more than 10 million monthly visiting users, generating a total of over 700 million conversations and 82 million manual votes.
These users not only help Arena design questions for free, but also grade the answers.
For example, when asking AI to write a business plan, why do users prefer Claude over GPT? Is it because of clearer structure, more accurate information, or content that better meets actual needs?
This information is very valuable for AI manufacturers.
In the past, large model companies relied on internal engineers to design tests. Now, Arena has obtained a large number of real questions and people's judgments on different answers through tens of millions of users.
More importantly, these feedbacks are generated continuously.
In September 2025, Arena officially launched its paid product AI Evaluations. The public leaderboard remains free, but if model companies want to know why they lost, which tasks they performed poorly on, and how to improve, they can purchase more detailed evaluation services.
The free leaderboard is responsible for attracting users, and the underlying evaluation data and analysis services generate revenue.
This business model was quickly proven viable.
In January this year, Arena's annualized revenue was about $30 million; by June, the figure had exceeded $100 million. It should be noted that the annualized revenue here is converted based on the business scale at that time, not the full-year operating revenue that has been actually realized.
Even so, it only took eight months from the launch of paid services to exceeding the $100 million annualized revenue threshold.
Nowadays, Arena is no longer satisfied with only ranking large models.
On October 8, along with the new round of financing, it released the Alignment Index, which starts to detect whether AI agents will perform operations without permission, misinterpret user intentions, and lie about unfinished tasks as completed.
In a study covering 27 models and about 90,000 real Agent sessions, Arena found that even advanced AI will have problems such as deleting files without authorization, claiming success without completing inspections, and other similar flaws.
From judging whether AI can chat to judging whether AI can work reliably, Arena is expanding the scope of its business.
- 02 - VCs are scrambling for "AI examiners"
Arena is not an isolated case.
On September 22, another Silicon Valley AI company Snorkel AI announced the completion of a $350 million Series E financing at a valuation of $3.5 billion, nearly three times higher than the $1.3 billion valuation in May last year.
This round is jointly led by Insight Partners and S32, with participation from well-known institutions including Greylock, Lightspeed and GV.
Snorkel was also born out of the Stanford University research system. Founded in 2019, it mainly helped enterprises automatically process training data in the early stage.
But it soon found that the needs of AI companies had changed.
In the past, model training required a large amount of text, images and manually labeled information, and these tasks could be handed over to outsourcing teams.
Nowadays, AI is getting smarter, and simple data is no longer sufficient.
For example, if an AI company wants its model to learn to analyze M&A transactions, it cannot only feed thousands of contracts to the model. It also needs professionals to design complex cases to judge whether the model has identified key risks.
This requires the participation of professionals such as lawyers and financial analysts.
Snorkel began to expand its business from selling data processing software to providing high-quality training data, professional tasks and evaluation environments.
The result is a sharp increase in revenue.
According to the company's disclosure, after the launch of new data services in September 2025, its business grew rapidly. By September this year, its annualized revenue had exceeded $375 million, a surge of more than 17 times compared with a year ago.
A company that provides data and exam materials for AI has already achieved hundreds of millions of dollars in annualized revenue.
Chinese capital has also begun to pay attention to this track.
At the end of September, according to 36Kr reports, Alibaba is considering investing in UniPat and plans to lead a roughly $300 million financing round, with a discussed post-money valuation of about $2.5 billion. Institutions including Tencent and HSG also intend to participate.
Founded at the end of 2025, UniPat focuses on AI model training, evaluation and real task data business.
The services it provides can be understood as helping AI enterprises build a set of examination systems.
When a large model company develops an Agent, it originally needs its own test team to design tasks, and then engineers to check whether the AI has completed the work. UniPat tries to productize this process.
Zhu Jun (pseudonym), a person in charge of the AI agent project at a large tech company, told Pencil News that in the past, when developing Agents, the test team needed to design tasks by themselves and then judge to what extent the tasks were completed. UniPat has packaged part of the work originally undertaken by the test team into products.
There is a clear economic logic behind this.
Assuming that an enterprise is preparing to launch an AI programming tool, building its own evaluation team not only requires hiring engineers, but also developing a test environment, designing questions, and verifying answers.
If a third party can do these tasks faster and cheaper, enterprises have reasons to pay for the service.
Moreover, this business has another feature: the continuous upgrading of AI models means that the exams need to be repeated constantly.
A new version of GPT is released today, Claude will launch a new model next month, and new products may appear in a few weeks. Every time there is an upgrade, manufacturers need to test new capabilities and check whether the old functions are broken.
In addition to Arena, Snorkel and UniPat, more companies have already made money from related businesses.
One of the representative players is Mercor.
This US startup founded in 2023 specializes in helping AI enterprises recruit professionals such as programmers, doctors, lawyers and financial experts to design tasks for large models, review answers and provide training feedback.
According to data from Sacra, Mercor's total annualized revenue reached approximately $2 billion in June this year.
Another company, Surge AI, has already proven that this industry can be profitable.
British media previously reported that Surge AI achieved revenue of more than $1 billion in 2024 and has been profitable. It mainly provides high-quality training data and human feedback, not just model evaluation services.
The businesses of these companies are different, but they have one thing in common: they did not invest huge sums of money to train general large models like OpenAI, but help large models become better and more reliable.
Some design exam questions, some grade answers, some recruit experts, and some convert errors into new training data.
The more complex AI becomes, the more professional services it needs.
This reminds people of the gold rush: some people dig for gold, while others sell shovels. Now, AI model companies are competing for the top spot, while a group of companies providing evaluation, training and data services have taken the lead in finding profitable opportunities.
- 03 - AI examiners are making money, who is paying the bill?
Companies that grade papers for AI are getting more and more valuable, but Zhang Yizhe, founding partner of Yonggui Fund, raised a key question: who is paying for these services?
He believes that the development of AI is similar to that of the Internet and mobile Internet. Every link in the industrial chain has its own value, and evaluation can also find its own commercial position. But how big this business can grow still depends on customer demand.
So far, there are mainly two types of payers.
The first category is large model companies.
In the past few years, companies including OpenAI, Anthropic and Google have invested huge amounts of capital in training models. Every time they release new products, they want to prove that their products are better than their competitors'.
But by 2026, it is no longer enough to just prove that the model is good at mathematics and programming. Especially after the rise of Agents, AI begins to perform tasks on behalf of people, rather than just answering questions.
Assuming a financial company asks AI to analyze five listed company financial reports.
AI may generate a beautiful report with complete figures and fluent language in only a few minutes. But as long as it omits a major liability or mixes the financial calibers of different companies, the report may become unusable. Worse still, users may not even know where the AI made mistakes.
Similarly, an AI programming tool may generate tens of thousands of lines of code a day, but introduce new bugs when modifying a function.
A high score in the model exam does not mean that it can actually work for people.
In 2025, OpenAI launched the GDPval evaluation, covering 9 industries and 44 occupations, with a total of 1320 professional tasks designed. The experts who designed and reviewed the questions have an average of more than 14 years of work experience.
This type of evaluation begins to require AI to deliver real work results, rather than simply answering multiple-choice questions.
It means that the cost of exams has changed.
In the past, a math problem only needed a correct answer; now, a complex contract or a set of financial analysis results may require experts with many years of work experience to judge.
It costs money to design these tests, to judge the results, and to improve the model after finding errors.
The second category of payers is enterprises that plan to purchase AI services.
This part of the market may be more worthy of attention.
A bank using AI customer service needs to know whether it will leak customer information; a software company purchasing AI programming tools needs to know how many real tasks it can complete; a manufacturing enterprise introducing agents first needs to confirm that it will not perform dangerous operations without authorization.
What enterprises really care about is success rate, stability and cost.
A study released by Arena in September this year tested 21 combinations of models and programming tool environments. In some test tasks, the same model can achieve similar task success rates, but the highest difference in usage cost is about 5 times.
This means that if an enterprise chooses the wrong model or tool, even if it can complete the same work, it may pay several times the cost.
If evaluation companies can help customers save this cost, they will have clear commercial value.
However, behind the capital boom, the challenges are also obvious.
The first is credibility.
In 2025, a paper titled *The Leaderboard Illusion* questioned the ranking mechanism of Arena.
Researchers found that before the official release of Llama 4, Meta had tested as many as 27 undisclosed model versions. The researchers believe that if manufacturers can test repeatedly and then select the best version for public release, the leaderboard may have selection bias.
Arena disputed some claims in the paper, and responded that it would further clarify the multi-version test rules, label models that withdrew from the test, and add provisional labels to some results.
This reveals a subtle commercial contradiction.
Arena has to grade model companies, and also charge fees from model companies.
The commercial relationship between the examiner and the examinee will inevitably make people worry about the fairness of the exam.
In addition, there is the problem of customer data.
After Meta invested in Scale AI in 2025, customers including Google and OpenAI were reported to plan to reduce or stop cooperation, worrying that their own data and R&D directions may be exposed to competitors.
The same risk may also appear in the AI evaluation industry.
When a company hands over its unreleased new model to a third party for testing, it is equivalent to handing over its product capabilities, technical weaknesses and R&D priorities to an external institution.
Who can access this information and how to ensure confidentiality will become important conditions for evaluation service providers to win large customers.
The last point is profit.
If an evaluation company needs to recruit new experts, design exam questions and build a new environment for every new customer it takes on, its labor costs will rise rapidly along with the growth of revenue.
On the contrary, if it can precipitate test tasks, expert experience and automated tools into standardized products, so that the same set of evaluation capabilities can serve hundreds or thousands of enterprises, it will have a better chance to obtain the profit margin of software companies.
Therefore, although all of these businesses look like AI evaluation businesses, the business models of different companies may vary greatly.
Some are more like software companies, some are more like data service providers, and some are closer to high-end talent outsourcing platforms.
For investors, valuation should not only look at the revenue growth rate, but also look at how much profit these revenues can finally retain.
AI examiners have obtained huge financing, and now it is their turn to take the exam of the market.
This article does not constitute any investment advice.