HomeArticle

Surpass Google's ace model, the new decision-making king takes the global top spot, full-size open source

新智元2026-11-17 18:54
The open-source decision-making model Karpo outperforms Google's closed-source models across multiple benchmarks and takes the top spot.

The most fiercely competitive new track in Silicon Valley has just seen the rise of a brand new open-source king!

Recently, Anthropic, which is about to list, disclosed a huge computing power gap in its IPO materials, pouring a basin of cold water on the AI industry that has been frantically expanding model parameters.

While general large models excel in programming and mathematics, the computing power bill becomes a bottomless pit once they are actually put to work for humans.

Long chain reasoning consumes more Tokens than text generation, and browser operation is even more costly than reasoning, with the average single cost 4.8 times higher than complex reasoning!

Amid extreme anxiety, the "decision-making model" war in Silicon Valley has fully broken out.

The first-generation benchmark Jev pioneered the fast-thinking track of "no writing, only judgment", followed by giants including OpenAI, Microsoft, Amazon and Cloudflare who all entered the market one after another.

A couple of days ago, Sundar Pichai, CEO of Google, also personally stepped forward and grandly launched the closed-source decision-making model Gemini Decision 1.

Amid the fierce battle among giants, a dark horse open-source model Karpo Decision 31B has emerged strongly!

On the authoritative overall ranking of the Decision Index where 300 models compete on the same stage, it rushed to the top of the list and won the No. 1 in the world!

https://huggingface.co/spaces/multimodalart/jev-decision-index

This time, the full-size Karpo Decision family from 0.8B to 31B has been fully open-sourced!

Among them, the winning 31B flagship model has a median single-card latency as low as 94 milliseconds, a calibration error of only 0.023, and can evaluate up to 255 options simultaneously for a single question.

Head-to-head against Google's closed-source flagship, the open-source dark horse wins three consecutive games

Different from traditional large models that generate text word by word, Karpo Decision is built specifically for high-frequency decision-making of AI agents.

You only need to input the current state and question, and it can directly calculate the probability of all options through one forward inference, without generating any Tokens throughout the process.

With this ultra-fast judgment mechanism, Karpo reached the top of the overall ranking immediately after open-sourcing, and defeated Google's closed-source flagship Gemini Decision 1 head-on in multiple evaluations!

What is more dramatic is that this winning open-source dark horse is post-trained based on Google's own open-source base model Gemma 4 31B.

On the two currently recognized most authoritative independent lists, Decision Index and JevBench, Karpo has defeated all closed-source and open-source competitors.

Among them, Decision Index covers 37 public benchmarks and undisclosed private question banks, attracting 300 models from all over the world to compete on the same list.

JevBench is a full offline blind test run by an independent organization on its own graphics cards, with 1200 out of 1500 examination questions permanently sealed.

On the overall Decision Index ranking, Karpo took first place with a score of 73.9, far exceeding the 60.2 points of the first-generation benchmark Jev 1.14.

Not only is its score leading, its median latency is only 94 milliseconds, which is twice as fast as Jev 1.14's 196 milliseconds!

What really widens the gap is the blind test of private question banks designed to prevent list brushing.

Once the top 10 models on the public list enter the private blind test of the same skill, their scores will drop by an average of 6.8 points.

Only Karpo lost only 4.3 points, with the strongest anti-drop ability among the top 10, and even scored 78.6 points on private questions in completely new fields!

In the confrontation with Google's Gemini Decision 1, Karpo has achieved a series of impressive consecutive wins.

Although the two sides tied at 92.2% on the JevBench public set, the gap immediately became apparent once they entered the real Agent test field.

The first competition focuses on the core action judgment capability of Agent.

Take the authoritative benchmark When2Call as an example, which specifically tests whether the Agent should call a tool, ask follow-up questions or stop during the conversation.

In this test, Karpo scored 90.4%, 10.5 percentage points higher than Google's 79.9%.

In the Mind2Web multimodal test that requires clicking buttons on web page screenshots, Karpo also outperformed Google's 86.2% with a score of 90.0%.

The second competition moves to the field that tests subtext understanding and fine-grained classification.

In the New Yorker cartoon caption test that tests the ability to understand implied tone, Karpo reached 90.5%, leading Google's 79.7% by 10.8 percentage points!

In the Amazon ESCI e-commerce classification with extremely fragmented product intentions, Karpo also steadily outperformed Google's 58.7% with a score of 64.7%.

The third competition is a comprehensive assessment of 37 public benchmarks.

In the simultaneous actual test of 37 public benchmarks of Decision Index, Karpo scored 75.0 points, directly leading Google's 60.9 points by 14.1 points!

Breaking down into the three dimensions of unfamiliar domains (72.3%), knowledge accuracy (56.0%) and safety judgment (93.4%), Karpo also won across the board.

At the same time, it also left the top three open-source opponents on the original list, including KnowLine, GLiDE, and Sieve, far behind.

In the newly released 1500-question official offline blind test of JevBench, Karpo also delivered an amazing result.

In this strict exam independently run by the list maintainer, Karpo immediately obtained a current capability score of 89.53 points.

What is more rare is that it scored 85.67 points on 1200 sealed private questions, even surpassing the 80.45 points of public questions, and its calibration score even soared to 96.00 points!

After being converted to the unified scale of the list, among 261 open-source models, Karpo broke into the top three in the world with a converted capability score of 87.2 points.

Compared with Google, which participated in the test at the same time, all three core indicators of its total comprehensive score (77.4), decision-making intelligence (80.2) and calibration (94.1) are all superior!

At the same time, Karpo's cost per thousand decisions is only 0.025 US dollars, which means it only costs 1 US dollar to make 40,000 judgments.

While the 31B flagship reaches the top, MachinePulse has also applied the same training recipe to five small and medium-sized models from 0.8B to 12B.

Whether it is a lightweight end-side model or a single-card main model, every parameter size has entered the first echelon of its level!

Filter 36 events in 1 second, and complete the order for 10 people in the blink of an eye

Apart from winning the benchmark tests, how fast is Karpo in real scenarios?

When traditional large models encounter problems, they always generate long analyses Token by Token.

However, Karpo does not generate any text at all, and only runs one forward inference to calculate the probability of all options in milliseconds.

Karpo's official website has newly launched two real-time interactive arenas, which intuitively demonstrate the speed gap of "finishing dozens of questions in one forward pass".

In the "City Scanner" challenge, the AI needs to help users pick a weekend destination from 36 events across the city.

Assuming you make an appointment with friends to go out on Friday night, the settings are "tonight, 2 people, within 50 USD per capita, within 5 kilometers, and available for chatting".

The model needs to check against these requirements and judge each event as "match, eliminate, or insufficient information" one by one.

For this purpose, the system sends the same request to Karpo and the general large model (Gemini 3.5 Flash) at the same time, and they compete against each other with timers.

After clicking start, Karpo spent only 1.12 seconds to finish all 36 questions!

Not only is it fast, it also accurately picked out 10 matches, 23 eliminations, and 3 insufficient information entries, 100% consistent with the rules for all 36 questions.

However, the general large model on the right, which outputs words Token by Token, was still waiting with the timer running for more than 5 seconds.

The other "10 Appetites, 1 Menu" challenge pushes the difficulty of multi-condition trade-off to the limit.

There are 10 customers sitting in the restaurant at the same time, each with their own calorie limit, budget, allergens and taste preferences.

We ordered dishes manually on the left side, closely following the dietary restrictions and budget, and ordered 10 dishes in a row, but accidentally violated two dietary restrictions even after working for 5 seconds.

On the right side, Kar