The cache hit rate reaches 99.93%, the most suitable Harness for DeepSeek is now available, having amassed 86,000 Stars on GitHub.

Great, great.

While the official DeepSeek version is still on the way, the folk "Best Harness" has already come out ahead of the curve! (It's so competitive that everyone is pushed to the limit.)

The project is named "Pi", an open-source programming Agent that has amassed roughly 86,000 Stars on GitHub.

The project is so popular mainly because it is extremely cost-efficient.

Take the developer Evan Kim for example. After he integrated DeepSeek into Pi, approximately 99.93% of the input tokens successfully hit the cache.

In other words, the cache miss rate is only 0.07%, which means most repeated contexts do not need to be recalculated???

Even the Composio team recently conducted a horizontal test of 8 mainstream Agent Harnesses, directly loading DeepSeek V4 Flash into them to run real-world tasks.

The result is a little dramatic —

Pi has the lowest average cost for completing a successful task, which only costs about 0.028 US dollars.

Claude Code: How come my price is 7 times higher than Pi's!!!

DeepSeek: My tokens are already sold at such a low price.

Pi and other cost-saving projects: It's okay, we can help users spend even less.

4 Tools Paired with DeepSeek, Cache Miss Rate as Low as 0.07%

Before introducing Pi, let's briefly talk about what Harness actually is.

As we all know, the large language model itself is more like a brain responsible for thinking —

To make it actually work in the code repository, we need to equip it with eyes, hands, feet and complete workflows.

For example, who reads files, how to call the terminal, and how to run tests after modifying code. These engineering systems running around the model are collectively called Harness.

That's why, if you change the Harness for the same model, the actual performance and invocation cost may have a huge gap!!

The "Pi" created by developer Mario Zechner is exactly such a programming workbench built for large language models.

After going through the whole project, I can simply sum it up: Pi's idea is very straightforward —

From the very beginning, it never planned to stuff all functions into its core...

Specifically, by default Pi only provides 4 tools for the model — read file, write file, modify file and execute command.

If users need planning mode, sub-agent, MCP, Git checkpoint or permission control, they can install them one by one through extensions and Skills～

Well... it's roughly like delivering a rough house with complete water and electricity first. Pi is responsible for building the four walls, and it's totally up to users to decorate it into a studio, e-sports room or three-bedroom apartment???

Coincidentally, this perfectly meets the requirements of DeepSeek...

As we all know, DeepSeek has strong reasoning and programming capabilities, but it has its own interface rules for tool calling, context management and thinking content playback.

As a result, if we directly use general Harness designed for OpenAI or Claude, we are likely to encounter problems such as incompatible tool formats, reasoning content playback errors and cache invalidation.

That's why before the official DeepSeek product is released, Pi has already done the adaptation work —

Back in April this year, Pi added native DeepSeek Provider support, and fixed the 400 error in V4 session playback at the same time.

It will retain reasoning_content as required by the DeepSeek interface, and map the internal reasoning intensity of Pi to the thinking levels supported by DeepSeek.

It's literally a folk exclusive customization.

But! What makes Pi and DeepSeek a perfect match is the "cache" feature.

After all, every time the Coding Agent takes a step, it has to send the system prompt, tool definitions, historical conversations and code to the model again, which leads to —

The longer the model task runs, the longer the request will become, and most of the content has actually been seen by DeepSeek long ago...

At this time, the automatic prefix cache can come in handy.

It can temporarily remember the beginning of the requests that have been calculated. In the next call, as long as the system prompt, tool definitions and historical conversations remain consistent, this large section of content can be directly reused, and the model only needs to process the newly added information at the end.

The "cache hit rate" measures how many tokens in this input successfully reuse the previous calculation results.

The higher the hit rate, the fewer tokens need to be recalculated, and the lower the invocation cost will naturally be.

But the trouble also lies here: if there is one extra timestamp in the prompt, or the order of tools is changed, the cache may fail immediately and recalculate from scratch.

But Pi's advantage is that it is simple enough: there are few default tools, the session content is appended continuously, and it rarely modifies the previous content.

In this way, after DeepSeek opens the log, it can directly skip the hundreds of pages it has read before, and only process the last few newly added pages, so the cache miss rate is greatly reduced～

The netizen Evan Kim has verified this through his own practice —

After DeepSeek is integrated into Pi, the cache miss rate is as low as 0.03%, corresponding to a cache hit rate of about 99.97%.

Converted to actual cost, the processing cost of 1 billion tokens is only about 19 yuan. If you don't use cache, the cost will be more than 900 yuan...

In addition, the Composio team recently conducted a horizontal test of 8 mainstream Agent Harnesses, directly loading DeepSeek V4 Flash into them to run real-world tasks.

The result is somewhat dramatic. For the same DeepSeek, if you change a set of Harness, the maximum cost gap can reach 7 times???

At first glance, it is true that Pi Agent has the lowest average cost for completing a successful task, which only costs about 0.028 US dollars.

The ones ranked after it are Deep Agents, Hermes Agent, OpenCode, Codex, Oh My Pi and Prime Agent respectively.

Let's take a look at the star product of Anthropic, Claude Code, which is well deserved as the "most expensive contestant" —

It takes an average of about 0.195 US dollars to complete a successful task. Yes, it is nearly 7 times the cost of Pi... (What a disaster)

Of course, this figure does not mean that Claude Code has poor capabilities. After all, it is a complete engineering system built around the Claude model.

It can only be said that after the underlying model is replaced with DeepSeek, the workflow originally optimized for the Claude ecosystem does not fully take advantage of DeepSeek's low cost and high cache efficiency.

The model is still the same DeepSeek, and Pi only makes it avoid reading a lot of content that it has already read.

Pi: I'm waiting for the official Harness to surpass me.

The Official Harness Team Has Been Formed, Targeting Claude Code Directly

Of course, no matter how suitable Pi is for DeepSeek, it is still a third-party solution at present.

After all, DeepSeek itself has decided to build its own Harness, and it is probably not far from the launch date? (My guess)

Back in May this year, Chen Deli, a senior researcher at DeepSeek, confirmed that the company is forming an Agent Harness team internally. To put it in a very straightforward way, its goal is —

Build DeepSeek Code Harness from scratch, to rival Claude Code.

In the following months, a large number of job openings for Harness product managers and R&D engineers were posted on the official website, just like:

At the end of July, Cui Tianyi, the head of the DeepSeek Agent Harness team and ACM gold medal winner, started recruiting relevant project developers online.

It's almost ready to launch.

Putting all the clues together, DeepSeek's development path has become gradually clear.

V4 is responsible for reinforcing underlying reasoning, programming and Agent capabilities; the official Harness is responsible for connecting the model to terminals, code repositories and toolchains; context management and test feedback can send real user failure cases back to the model team.

Of course, to be honest, Pi will not lose its position after the official product is launched.

DeepSeek Harness focuses on out-of-the-box experience and deeper collaboration with its own models.

Pi leaves the workbench for users to modify by themselves, which is more suitable for developers who like to control tools, customize workflows and switch models at any time.

After all, Pi never tied itself to DeepSeek from the very beginning...

It supports Claude, OpenAI, Gemini, Kimi, MiniMax and many other models. Users can save tokens by using DeepSeek today, and switch to other models to handle special tasks tomorrow, they can change the Provider at any time.

Well... this also means that what DeepSeek official Harness really has to face is not only a large number of Claude Code-like products, but also open workbenches like Pi that have been modified by developers to be very easy to use.

So now, the pressure is on the official version —

The Harness developed by DeepSeek itself can't be worse than third-party solutions in saving tokens, right???

Reference Links:

[1]https://x.com/composio/status/2086814488162972027?s=20

[2]https://github.com/earendil-works/pi

This article is from the WeChat official account "QbitAI", written by Mengyao, authorized to be republished by 36Kr.