Harness is already very accurate, why do we still need an Agent Native Database
RAG at the Crossroads
Retrieval-Augmented Generation (RAG) has almost become the "standard configuration" for large language models to be implemented in enterprise knowledge scenarios over the past two years. By integrating retrieval from external knowledge bases with generative models, it significantly improves the timeliness and accuracy of generated content and drastically reduces the hallucination rate. This logic sounds flawless: if the model does not know the answer, it looks it up in real time and responds after retrieving the relevant information.
However, reality is far more complicated than logic.
When a large number of enterprises actually deploy RAG into production environments, a frustrating consensus is taking shape: RAG works well, but it is far from good enough. Naive RAG — the standard pipeline of "splitting documents into chunks, vectorizing them, and then calculating similarity" — frequently fails in practical scenarios. How to determine the chunking strategy? Which Embedding model to use? What similarity threshold to set? Every link requires extensive parameter tuning, and even after tuning, the retrieval quality still cannot be guaranteed. The retrieved text chunks are irrelevant to the answer, and the mixed interfering items in the context will instead disrupt the reasoning process of the LLM.
Thus advanced RAG emerged, such as DeepDoc of RAGFlow for parsing PDFs, various advanced Chunkers, Hybrid Search, knowledge graph enhancement of GraphRAG, recursive summary tree of RAPTOR, context-aware chunking of Contextual Retrieval... The technology stack is getting more and more complex, and the architecture diagrams look better and better, but the improvement of effect does not match the increase of complexity. Complexity cannot bring corresponding performance improvement.
In 2026, two key events pushed this discussion to a new height.
The first event: Andrej Karpathy put forward the concept of "LLM Wiki". He advocates taking original notes as source materials, and using LLM to compile them into a continuously updated Wiki with clear citations and readable word by word — instead of temporarily piecing together several similar fragments every time a question is asked, the knowledge is compiled once and then kept updated, rather than being re-derived for each query. This idea quickly spilled over from personal knowledge management to the enterprise RAG field, triggering widespread doubts about the traditional RAG paradigm.
The second event: PwC US's research report raised a question with a rather provocative title — "Is Grep All You Need?" Researchers equipped the Agent with both vector search and Grep, and found that Grep (exact string matching) often provides higher accuracy than semantic search. Further comparative experiments show that the quality of Agentic RAG implemented only with string matching is already very close to that of LLM-Compiled Wiki.
These two events jointly point to a conclusion: the traditional RAG paradigm of "Query → Matching → Sorting" is being subverted. The core of retrieval is shifting from "designing a stronger recall algorithm" to "designing a decision framework that can dynamically orchestrate across multiple tools". Search is no longer an engine, but a Harness (orchestration framework).
However, although Harness + Grep constitutes an extremely strong Baseline, it is not the end point. When the data scale expands, relying entirely on Harness's iterative exploration and Grep's exact matching will lead to the Agent needing to search repeatedly to find the answer area — high Token consumption, slow response speed, and every new context window has to go through the exploration process all over again.
This is exactly the entry point for knowledge compilation. The core proposition of this paper is: Search Harness is the engine of Agentic Search; knowledge compilation is its context supply layer. Knowledge compilation is not meant to replace Search/Grep, but to provide Harness with richer, more compressed, and more traceable knowledge artifacts, so that the Agent can accurately locate the answer with fewer iterations and lower Token consumption.
The Fourfold Dilemma of RAG
To understand the value of knowledge compilation, we must first see clearly the real dilemma that RAG is facing today. These dilemmas are not defects of a certain product, but structural problems at the entire paradigm level.
Dilemma 1: Naive RAG — "Inaccurate Retrieval" is Its Predestination
The essence of naive RAG is a single path of "Query → Embedding → Similarity Sorting". Facing original documents, similarity is almost the only entry point. But the problem is: Similarity is not equal to relevance, and relevance is not equal to correctness.
An exact date query (such as "What is the revenue in Q3 2025?") will only introduce a large amount of semantically similar but completely irrelevant noise when using vector search. A vague conceptual question (such as "What is the core idea of microservice architecture?") cannot find relevant paragraphs at all with Grep — because no one will write a sentence in the document that is exactly the same as your query. Sometimes the answer is not in a single document chunk, but requires first locating a batch of candidate documents and then reading them in depth one by one.
A single path cannot cover diversified query types. This is a structural defect of naive RAG, which cannot be solved by parameter tuning.
Dilemma 2: Advanced RAG — "Complexity Cannot Bring Expected Performance Improvement"
Advanced RAG tries to solve the above problems by introducing a more complex architecture. GraphRAG uses knowledge graph to enhance retrieval, RAPTOR uses recursive summary tree to organize documents, Contextual Retrieval uses context-aware chunking strategy... Each solution has its own papers, its own Benchmark, and its own followers.
But the problem is: The deployment and maintenance costs of these solutions rise sharply, while the performance improvement is far from meeting expectations. GraphRAG takes the fragmented graph structure as the main source of retrieval. After consuming a large number of Tokens, it does not bring a fundamental improvement in retrieval performance — it only improves performance in some multi-hop question answering scenarios. Worse still, these complex solutions often make the system difficult to interpret, debug and maintain.
A RAG system that requires experts to optimize cannot stand firm in the real enterprise production environment.
Dilemma 3: Challenges of Agent Harness + Grep
PwC's research found that when the Agent is equipped with both vector search and Grep, Grep often provides higher accuracy than semantic search. This finding was quickly simplified in the industry into a rather provocative conclusion: "Grep replaces RAG".
However, comparative experiments with Vector RAG show that relying solely on Grep cannot outperform RAG in all scenarios. A more accurate conclusion should be: A well-designed Agent Harness, even if only equipped with simple tools like Grep, can achieve performance close to or even surpassing complex RAG in many scenarios. This fundamentally challenges the assumption that "RAG must rely on a complex retrieval architecture".
Nevertheless, although Harness + Grep constitutes an extremely strong Baseline, it is not the end point. When the data scale expands, relying entirely on Harness's iterative exploration and Grep's exact matching will lead to the Agent needing to search repeatedly to find the answer area — the number of iterations increases linearly, and Token consumption rises exponentially. This is exactly where knowledge compilation needs to fill the gap.
Dilemma 4: The "Cold Start" Problem of RAG — The Deepest Problem
This is the deepest problem, and it is also the pain point precisely hit by Karpathy's "LLM Wiki" concept.
The traditional RAG works in a way that starts from scratch every time a query is made. The user raises a question, the system vectorizes the question, retrieves similar document chunks in the vector library, splices the retrieval results into the Prompt, and the LLM generates the answer — then everything returns to zero. For the next query, the same process repeats again. There is no accumulation, no state, and no cognitive base that can continue to evolve.
Every query is a "cold start". Every time, the system has to re-"understand" what is in the knowledge base, what the relationship between documents is, and which sources are authoritative. Running the same task twice on the same data, the Agent may return different answers, and there is no record to explain where the difference between the two results comes from.
Pinecone's estimation points out that 85% of the Agent's computing energy is consumed in the "rediscovery" cycle, not in the real task completion. This is not a bug of RAG, but a paradigm defect of RAG. Knowledge compilation is a systematic response to this defect: moving part of the "understanding" work from query time to indexing time, so that Search Harness does not have to start from scratch every time.
The Rise of Grep and the Boundary of Harness
In Chapter 2, we saw the fourfold dilemma of RAG, among which the third one is particularly thought-provoking: a simple Grep tool, under the orchestration of the Agent, can even achieve performance close to or surpassing complex RAG. This discovery subverts the industry's inherent cognition that "retrieval must rely on semantic understanding", but also raises new questions — where is the boundary of Harness + Grep? This chapter will deeply analyze why Grep works, what role Harness plays, and how the limitations of this combination lead us to knowledge compilation.
3.1 Why Can Grep Beat RAG?
The answer to the question "Is Grep All You Need?" is far more complicated than it seems on the surface.
Researchers at PwC equipped the Agent with both vector search and Grep, and found that Grep often provides higher accuracy than semantic search. This finding is easy to be simplified as "exact matching is enough" — but the real insight lies in the conclusion at the end of the paper: Retrieval is not simply returning knowledge to the Agent. What matters is the framework — it determines what can be filled into the context, and what operations the Agent must perform before utilizing the context.
In other words, what makes Grep powerful is not the tool itself, but how the Agent orchestrates it.
This forces us to distinguish two fundamentally different types of problems:
• Semantic Discovery Problem: The answer is hidden at the conceptual level, and semantic understanding is required to find it
• Evidence Location Problem: The answer exists in literal evidence — names, dates, file paths, function names, error strings
For Coding Agents, a large number of tasks belong to evidence location: finding symbols, tracing call chains, checking differences, reading failed test cases, locating exact lines of code. In this case, an exact string matching tool is far more direct and effective than semantic search.
It is worth noting that the way Agent uses Grep is completely different from that of humans. OpenAI's analysis of GPT-5's search behavior on the BrowseComp-Plus dataset shows that quotation mark search is used in 98% of sessions (much higher than human users), and the median query length exceeds the 99th percentile of human users. AI is more "precise" and "structured" than humans when using exact matching — it can construct complex queries with multiple constraints that humans are unlikely to input at one time. Grep itself has not changed, but its user has changed from human to Agent, and the way of use has undergone a qualitative change.
3.2 Limitations of Harness + Grep: Exploration Cost and Token Cost
But although Harness + Grep constitutes an extremely strong Baseline, it is not the end point. The reason is:
When the data scale expands, relying entirely on Harness's iterative exploration and Grep's exact matching will greatly increase the cost of Harness's repeated iterative exploration. The Agent needs to Grep repeatedly to find the answer area, leading to two serious consequences:
1. Slow: Each Grep is an IO operation, and the linear growth of the number of iterations leads to a sharp rise in response time
2. High Token Consumption: Each iteration loads the search results into the context, and the cumulative Token of multiple rounds of iterations is huge
Facing every new context window, the Harness-centric Agentic retrieval will waste call counts and re-learn what effective search strategies are over and over again. This is both expensive and slow.
Therefore, a natural question arises: How to shorten its exploration path while retaining Harness's orchestration capabilities?
3.3 A Richer Tool System: Enlightenment from Microsoft AgenticRAG
Microsoft's design in enterprise-level AgenticRAG (also known as Agentic Harness for Retrieval) gives a clear direction: equip Harness with richer tools, instead of relying only on Grep.
Its core architecture is to equip a lightweight Agent for enterprise internal data search, which significantly surpasses the results of single retrieval through iterative search. Each iteration of this architecture mainly relies on three types of tools:
SEARCH is responsible for recall, and FIND and OPEN are responsible for in-depth reading — this tool combination elevates search behavior to atomic operations that can be orchestrated by the Agent.
Microsoft reports that it has achieved significant improvements on BRIGHT, WixQA, and FinanceBench. Ablation studies show that the biggest improvement comes from "shifting from single retrieval to Agentic tool usage". This confirms a judgment: Harness should not only have Grep as its only tool. The division of labor among SEARCH (wide-area recall), FIND (precise positioning), and OPEN (in-depth reading) allows the Agent to operate at different levels: first narrow the scope with SEARCH, then locate precisely with FIND, and finally obtain the complete context with OPEN.
Further research shows that the combination of BM25 + Grep can make Harness perform better — BM25 performs weakly in single retrieval, but is much stronger inside the Agent Loop. This means that we do not need complex Embedding models and vector databases, and only relying on the combination of lexical-level retrieval tools can provide Harness with sufficiently strong retrieval capabilities.
3.4 Retrieval as Harness: From Tool to Orchestration Framework
Based on the above analysis, we can draw a core conclusion: Retrieval itself is a Harness problem.
It is not just "what tools to provide to the Agent", but includes the following design dimensions:
- How queries are encouraged or constrained
- How results should be presented
- Whether the search return results contain sufficient context
- Whether the model can issue subsequent searches
- When the Agent decides that it has enough information to answer
- How Harness routes and orchestrates between different tools
In other words, Search is no longer an engine, but an orchestration framework. A well-designed Retrieval Harness, even if only equipped with "simple" tools such as BM25 and Grep, can achieve performance close to or even surpassing complex RAG in a large number of scenarios through iterative orchestration.
But as mentioned earlier, we cannot stop here. When the data scale further expands, even the optimal Harness orchestration will face the challenges of iteration count and Token consumption. This leads to the theme of the next chapter: how to provide Harness with higher-quality tools through the knowledge compilation layer, so that each tool call can cover higher-quality context, thereby reducing the number of iterations and lowering Token consumption.
Knowledge Compilation: Providing "High-Density Context" for Harness
In Chapter 3, we discussed the value and limitations of Retrieval Harness: it makes simple tools (BM25, Grep) exert amazing effects through orchestration, but it still faces the problem of excessively high exploration cost under large-scale data.
Knowledge compilation gives the answer from another perspective: Move the work of knowledge compilation from query time to indexing time.
"Instead of temporarily piecing together several similar fragments every time a question is asked, the knowledge is compiled once and then kept updated, rather than being re-derived for each query."
This idea quickly spilled over from personal knowledge management to the enterprise RAG field. Pinecone, a pioneer in the vector database field, introduced the concept of Context Compiler in the Nexus Knowledge Engine released in 2026 — it automatically completes knowledge compilation operations through built-in Skills (basic "functions" such as document processing, entity extraction, and text chunking), and processes user data into structured Artifacts. Different from traditional RAG that directly indexes original documents, the retrieval objects here are no longer unprocessed text fragments, but a collection of pre-compiled knowledge artifacts rich in semantics and entity