Easy to develop, hard to deploy: How can you get an enterprise-grade Agent ready for launch? You have to master "evaluation" first?

In the article Before Billions of Agents Go Live, Amazon Taught Them to Manage Inventory and Hire Talent, Matt Garman, CEO of Amazon Web Services (hereafter referred to as "AWS"), predicted that billions of Agents will operate widely across all industries in the future. While this outlook is undoubtedly exciting, some realistic caveats are necessary.

Last July, a report released by MIT Project NANDA, based on interviews with over 300 AI projects and 52 organizations, as well as surveys of 153 executives, found that despite enterprises investing $30-40 billion in generative AI (GenAI), only 5% of organizations have successfully achieved large-scale deployment and obtained significant financial returns. This phenomenon is known as the "GenAI Divide": the vast majority of organizations are trapped in the pilot phase marked by "high adoption rates but low conversion rates".

This divide also exists in the Agent domain: demos perform well, but Agents fail once connected to real-world scenarios. If billions of Agents emerge in the future, it remains uncertain how many of them will actually be usable and effective. However, the root cause of the problem may not lie in insufficient model capabilities, because Agents need to be deeply integrated into specific business scenarios, where engineering issues can be more critical than model performance. The recently released AWS Guide to Enterprise Production-Grade Agent Development and Deployment (hereafter referred to as "the Guide") notes that the failure of traditional software engineering methods for Agents stems from three essential differences between traditional software and Agents:

Non-determinism. The operation logic of traditional software is deterministic, with clear criteria for right and wrong. Agents run on large models, and their outputs are probabilistic: the same input does not necessarily produce the same output. A passing test yesterday does not guarantee stable performance today. No mainstream model provider currently promises fully deterministic outputs.

Prompt as source code. In traditional software, code modifications leave traces, with version control and static analysis tools available. But prompts do not. Even a minor tweak to a single word in a natural language prompt can trigger drastic fluctuations in Agent behavior, and there are no mature industry tools to assess the scope of impact of such changes.

Implicit dependencies. Agents have implicit dependencies on underlying large models: if a model provider quietly upgrades its backend systems without any changes to the Agent's code, the service quality of the Agent may have already shifted.

The superposition of these three differences renders the traditional software evaluation and testing system completely ineffective for Agents, causing enterprise Agent projects to stagnate when entering the production phase. So what should enterprises that aim to reduce costs and improve efficiency with Agents do specifically?

01 From SDLC to ADLC: Evaluation as the Core

Xiaosong CHU, Amazon's Global Vice President, once put forward a viewpoint: when enterprises build AI Agents, underlying technology platforms can be acquired through procurement, but evaluation criteria must be independently controlled by the enterprises themselves. The core competitive barrier of an enterprise lies in its proprietary golden datasets and evaluation criteria. This claim seems counterintuitive: models, infrastructure, and development tools can all be purchased (and self-development is indeed unrealistic for the vast majority of enterprises). Why are evaluation criteria so critical?

Looking back at history, the rapid development of computer science in the 1960s gave birth to the prototype of the production framework that eventually evolved into SDLC (Software Development Life Cycle). The core of SDLC is to divide development work into multiple phases: requirement analysis, design, coding, testing, deployment, and maintenance. As AI Agents take over a large number of development tasks, the ADLC (Agent Development Life Cycle) methodology has emerged. The biggest difference from SDLC is: ADLC is a flywheel, not an assembly line.

ADLC does not end after a single run; instead, it rotates continuously and iterates persistently. Its six links — defining criteria, development and implementation, performance evaluation, canary launch, continuous monitoring, and improvement iteration — flow back from the last link to the first, updating evaluation criteria and benchmark datasets. If traditional software follows the path of "development → testing → launch", Agents follow the path of "defining criteria → development → evaluation → launch → monitoring → identifying failures → updating criteria → re-development". Evaluation is both the starting point and the end point.

If you are an enterprise manager preparing to launch an Agent project and have identified a suitable scenario, you need to define what a "good" Agent is before starting development. This includes Agent definition (what it is and what it aims to do), tone and personality (how it communicates), tool and parameter definition (what tools it can use and how to use them), and benchmark datasets (what counts as "successful completion").

After the Agent is launched, data from the production environment must continuously flow back into the evaluation system, which requires you to establish a comprehensive observability system (the Guide recommends using OpenTelemetry). Without observability, there can be no continuous evaluation, and the flywheel cannot keep rotating. Finally, you need to make the system architecture evaluable — this is the most engineering-intensive part and the infrastructure that determines whether evaluation can be implemented. The Guide recommends a three-layer design: authentication layer (verifying user identities), authorization layer (Gateway, controlling what the Agent can do), and session isolation layer (ensuring no interference between different users).

Once the Agent is built, how can you judge whether it performs well? This is the most difficult question to answer. The transition from underlying large models to Agent deployment is like a big tree growing branches, leaves, flowers, and fruits: you may only need one or a few large models, but Agents are countless and diverse. The same applies to Agent evaluation — each Agent should have a dedicated evaluation scheme, and it is unacceptable to launch an Agent based on a vague feeling that "it seems to be working fine".

02 Evaluation Methodology: Two Pillars

Those who have built Agents may be familiar with this scenario: testing shows no issues, but once connected to real traffic, the Agent starts to "fail intermittently" — for the same type of request, problems occur once or twice out of ten, which is quite frustrating.

For Agents, capability and reliability/consistency are not the same thing. "Being able to perform a task" does not equal "being able to perform it every time". Agents couple multi-step reasoning, tool calls, and external state writing together, and the randomness in any link will be amplified in a chain reaction. Only through large-scale and repeated evaluations can we approach the goal of "performing the task successfully every time".

The Guide proposes the "two pillars" of the evaluation methodology: the first pillar determines the depth of evaluation granularity, ranging from only examining the final response (black-box), to reviewing the complete execution trajectory (glass-box), and then to inspecting single-step details (white-box); the second pillar determines the weight of each score, ranging from mechanically verifiable (Layer 1), to semi-objective (Layer 2), and then to default subjective evaluation (Layer 3).

Black-box evaluation focuses on the final output: a user asks a question, the Agent provides an answer — is it correct? Glass-box evaluation examines the complete trajectory: what decisions the Agent made, which tools it called, and whether each step of reasoning is reasonable? White-box evaluation focuses on a single step: whether a certain tool call is correct, or whether a certain piece of reasoning is valid. These three granularities, from coarse to fine, answer the questions of "is the result correct", "is the process correct", and "is each step correct" respectively. In daily development, glass-box evaluation is the main approach, with black-box and white-box evaluations as supplements.

The first layer of the three-tier evidence weight system is mechanical verification, which checks whether the format is correct and whether JSON can be parsed — it is fully automated with zero subjective judgment. The second layer is semi-objective pinned evaluation, which uses fixed evaluators and clear scoring standards to score specific dimensions. The third layer is subjective default evaluation, which has no fixed standards and relies on the judgment of humans or LLMs. They correspond to three types of scorers: code rules, models, and humans.

The two pillars are mutually orthogonal: indicators at the same granularity can come from different evidence levels, and vice versa, forming a 3×3 matrix. With the same set of indicators, you need to select both the granularity and the evidence strength. This combined approach can basically conduct a comprehensive evaluation of the Agent's output. Take the customer service Agent developed by AWS as an example: the biggest risk in customer service scenarios is incorrect intent recognition, where what the user says is not what the Agent understands. AWS uses a dual-track evaluation method of "real data + virtual customer simulation" to expand the test coverage to various edge scenarios at a relatively low cost, testing both the accuracy of intent recognition and the coherence of multi-turn conversations.

It is worth mentioning that since model-based automatic evaluation is also used in the evaluation process, the quality of the evaluation dataset determines the upper limit of evaluation quality. Enterprises need to build high-quality test sets that have been manually labeled and verified by business practices. This dataset will become a core asset of the enterprise.

There are many detailed dimensions for evaluating Agents, and different types of Agents focus on different dimensions. For example, customer service Agents prioritize intent recognition accuracy and conversational coherence, tool-using Agents prioritize tool selection correctness and parameter accuracy, and multi-Agent collaboration systems prioritize rational task decomposition and execution stability. These details will not be elaborated here.

The ideas and methodologies provided by AWS are just one perspective. There may be other paths and practical cases for building excellent Agents. But one thing is certain: Agents are productivity tools, and the ability to deliver measurable business results is the ultimate criterion for judging whether they are "good".

Entering the Agent era, enabling Agents to truly enter production environments is a new technical and business challenge for enterprises. Even with the Guide, whether the path can be successfully traversed depends on the determination and investment of the enterprises themselves.

This article is from the WeChat Official Account "DoNews" (ID: ilovedonews), written by LI Xinma, and published with authorization from 36Kr.