The Power of Routing: Who Can Dominate the Entire Long-tail Game of AI?
Shenyi Post is a translation team under 36Kr, focusing on technology, business, workplace, life and other fields, introducing new technologies, new ideas and new trends from overseas.
Editor's Note: "Building a proprietary AI from scratch" is probably nothing but self-indulgence. Simulation data shows that the effect of refining experience from competitors' pitfalls outperforms private data, and centralized routing platforms are reconstructing computing power distribution. This article is translated from foreign sources.
Earlier this month, Stripe agreed to acquire OpenRouter for $7.5 billion.
This triggered a surge of intensive comments. Sam Lessin from Slow Ventures put it bluntly:
OpenRouter has almost no technical content, it is just an API key transfer station wrapped in excellent marketing tactics.
While @patrickmandia stated:
I am confused by the wave of routers that has emerged in the past few weeks. Every modern application-layer enterprise will evolve into a router on its own. This is the foundation of their survival, and they will never outsource it to another company.
My intuition tells me that Sam and Patrick have not grasped the essence at all. I have become increasingly convinced recently that "Outcomes Pricing" is most suitable to take root at the router layer (depending on the market structure of computing power, it may also be located at the model layer), rather than the application layer. If this judgment is true, it is reasonable that the market pricing power held by routers will also follow the market monopoly logic of the digital advertising technology (Ad-Tech) system.
Outcomes Pricing Belongs to the Infrastructure Layer
Ten months ago, I wrote an article criticizing the outcomes pricing model at the application layer. I thought their analogy to advertising technology was very stupid - in the advertising technology field, what you sell is scarce user attention, while machine intelligence will soon become a public resource "too cheap to measure". I believe that the pricing of delivered results should be determined by market mechanisms (for example, depending on the abundance of computing power supply), rather than being set as a fixed unit price in an annual contract after a polite chat with former Google and former Salesforce executive Brett Taylor.
At that time, I already felt that, contrary to the vision described by Sierra, the interest demands at the application layer were not aligned with their customers. As the application layer gains deeper insights into the ability of Agents to solve customer problems, while foundation models become increasingly better and cheaper, there is no obvious mechanism for customers to share this efficiency dividend. All excess surplus value is deposited at the application layer. Due to the extremely high potential migration cost and customers' almost no visibility into the operating mechanism of underlying Agents, customers have almost no bargaining power when negotiating more favorable terms.
Promoting outcomes pricing at the neutral infrastructure layer is exactly expected to solve this dilemma. The model layer can certainly also provide outcomes pricing - OpenAI is reportedly piloting this - but it may face the same drawbacks as the application layer: either seeking ecological lock-in, or prioritizing promoting its own models, rather than carrying out global optimization purely to deliver results in the most cost-effective way.
From this perspective, the router layer, or inference service providers and new computing power clouds that extend upward to the router field, have the most consistent business stance with customers. Their only demand is to sell more computing power - they don't care at all which model is specifically called.
However, even if this premise holds, it remains doubtful whether the router layer truly has the natural right to rise as a centralized super platform. The evolution of the digital ad tech market pattern is of great reference value here.
Market Discourse Power of Ad Tech
The internal logic between ad tech and outcomes pricing infrastructure is very simple. Ad tech tracks the final conversion results on customers' websites or applications through technical probes. For example, marketers use Meta's Conversions API to synchronize to Meta in real time when users visit a landing page or purchase goods. Marketers do this - even at the cost of sharing private business context data! - so that Meta can deliver the conversion results they expect at the optimal price.
But it is undeniable that giving such data to Meta objectively empowers competitors. For example, if you are Neiman Marcus, when you tell Meta that a user with a specific profile has just placed an order, you are actually helping Meta iterate its advertising algorithm, so as to more accurately push advertisements to your arch-rival Saks Fifth Avenue. This obviously deviates from the "learning flywheel" principle advocated by people like Satya Nadella.
This means that the real opportunity does not lie in selecting the optimal model, but in building a learning closed-loop above the model that allows intellectual capital and Token computing power capital to compound continuously.
If a centralized router or model layer can really deliver business results more efficiently than a single enterprise itself, the theoretical basis of "every enterprise should build its own exclusive learning flywheel" begins to shake. This ending is inevitably creepy, because it directly subverts the industry consensus that vendors such as Databricks have been vigorously promoting for more than a decade - that enterprise private data is the lifeblood of every company, and giving away any data is no different from self-destructing excess returns (Alpha), losing technical autonomy and stifling future growth space, etc.
However, the development history of ad tech has not spared Nadella and others' grand narrative. The decline of the Open Internet has many incentives, but there is no doubt that the advertising infrastructure inside the Walled Gardens, with its highly integrated minimalist experience, is far more sophisticated than the fragmented open network. Meta is still one of the platforms with the best advertising performance across the network, and its end-to-end connected "continuous learning" technology stack is the key to winning. To be fair, the two cannot be completely equated, but marketers who try to "build their own closed loop" on the open network often have a much lower return on investment than those who directly entrust Meta or Google for hosting.
To put it further, there is an essential difference between the market mechanism of ad tech and outcomes pricing: Entropy. Ad tech is directly built on a high-entropy environment - the ever-changing user preferences, and the torrent of products that merchants keep iterating to cater to these preferences. If public preferences are constant, even if Meta does occupy scarce user attention, the value of the technical architecture and data barriers it builds will probably shrink significantly; because once you have a more accurate model of user preferences, and this preference no longer drifts, the system is almost finalized. At that time, any player that gathers attention can easily replicate this advertising engine by detecting these fixed and unchanging preferences.
This is in line with a long-term inquiry of mine: whether the long-term fate of cutting-edge AI laboratories depends on how rich the "unknown surprises and variables" (Surprise) in the real world are. I later formally modeled this hypothesis here.
Now it seems that "the scale of entropy in the real world" may be the decisive physical quantity: it will finally determine whether the centralized router platform can capture stronger economic benefits by virtue of its excellent generalized learning for continuously emerging new models and new problem forms; or as @patrickmandia envisioned - routers are just bulk standardized components deployed on the edge side. Because the real world is not as complicated as imagined, the local system can quickly figure out which Agent runtime (covering models, operation scaffolding, external toolchains, etc.) is most suitable to solve edge business pain points, making third-party routers unable to form any learning barriers at all.
I am conducting simulation test research on this subject, but before unfolding the data, it is necessary for us to clarify: why is the router platform more advantageous in implementing outcomes pricing than cutting-edge AI laboratories?
Outcomes Pricing: Why Routers Outperform Cutting-Edge Laboratories?
As mentioned earlier, there are reports this week that OpenAI is piloting outcomes pricing among a small group of customers, which has also been confirmed by *The Information*. Before that, Anthropic's official document on "Define Outcomes" has been quietly online for several months.
Ad giants represented by Meta and Google, against Nadella's warning, have successfully convinced countless applications and websites to continuously send back telemetry data - that is, real conversion results! So why can't or won't cutting-edge private model laboratories replicate this set of tactics?
I think the fundamental difference between the two is that advertising platforms firmly control a kind of naturally scarce resource - attention; while model laboratories have not yet controlled a scarce product of the same significance (unless computing power itself is counted?). Advertisers such as applications and sites are forced to share data with advertising platforms because the latter holds the scarce attention entrance that brands dream of. Although this does cause ecological lock-in, advertisers often have no choice. However, AI users have sufficient retreat options - they can call various open source models on a wide variety of computing power clouds at any time.
The underlying demand of OpenAI and Anthropic is to sell high-margin model inference services. The goal of routers and inference providers is only to consume computing power. They don't care which model runs on the end. For this reason, neutral open model platforms are naturally more credible when promoting "delivering deterministic results at the optimal price" than major closed-source laboratories.
If we agree with this logic, the next core question is: can a centralized platform that continuously aggregates massive verified result streams build an insurmountable competitive barrier compared with the local router built by the enterprise on the edge side?
Value Proposition Simulation: Centralized Platform vs Local Router
Transaction Mechanism for Purchasing Deterministic Results of Agents
Imagine specifically that a centralized router like OpenRouter or Router.com starts billing based on business results. Its operation paradigm will deeply follow the example of ad tech: invite buyers to integrate telemetry interfaces in their business applications (similar to Meta's Conversions API) to feed back to the router in real time whether the expected business goals are actually achieved. Buyers only need to pay when the telemetry probe confirms that the result is achieved, and the "Definition of Done" of the task is independently defined by the buyer, which is exactly the same as the result specification set by Anthropic.
Admittedly, this may create moral hazard: result buyers may have an incentive to maliciously tamper with telemetry signals, deliberately underreport or misreport successful results. But this concern can be hedged through game mechanisms - because the router will carry out reverse algorithm tuning based on the actual effect reported by the customer, and malicious underreporting will directly lead to the decline of the system's service delivery efficiency for the customer (or even become more expensive!). Specifically, the router may start to frequently reject the customer's Agent call requests (because the system cannot confirm whether it is cost-effective to spend computing power on such a picky customer with extremely low book conversion rate), or greatly increase the billing threshold to ensure that gross profit can still be maintained in the face of potential high false positive risks.
Of course, the transaction logic of Agent labor is still subtly different from advertising. Purchasing conversions from Meta essentially still buys scarce user attention, except that the bidding distribution mechanism is unilaterally optimized for conversion rate. I will publish a long article to discuss this game dynamic in depth soon.
Underlying Economics of Centralized Routers
So, can such a centralized router really create real incremental value for the application layer?
The idea of this centralized router directly hits Nadella's core argument: buyers no longer stick to their own internal data learning closed loop, but actively synchronize verified business result data to the router continuously, in exchange for the optimal unit result procurement cost.
The underlying logic is that centralized routers can absorb the uncertainty brought by long-tail business problems and the evolution of cutting-edge models at a response speed far faster than a single customer and at a much lower cost. A single customer may need a very long cycle to experience a complete distribution of business problems, while the router platform can overview all problem forms encountered by the entire customer group in real time. As long as there is a time window for each customer to step into the pit one after another, the aggregation effect can bring forward-looking hedging benefits to subsequent customers. For example, your competitor may have encountered a type of occasional failure before you; the router platform can use the verified result stream from the competitor to solve the problem for you in real time; the premise is always that the service procurement price of the router must be lower than the difference between the R&D cost of your own exploration and the opportunity cost caused by transferring part of the data information.
I visualize this core concept as follows:
How the router platform uses the aggregation mechanism to complete predictive learning before a single customer encounters a new blind spot.
If the commercial foundation of cutting-edge laboratories depends on the "scale of variables and unknowns" contained in the world, the same is true for centralized routers. For routers to maintain their core value proposition to customers for a long time, they must rely on a premise: the world must be "always changeable and full of unknowns", so that directly purchasing hosted task results is significantly more cost-effective than building Agents from scratch on your own. If the objective world is calm and highly certain, there is no reason for enterprises to give up profits to external hosts for no reason; on the contrary, if the world is changing rapidly and full of turmoil, subcontracting this complexity entirely to a third party to handle becomes the wisest choice.
We might as well refine this economic value into "Residual Transferable Uncertainty": the unknown redundancy that remains after completing local private learning and can be migrated and reused across different customers.
To verify this, I built a simulation system: simulating a router platform that monitors business results across customers, to test whether it can deliver these results more efficiently than an independent enterprise working alone. Here we first assume that system optimization only occurs at the dimension of "Agent runtime" (covering model calls, scaffolding, tool scheduling). I further simulated whether the continuous stream of transferable unknown variables is sufficient to solidify this comparative advantage into a lasting barrier, and whether this lasting technical dividend can finally be realized as a monopolistic market discourse power.
Specifically, I focused on evaluating the following core propositions:
Can Agents achieve progressive self-healing by continuously observing real business results (or can high-order models get it right in one single inference)?
Can the strategic experience learned by an Agent from one enterprise be seamlessly reused in another (similar? Or completely different?) enterprise?
Does the private data of customers contain unique increments that competitors cannot provide - in other words, is Nadella's proposition that "all enterprises are compound interest closed loops" and Applied Compute's assertion of "Specific Intelligence" logically tenable at the bottom?
Does the excess advantage of the router platform directly depend on the size of the fresh blind spots for the target enterprise, so that as long as the enterprise continues to encounter unknown new problems, this advantage can last forever?
Compared with individual customers fighting on their own, can the scale effect of routers enable them to access low-cost models or vertically fine-tuned models at a faster pace and with lower thresholds?
In order to deduce and answer these doubts, I adopted Sierra's tau2-bench evaluation framework.
This is a sandbox benchmark for quantitative evaluation of multi-domain customer service Agents. Each domain defines the business specifications that Agents must abide by, the set of tools that can be called, and a series of business work orders used to measure the overall performance of Agents.
Covering three major sectors: air transportation, retail consumption and telecommunications operations. I set each task as a customer support work order: simulating a customer with implicit demands calling an Agent that masters the company's business specifications and toolchains; the result of the work order is either success or failure. I score the Agent with points - 1 point corresponds to 1 percentage point of the successful closed-loop rate of work orders. The participating customers are synthetically generated enterprise entities, which share the business processes and toolchains in the telecommunications field, but have significant differences in the distribution of problems (different customer group sizes, mutually exclusive pain point types) - for example, a occasional failure that one company has never encountered may be a common occurrence for another company. Work orders are injected in batches, and Agents continue to reflect and learn based on these work orders with absolute real labels.
Learning evolution mainly occurs in the peripheral scaffolding layer around the fixed foundation model, which is divided into two evolution paths: (i) Playbook Learner: deliver the interaction transcription text of failed execution to a higher-order model, which refines or corrects a refined operation memo and dynamically injects it into the Agent's system instructions; (ii) Case Retrieval Learner: before the Agent starts each round of dialogue, retrieve and present three historical work orders with the highest similarity (two have been properly resolved, one is unresolved) to it. Each learner is subdivided into three variant versions with different historical data read permissions: (1) Only allowed to read the private logs of its own enterprise; (2) Read the aggregated logs of the whole industry; (3) Only allowed to read the historical logs of other competing enterprises.
The experimental results are revealed. Overall, it is not unexpected - experience-based learning mechanisms do have immediate effects, but as the intelligence level of models rises, the marginal benefit of this learning dividend is narrowing significantly. A medium-performance model (gpt-5.6-luna was used in the experiment) initially could only handle 17 out of 100 tricky work orders in the benchmark test; but once it was allowed to reflect and iterate on its own failure cases, the number of work orders it could solve quickly soared to 53.