Just now, Anthropic open-sourced FlashProve, issuing an "anti-counterfeiting ID card" for large language models.

Just now, Anthropic published a statement claiming "New progress has been made in verifiable reasoning of large language models, and a system named FlashProve has been open-sourced." Based on zero-knowledge cryptographic proof technology, it enables model providers to prove that a certain response is indeed generated by the specified model without disclosing the model weights.

What does this mean?

In daily scenarios, is the AI you pay to call really the model promised by the service provider? Suppose an enterprise purchases the API service of a top large model, and the service provider promises to use the model with the strongest capability, but secretly replaces it with a smaller model with lower cost in the background. Can users detect this?

Or, when a third-party model routing platform forwards requests, it secretly replaces the originally specified model. Even if the final returned response is of good quality, what way can users prove that it indeed comes from the model they paid for?

At present, users usually can only trust the promises of service providers, and it is difficult to independently verify from a technical perspective. The latest research announced by Anthropic is trying to solve this problem.

What is more noteworthy is that this technology has been tested on a Mixture-of-Experts model with a total parameter count of 400B: a single NVIDIA H100 GPU can generate inference proofs corresponding to about 110 Tokens per second, and the proof file for each 1024 Token request is approximately 19 MB.

In the past, most research on zero-knowledge machine learning focused on smaller-scale models, while FlashProve further advances verifiable reasoning to the large language model scenario.

At present, Anthropic has open-sourced the proof generator of FlashProve, C++ and Lean verifiers, as well as relevant technical documents and formal proofs.

• Official Blog: https://www.anthropic.com/research/provable-inference
• Github: https://github.com/anthropics/FlashProve

Issuing a "Proof of Identity" for AI Responses

Anthropic calls this technology "Provable Inference". In the official blog, the team first puts forward four types of trust issues faced by AI services, trying to meet such a demand: when a user sends a request to an AI service, they cannot confirm which model the returned response actually comes from.

For example, third-party service providers may replace the high-performance model specified by users with a cheaper one to reduce costs; the inference server may also be attacked, leading to tampering of model weights or output Tokens. For researchers conducting security assessments, if they cannot confirm that the test object is consistent with the actually deployed model, the evaluation results will lose reliability.

The core idea of this technology is that the model service provider first generates a Commitment for a set of determined model weights through cryptographic methods, which is equivalent to creating an encrypted fingerprint for the model version. After that, every time a user sends a request, the service provider can attach a zero-knowledge proof (zkSNARK) to prove that this response is indeed generated by the previously committed model based on the corresponding input.

In this way, users do not need to obtain model weights or re-run the model by themselves, and can independently verify the identity of the model through the model commitment, input, response and proof file. The proof process will not leak model weights, but can confirm whether the output comes from the specified model. Of course, it proves the source of the response, not whether the content of the response is correct.

400-Billion-Parameter Model Achieves Proof Speed of 110 Token/s

To verify the feasibility of the solution, Anthropic selected Meta's Llama 4 Maverick for testing. This is a MoE model with 400 billion parameters, and about 17 billion parameters are actually activated for each Token.

On a single NVIDIA H100 GPU, the proof generation throughput of FlashProve reaches approximately 110 Token/s, and each proof file is about 18.78 MB. The verification end does not require a GPU, and it takes about 17 seconds to verify a proof with a 24-thread CPU. It should be noted that the current test uses a request window of 1024 Tokens, and each proof actually checks one output Token among them.

This efficiency improvement is related to the optimizations made by FlashProve for the MoE architecture: the proof calculation mainly focuses on the actually activated experts, instead of processing all parameters of the model every time. At the same time, the system adopts batch post-proof, which uniformly generates proofs after the inference is completed, reducing the overhead caused by repeated calculations.

However, generating proofs still incurs additional costs. According to Anthropic's tests, the GPU time required to generate proofs is about 70-95 times that of ordinary vLLM inference. In addition, the current system mainly supports a context window of 1024 Tokens, and is verified for integer models and Greedy Decoding, which cannot directly cover all models and inference methods.

To further improve the credibility of the system, Anthropic also uses the Lean formal verification language to perform machine-checkable proofs for core security properties such as the completeness, reliability and zero-knowledge of the protocol, and open-sourced the code, protocol description and relevant proofs under the Apache 2.0 license. However, formal verification does not mean that all actually running codes get the same guarantee, and further audit is still needed between the fast C++ verifier and the formal protocol.

A Specific Group of Users May Have Strong Demand for This "ID Card"

At present, many users call models from different manufacturers through third-party API transfer stations or model aggregation platforms. Such services lower the threshold of model invocation, but also bring information asymmetry: users can select the model name, but may not be able to independently confirm the model actually running in the background.

For example, in a multi-layer forwarding call link, the service provider may theoretically forward requests that should have been processed by high-performance models to models with lower invocation prices for cost considerations. For users, even if they find that the quality of the response has changed, it is difficult to judge whether it is due to the randomness of the model itself, the change of service status, or the change of the actually invoked model.

In the absence of independent verification means, it is difficult for users to confirm exactly what model they invoked only based on the returned results.

One netizen believes that when ordinary users call models through third parties, it is difficult to confirm whether the response on the screen comes from the promoted large model, or is replaced by a lower-cost model by the service provider, and may even suffer from man-in-the-middle tampering.

FlashProve provides a new inspection idea. If the model service provider announces the weight commitment of a specific version in advance, the service can also provide the corresponding cryptographic proof when returning the response. Users do not have to fully rely on the verbal promises of the service provider, but can verify the proof to confirm whether this request is indeed completed by the model with the specified weights.

There is Still a Gap Before Large-Scale Application

Of course, while Anthropic describes a bright prospect, it also clearly points out that FlashProve is still a research prototype at present, and there are several engineering and theoretical constraints.

First, the proof cost is still relatively high. Although it is several orders of magnitude faster than previous generation systems, the GPU time required to generate proofs is still about 72 to 94 times higher than that of ordinary inference (vLLM), which means that if real-time proof generation is required for each request, the service cost will increase significantly.

Second, the currently supported calculation forms are limited. The current experiment targets the integer operation version of the model and Greedy Decoding. For other inference methods such as floating-point calculation and random sampling, further expansion is needed.

In addition, the context length is limited. The current experiment demonstrates a maximum context proof scale of 1024 Tokens. For long contexts, large-scale Agent workflows and complex multi-turn interactions, how to generate and verify proofs efficiently remains a problem to be solved.

One more point that needs special explanation is that FlashProve proves that the response indeed comes from the specified model, and does not prove that the response itself is correct. That is to say, if the specified model generates hallucinations, FlashProve may still successfully verify this response. It solves the problems of model identity and inference integrity, not the problems of model capability and factual accuracy.

However, it is still worthy of recognition that the open source of FlashProve brings new possibilities for the trusted governance of the entire AI industry chain. With the increasing requirements of compliance supervision, model supply chain security and high-value scenarios (medical diagnosis, financial decision-making, legal analysis) for model identity confirmation, the "untamperable model ID card" has stepped from pure academic conception to the stage of engineering practice.

In the future, what model service providers deliver to users may no longer only include responses and Tokens, but also an additional cryptographic credential that can prove "this is indeed generated by the model you specified".

References:

https://x.com/AnthropicAI/status/2117710942826123294

https://www.anthropic.com/research/provable-inference

This article is from the WeChat Official Account "Machine Heart" (ID: almosthuman2014), written by Machine Heart focusing on AI, and published with authorization from 36Kr.