Anthropic’s Frontier Red Team has released FlashProve, an open-source research prototype for provable inference. FlashProve is a zkSNARK system from Anthropic that proves a reply came from a committed model. The proof reveals nothing about the model weights. On Llama 4 Maverick, a 400B mixture-of-experts model, it proves about 110 tokens/second on 1 H100. For developers, this targets a real gap: today you cannot check which model actually answered your API call.
TL;DR
- Size: Proves a 400B MoE model (~17B active per token); context capped at 1,024 tokens per request.
- Runs on: Prover needs 1 H100 80 GB, 24 CPU cores and ~290 GB host RAM. Verifier runs on a CPU, no GPU.
- Performance: Orders of magnitude faster than prior open-source zkML systems on the same 400B model.
- Best: 110 tokens/second proving on 1 H100, with 18.78 MB proofs checked in 17 seconds.
- Worst: Proving costs ~70 to 95x the GPU time of plain vLLM inference.
What problem does provable inference solve?
A user querying a model cannot verify which model produced the response. A routing service could swap in a cheaper model. An attacker could tamper with the token stream. Judging from output quality alone is unreliable.
Anthropic lists 4 use cases:
- Regulators could confirm hardware serves inference, supporting AI pacing rules.
- Labs could detect tampered inference servers.
- Users could verify text provenance.
- Regulators could enforce serving of approved models only.
How does FlashProve work?
FlashProve has a prover (the model provider) and a verifier (the user). It runs in 2 phases:
- Offline: The provider publishes a commitment, a short fingerprint binding one exact set of weights. This takes about 39 minutes and covers about 16 million requests.
- Online: For each request, the GPU produces a proof that the committed model produced those tokens.
The proof system compiles an interactive oracle proof into a cryptographic proof using hash functions. Anthropic says this offers post-quantum security. It is the same family used by validity-proof L2 networks such as Starknet.
Three engineering choices drive the speed. Prover work scales with the ~17B active parameters, not all 400B. Requests are proved in batches after the fact, like prefill. Weights stream from host memory, so a single 80 GB GPU suffices.
How fast is it? Benchmarks
Anthropic measured the public 4-bit Maverick weights on 1 H100 with 24 CPU cores. Per the repo performance guide:
- Batch of 24: 111.23 tokens/second, ~221 to 226 seconds per batch.
- Batch of 1: 17.41 tokens/second. Batch of 64: 123.31 tokens/second, flagged as unreliable on 80 GB cards.
- Proof size: 18.78 MB. Client downloads ~6.5 MB once per commitment.
- Verification: 17 seconds on 24 threads, 21 seconds on 1 thread.
Against vLLM running FP8 Maverick on 8 H100s, proving costs about 72x the GPU time per batch. At vLLM’s highest tested load, the gap is about 94x.
