Theorem and Anthropic report LLM-scaled formal verification is getting cheaper
Theorem and Anthropic describe LLM-scaled formal verification and say Opus 5.5 reached a 50% solve rate on small tasks with about 40% fewer tokens than Opus 5.
Theorem and Anthropic say large language models are making formal software verification feasible at a much larger scale. In a joint blog post titled "LLMs are making formal software verification feasible at a much larger scale," the companies report that on a set of 36 small verification tasks, Opus 5.5 reached a 50% solve rate using about 40% fewer tokens than Opus 5.
Formal verification uses mathematical proofs and logic to show that software or hardware meets a precise specification. Testing, patching and scanning can find many bugs, but they leave you unsure whether any remain. Clark Barrett, co-founder and co-CEO of &Truth, a competing formal verification company, put the difference this way: the question is "how do you know when you're done." A proof, by contrast, is checked by a program.
The cost problem comes from tokens, the units of text a model reads and writes. The post says formal verification remains token-expensive. Its argument is that the cost is falling, and the drop from Opus 5 to Opus 5.5 is the example it gives. The post also expects the trend to continue as more models are "trained on long verification tasks," but that is a forecast, not something the reported figures demonstrate.
The excerpt doesn't say what the 36 tasks were or how large they were, and the reported solve rate is for small tasks only. It gives no absolute token counts, so a reader can't tell how much a single verification costs in dollars or time.
Theorem also published a proof-efficiency evaluation that measures "KB of machine code verified per hour" by model release date, with 95% intervals. Theorem's homepage makes a broader claim, stating that "Verification is the only defense that holds while adversaries scale." That is the company's argument for why verification matters, not a result from the post.
The Anthropic diagram in the post shows the workflow as moving "from code to a machine-checked guarantee," and in that example it uses Rocq rather than Lean. Lean is an open-source programming language and proof assistant, and &Truth builds on it. The excerpt doesn't say how much of the reported cost reduction depends on the proof system used.
What changed
On 36 small formal verification tasks, Opus 5.5 reached a 50% solve rate using about 40% fewer tokens than Opus 5.
Type: Research · Applies to: Opus 5.5, Opus 5
Why it matters
Formal verification is moving closer to being affordable for security work on real codebases, which could change how teams check AI-written code.
Original source
We have no link to the original source for this story yet.
Where we saw it
Latent Space
AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it. How it works