180 Million Lines of Proof: AI Solves Software Verification

For decades, the idea of mathematically proving that a piece of software works perfectly in every possible scenario seemed like a luxury reserved for the most critical systems. Now, a new approach suggests that artificial intelligence can handle this heavy lifting on a scale previously thought impossible. Anthropic has announced that its Claude model successfully used formal methods to prove that existing software code behaves correctly, marking a significant shift in how developers might ensure their programs are trustworthy.

The scale of the effort is striking. In a matter of weeks, the model generated approximately 180 million lines of mathematical proof code. This stands in sharp contrast to traditional methods, where specialists required 20 person-years to verify seL4, a small operating system kernel. Because of that high cost, formal verification has largely been confined to extreme cases such as avionics software for aircraft. The new process relies on test programs from 123 open-source libraries within Google's OSS-Fuzz security project, allowing the AI to draw on a vast amount of existing data.

The work was a collaboration between Anthropic's Frontier Red Team and a partner called Theorem. The goal was not to write new software from scratch, but to verify that existing code functions as intended. Claude achieved this by turning crash tests into mathematical problems and analyzing the compiled machine code. It then constructed extensive proofs to demonstrate that the code behaves correctly under specific conditions. The process is described as fully autonomous, meaning the AI handled the generation of the proofs without constant human intervention.

A separate verification program named Rocq automatically checks those proofs. According to Anthropic, this automated checking means the process requires almost no human reviewers. However, the researchers emphasize that the underlying tools and formal foundations were built by humans at Theorem. This distinction is important for understanding the limits of the AI's role, as it is executing a complex task using a framework created by human experts.

During the verification process, Claude encountered hundreds of bugs and was permitted to fix them. The researchers noted that many of these issues would likely have escaped conventional testing because they only appear under very narrow conditions. For example, some bugs surfaced only when dealing with unusually large input files or a specific sequence of memory accesses. In one notable instance, the team found as many security bugs in a widely used PDF reader in one week as are usually reported for that software in an entire year.

The AI also identified a logical error that had gone unnoticed for more than a decade in another application. Anthropic adds that even when a complete proof fails, the attempt can still reveal real bugs. This suggests that the process has value even if it does not always result in a perfect, verified proof. The ability to catch these rare, deep-seated errors is a major advantage over traditional testing methods, which often miss these edge cases.

Despite the impressive results, the company is careful to clarify that formally verified does not mean error-free in every real-world context. The proofs hold only under defined conditions, including a particular processor architecture, single-threaded programs, and inputs smaller than 64 MB. Many real-world deployments of these libraries run on different hardware and configurations, so the verification does not cover every possible scenario. This limitation is a crucial detail for developers who might rely on these proofs for their own projects.

Cost remains a significant constraint for this approach. Individual proofs consume hundreds of millions of tokens, and Anthropic has not published the total project costs. However, the company argues that if AI can verify its own code, developers could trust AI-generated software more and spend less time reviewing it. This could lead to lower costs across the coding process. A smaller test run indicated that proofs can be reused, and efficiency is improving, with newer models solving proofs using fewer tokens.

This project shifts the focus from whether AI can write code to whether it can prove that the code is safe. Anthropic expects formal verification with AI to become as practical as standard testing in the future. The roadmap for this work already targets critical infrastructure, including major libraries like glibc, OpenSSL, and cURL. As these tools become more efficient, the line between writing code and proving its correctness may blur, changing the landscape of software development.