As of August 19, 2026, Specula has identified 382 bugs in 67 open source systems and is used by developers in multiple companies and open source communities.
Today, coding agents are able to write functionality, supplement testing, and promote PR, but it's still hard to get rid of expert guidance once the formalized regulations are met. The TLA+ Foundation and NVIDIA hosted their first TLAi+ Challenge, hoping to lower the threshold of code-writing with AI. The Specula team took first place.
This is exactly what the open-source project Specula is aiming at. It allows coding agents such as Claude Code, Codex, Copilot CLI to read the code, documentation, test and file history of the target system, automatically generate TLA+ models and correctness variables, run model checks, and then take found instances back to real code and pack the tests. The entire process can be fully automated, without the need for developers to learn TLA+ or model checks, which leaves the formalization method trapped in the expert circle for a long time, with the first chance to become an engineering tool available to ordinary developers.
Specula is the official website:
Specula's paper:
Specula GitHub：
Specula Bug tracking chart:#gid=996176958
TLAi+ Challenge：
As of August 19, Specula has detected 382 bugs in 67 open source systems covering complex systems such as MongoDB, Etcd, ScyllaDB, HashiCorp Raft, Rabbit MQ/ra, GCC libgomp and LLVM libomp.
Why is it hard to find bugs?
Why is it more difficult to write formal regulations?
So-called synchronous bugs often do not write a single line of code directly wrong, but several threads or nodes each seem to do the right thing, but together drive the system to error in a rare sequence. A combination of thread switching, message sequencing, node failure and disk delay can form a huge number of execution paths. Normal tests can usually only draw on a small portion of them, and the most tricky problem lies in paths that rarely occur but are actually achievable.
The idea behind the TLA+ formalization approach is to abstract and transform system behavior into a state, and then allow the model checker to systematically explore the achievable state: if a path breaks through "submitting indexes cannot be reversed" and "majority-confirmed data cannot be lost", the tool returns an inverse case.
The real difficulty is that a regulation that can be used to find bugs must go through four steps simultaneously: refining the correctness of the system's true promise from complex code and history; preserving enough behavior to expose bugs while controlling the state space; ensuring that the model is consistent with the actual execution; and having the model's counter-examples revert back to the code, eliminating the false bugs that exist only in abstract models.
The core principles:
Judgment and decision-making come from artifact, agent learning in advance
Specula does not end with a large model generating a TLA+ file at once. It breaks the work down into four links that can be independently accepted: understanding the correct nature, generating effective models, checking model-code consistency, reproducing abnormal behaviour in real code. Each loop leaves a clear artifact, the next loop uses it and checks it; the agent's critical judgments must also be accompanied by evidence such as code, issue, submission or execution trajectory.
Step 1: Extract variables from systemic evidence
Specula summarizes the protocol-level and implementation-level variables in code, annotations, documents, tests, issues, and historical corrections, and asks the agent to provide evidence for each characteristic. In MongoDB, for example, it does not move the persistence assumption of the Raft textbook edition, but will do so based on achieving historical recognition: "Most nodes have design choices that can be submitted in memory". In paper statistics, 87.35% of the variables cite code or annotations, and 74.34% cite issues, PR, or security announcements.
Step Two: Modeling around critical scenarios
Specula then extracted high-risk scenarios from the documentation, test, issue, and submission history to generate customized models for each scenario: retaining relevant variables, actions and failures, abstract irrelevant details, avoiding state space explosion. ScyllaDB Raft's voter demotion has been repaired three times in a row, and Specula thus independently modeled the old configuration with ReadBarrier, eventually discovering a new bug in the card read barrier during the downgrade.
Figure 1: A simplified modeling program generated by Specula for the ScyllaDB Raft library: refining scenarios, variables, actions and invariants from historical repair and code evidence.
Step 3: Check the model with real trajectories
Specula is automatically programmed to plug in, collect actual execution tracks, and then gradually check whether these tracks are acceptable to the TLA+ model; once forked, it can locate the differences between the model and the implementation.
Step 4: Bring back the code and replicate the anomaly
When a model inspection detects that a non-variable is violated, Specula converts the counter-case into a defined sequence of events, controlling the failure with a concurrent sequence in the real system, repositioning the same behavior, and encapsulating the iterative process into a test.
Two self-evolutionary closures
Let the agent correct mistakes and evolve in failure
The above four steps are not just a forward flow. Specula assumes that the agent will make a mistake and therefore specifically designs two interdependent self-evaluation locks. Each failure must bring back new code evidence, execute trajectories or countercases to drive the next round of judgement, rather than a simple retest.
The first is a model-built code consistency loop: trajectory verification ensures that true code behaviour occurs in the model, and model checks prevent agents from loosening the model or weakening the non-variable in order to accommodate the trajectory. The second is a bug replication loop: if the counterfactual cannot be re-placed in the code, the system returns the split state to the previous loop, re-checking the model, plugging or not replicating; if the replication succeeds without visible consequences, it continues to determine whether the nature is too strong, or whether the consequences are disguised by the system's recovery mechanism.
Figure 2: Specula's two self-evaluating locks. The model and code consistency cycle alternates between trajectory verification and model inspection; the bug replication cycle returns unplayable countercases back to model corrections.
67 open source systems, 382 bugs
From databases, distributed systems, to compilers, Specula has found problems that can be replicated in real code. The following GCC case is the best explanation for why these rare concurrent bugs escape conventional testing.
A man who has been hiding in the GCC for five years
The example given in the paper comes from GCC's OpenMP running time library libgomp. When triggered, an external thread has to wait until all the other threads have stopped waiting for the barrier to loop to complete a separation task; however, this awakening path misses the mark that the barrel is still on the task of processing the barrel. When the thread is awakened it does not see the task and falls asleep again, eventually all the threads are unable to complete the barrier. This problem requires very specific thread switching, which has been lurking in the code library for at least five years since its introduction in 2021.
Figure 3: Specula's deadlock found in the GCC libgomp. A wake path omits processing labels, violating the activation invariant "paths carrying wake tasks must be set to processing labels". Source: Specula second half of Figure 9.
Just installing TLA+ tools on a coding agent is far from enough.
The paper also compares Specula to two more direct approaches: one is the original Claude Code, and the other is to add TLA + tools only to Claude Code. In five representation systems, Specula found 62 bugs, the first two methods found 2 and 3 respectively. The gap is mainly due to scenographic modeling, modeling, string code consistency checks, and a closed loop that returns code duplicates by default.
Figure 4: In the same set of five controlled experimental systems, Specula found 62 authenticated bugs, two and three, respectively, of the original coding agent and the coding agent equipped with TLA+ tools.
A more direct change is speed. On 48 projects in the paper, one end-to-end inspection takes 1.43 tons of 9.86 hours, with a median of 3.69 hours; tokens cost $19 tons of 168, with a median of $57.
Currently, the project supports agents such as Claude Code, Codex, Copilot CLI, OpenCode and Pi, and other agents are continuing to be adapted. Users can run on their systems with just two commands, or customize their own scenarios and formalized models.
Written at the end
What's really worth noticing about Specula is that it transforms its long-term, expensive formalization method, which relies on a small number of experts, into a fully automated engineering process that can be started by ordinary developers. For the core software team, it continues to track a new layer of defense against deep bugs beyond testing, code review and static analysis; for formalization method experts, it releases time from repeat modeling to more critical nature design and outcome judgement; for the broader computer practitioner, it's the first time that it allows the use of formalized methods to check the real system bugs.
Introduction by the author
This article was co-authored by Shen Yun, Don Rize, Huang Yu, and Xu Tian.
Subject: Ph.D. student at Nanjing University's computer department, Professor Huang Yu, focusing on the formalization methods, model checks, and regulation generation and validation of generated AI-assisted real software systems.
Don Rize: Senior Research Fellow at the Microsoft Asia Institute, Ph.D. in Computer Science at Nanjing University, focusing on formal verification and system accuracy, with particular focus on concurrent verification with distributed systems and AI-assisted verification.
Huang Yu: Professor in the Department of Computer Science, Ph.D. student assistant at Nanjing University, research areas include distributed algorithms and systems, formalized regulations and verification, correctness and reliability of system software.
Xu Tian: Associate Professor in the Department of Computer Science at the University of Illinois at Urbana-Champaign, where he studies focused operating systems, cloud computing and data center systems, and large-scale system reliability and configuration management.
