As the complexity of research into network systems increases, research results often require researchers to replicate the entire system according to the paper description. However, the open source implementation in the network field is low, and the system form is highly heterogeneous, involving algorithmic logic, protocol processes, experimental configurations, and different programming languages and operating environments, such as Python, C++, Go, and P4. This makes manual replication not only costly and cyclically long, but also easily biased due to the lack of implementation details.
In collaboration with the team of Prof. Jo, Ph.D., Ph.D., Ph.D., Ph.D., Ph.D., and Ph.D. students from Shanghai Transportation University, Beijing Postal University, JNU, and Qingdao University, the RepLLM is a multi-intelligent framework for network research and replication. It can proceed from the contents of the paper, automatically complete information analysis, system architecture design, code generation, and audit repairs. In seven representative network system replication tasks, the RepLLM generated all the code can be run and successfully uploaded to real data sets; after a limited calibration, complete artificial replication can usually be completed in about two hours. The first author of the paper was Ph.D., Ph.D., a first-year master's degree student at the University of Xiamen.
The paper will be published at the top conference in the field of computer networking, ACM SIGCOMM 2026. ACM SIGCOMM is the flagship academic conference in the field of computer networking, CCF A.
RepLLM: Toward Automatically Reproducing Network Research Results
The website of the meeting:
A preview of the paper:
Background
You can read web papers, but the code doesn't run?
In computer network research, reproducing a paper is often more difficult than reading it.
While papers typically present only system architecture, core algorithms, and experimental results, they can't fully accommodate modular interfaces, data formats, environmental configurations, script organization, and a large amount of engineering detail. It may take weeks or even longer for researchers to realise an operational system, even if they understand the paper.
A more realistic problem is that many papers do not disclose the full code.
Based on the open-source nature of the 2016-2025 SIGCOMM and NSDI papers, the results show that only about 38% of papers in the last decade have been made publicly available to authors. At the same time, an average network system paper needs to reproduce 2.29 existing systems as a comparison, and research costs are extremely high.
Lack of code, structural heterogeneity, environmental complexity, intermodal dependency - all of these factors together constitute a high threshold for network research to replicate.
Although large models have already demonstrated strong code generation capabilities, simply handing over the entire paper to one model and requiring it to generate the entire codebase at once is not ideal: long papers are easy to miss critical information, multi-file code is easy to generate interface conflicts, and code that can run successfully does not necessarily actually implement the algorithms in the paper.
How did it break?
RepLLM's answer is not to have one big model do all the work, but to organize a team of AI replicators who can execute and repair code.
RepLLM：
From paper to executable system
RepLLM is an end-to-end multi-intelligent reproduction framework for network systems papers.
After the user provides the paper and related data, the system completes four stages in a row:
Content analysis: extracting chapters, graphs, formulas, algorithms and their relationship to each other from the paper;
Architectural design: Transforming the paper descriptions into modules, interfaces, execution sequences and environmental dependencies;
Code generation: generating code in stages based on structured algorithmic logic and data contracts;
Auditing and Repair: Performing static checks and sandbox detection problems and automatically generating patches.
The end result is not just scattered snippets of code, but a system-level code repository containing input scripts, module implementations, config files, operating environments, and experimental processes.
We have four AI intelligences.
Creating a paper to recreate the team's work
The core of RepLLM is the collaborative work of four types of professional intelligence.
1. Content Analysis of the Intelligent Body: First, really read the paper
Content analytics isn't just about converting PDFs into a long piece of text, it's about building a structured document knowledge tree.
Chapters, charts, formulas, and algorithms are independently identifiable and linked by reference relationships. When subsequent modules need to implement a function, the system can retrieve the contents of papers related to that function, reducing the problem of loss of information in the context.
2. Architectural design intelligent agents: first design, then code
Architectural design minds transform the high-level descriptions in the paper into system blueprints, including module allocation, execution steps, input output interfaces, data structures, numerical constraints, and environmental dependence.
This step addresses the problems of interface drift, re-definition, and loop dependence that are common when large models generate multi-file projects.
3. Code generates intelligent agents: using structured reasoning to connect papers and processes
RepLLM introduces structured thinking chain SCoT, converting the natural language algorithms in the paper into intermediate representations with input-output types, conditional branches, loop logic, and data dependence, and then generating specific code accordingly.
It is equivalent to adding a layer of verifiable engineering contract between the syntax of the syntax and the syntax implementation of the syntax: both maintaining algorithmic logic and not prior to binding specific syntax to a particular programming language.
Auditing and Repairing Intelligences: Let the Code Really Run
RepLLM will run the generated system in an isolated sandbox and use two layers of repair mechanism:
Static audit is responsible for examining syntax, interfaces, dependencies, and semantic biases between papers and code.
Dynamic fixing is based on running log handling port conflicts, dependency deficiencies, data format errors and compilation failures.
They form a closed loop that generates hydrogen, checks hydrogen, runs hydrogen, and repairs hydrogen, making the system look like a code hydrogen further from the hydrogen to the hydrogen capable of performing experimental hydrogen.
Figure one: RepLLM system architecture
Sharing memories:
It's the same system that keeps many intelligent agents facing the same system.
The most common problem with multi-intelligent systems is that they have different versions.
RepLLM designed explicit shared memory for this purpose: the contents of the paper are organized in the paper space, while the system architecture, interface definitions, code status and repair records are kept in the system space.
The dissolution results of the experiment showed that after removing the shared memory, the system could generate a loop running a successful loop without actually implementing the paper algorithm; removing the interface memory makes it easier for subsequent modules to deviate from the previous interface agreement.
This also reveals an important fact: Smart entities can not only evaluate the number of tokens and whether the program is withdrawn, but also whether the system actually covers the function described in the paper.
The seven representative papers.
It's all about running and data loading.
The team selected seven representative network studies to test, including:
NCFlow for traffic engineering systems;
DNS configuration verification system GRooT;
Rateless IBLT is a collection of coordinated algorithms;
Configuration of the mining system SelfStarter;
NetDice, the probability network verification system;
Learn to accelerate the flow engineering system Teal;
NetChain is a programmable data plane system.
These systems cover traffic engineering, DNS verification, coding theory, network configuration mining, probability verification, and network computing, with original implementations involving Python, C++, Go, and P4, and relying on different tools and operating environments such as Gurobi, CUDA, Mininet, BMv2.
The results of the experiment showed that the code generated by the RepLLM was able to run on all seven tasks and successfully load the corresponding datasets of the paper.
In contrast, single-size model APIs call code generation methods cannot run on tasks such as Rateless IBLT, Teal and NetChain; nor can they accurately access real data in tasks such as NetDice. Some general code smart entities (e.g. Claude Code) can generate executable programs, but it is easy to simplify the real system into a simulator, bypassing data planes or experimental environments on which papers depend.
Particularly in complex NetChain replications, the system must allow P4 data planes, Python control planes, clients, and script editing to work together. RepLLM is the closest to end-to-end perfect replication of all evaluation methods and successfully completes packet interactions in a BMv2 testing environment after human calibration.
From the paper to the results,
82 minutes as soon as possible.
The code can run, and does not mean that the results of the experiment are consistent with the original paper.
To this end, the team further allowed the researchers to perform a small amount of manual calibration on automatically generated code. The automated generation phase of the seven tasks took 38 to 84 minutes, while manual repair took 23 to 93 minutes, with a total replay time of 82 to 138 minutes.
In other words, a network system that may have taken weeks to complete in the past is now expected to be operational, customizable and close to the results of the original paper in about two hours.
After the calibration:
NCFlow obtained a total throughput on the main data sets that is almost consistent with the official implementation;
NetDice passed all 11 tests;
GRooT fully reproduces the findings of violations in the reference test;
Teal fully executes the FlowGNN, COMA and ADMM processes;
SelfStarter fully reproduces expected ACL patterns and abnormal detection results;
Rateless IBLT fully enables end-to-end verification of real protocol paths;
NetChain reproduces the synchronous operation of the P4 data plane and control plane in a p4c/BMv2 environment, with core functional logic consistent with the original implementation, while reproducing modern code compatible with Python 3.
Experimental results show that RepLLM can replicate about 95% of the original benchmark results in about two hours, while reducing unnecessary contextual transmission and Token consumption.
Supporting academic efficiency
RepLLM is not just about saving time writing code.
It attempts to create a more complete link to the automation of scientific research:
Paper understanding → System design → Code implementation → Environment building → Running verification → Results replicated
For researchers, RepLLM can help to quickly build a baseline system, validate paper ideas and reduce the cost of cross-directional replication; for authors and reviewers, it is also expected to be a tool for examining paper replicability, generating experimental products, and finding missing descriptions.
Of course, there are still limits to the capabilities of RepLLM. Requirements to modify the kernel or drive, rely on dedicated hardware and large-scale clusters, or have to patch on large existing projects such as Kubernetes, LLVM, are currently difficult to fully verify in lightweight sandboxes. In the future, RepLLM will gradually improve its functionality.
RepLLM's research shows that building dedicated agents around specific research tasks can integrate large model capabilities more deeply into academic research processes.
The value of a research agent lies not only in the automatic generation of code, but also in the understanding of scientific research tasks, the organization of expertise and the process of research. It can help researchers complete time-consuming work such as document analysis, system construction, environmental configuration, code debugging and results verification, reduce repetitive engineering inputs, and allow researchers to devote more effort to problem discovery, method design and theoretical innovation.
In the future, the Research Agent will also be able to further serve links such as literature research, paper reproduction, network system design, experimental execution, data analysis and research writing, forming a chain of intelligent tools covering the entire process of network research.