This time, the Shanghai Artificial Intelligence Laboratory (shortened to: Shanghai AI Laboratory) opened not only open source model parameters, but also the "black box" of large model research and development. The full set of Checkpoints, training logs, internal architecture adoption and replacement of the entire process experiments are open to the public.
Farewell to "The Way News": Why do we need an open source paradigm revolution?
Over the past few years, large-scale model architectures have evolved rapidly and new approaches have emerged. But what attempts are really effective, which are ultimately abandoned, and on what basis are compromised, the outside world often does not know. For the work of LLM architecture innovation, if anything is more important than a new idea, it may be an open, complete verification process, and an accurate record of successes and failures.
Another reality that cannot be ignored in today's advanced big language model (LLM) is that cutting-edge research and development is increasingly concentrated in the hands of tech giants who possess vast computing resources.
The pace of research and development of the tech giants is extremely fast, and the traditional academic community, from experiments, writing papers to posting, reviewing and publishing long cycles, stands out in the face of the industrial revolution on a daily basis.
This has led to an embarrassing phenomenon: cutting-edge technological details and R&D experience in AI are increasingly dependent on private communication, industry rumours, and even "mini-message" dissemination. Both academia and industry lack an open, standardised and reliable public innovation platform that can fully trace the results of experiments.
This issue is particularly prominent in basic model architecture innovation. Each generation of open source model releases often brings with it new architectural designs, but the exploration process behind these designs, which schemes have been verified, which attempts have ultimately failed, and why replacements have been made, is rarely fully publicised. A seemingly effective innovation for how large-scale models can remain effective after a complete training process, brings with it actual gains or additional costs, and often lacks public, verifiable experimental results as support.
Big model development shouldn't be a "black box" of just showing results, hiding processes. For this reason, the Shanghai AI Lab team announced that they will adopt a new development and open source model: not only opening the final model parameters, but also gradually disclosing experimental records, architectural samples and verification results during the pre-training process, pushing Big Model pre-training from "results open source" to "process open".
ArchSpace architecture innovations open to verify the visibility of the web
The official website:
This is the first time I've seen this.https://github.com/InternLM/archspace）
Weights & Biases (W&B) log recording platform:
Huggingface model weight collection platform:https://huggingface.co/ArchSpace-Collection
It's not just open source final parameters, it's open source research and development of life lines!
Looking at the open source practices of large models at home and abroad, the so-called "open source" remains largely at the result level: publishing the final model weight, or providing a technical report, as far as trial-and-error experiences in research and development, structural weights and failure experiments, which are often difficult for the outside world to understand.
The team believes that true openness is not just sharing "the end", but sharing every footprint of exploration.
In the view of the students' team, open source should not only deliver the final results, but also the process of producing the results as public as possible. The Shanghai AI Laboratory said that it will gradually open up all aspects of model R&D and training, including:
Checkpoints and complete training records of open source models: real recovery models grow from zero to one trajectory;
Open-source exploratory small-scale experimental processes and conclusions: attempts at all single-point architectural innovations in open-source internal evolution;
Open Source Architecture Options and Alternative Logics: Why choose this scheme? Why abandon that architecture? So that the experimental data and basis for every R&D decision can be traced;
Open Source Complete Verification Process: Community-proposed architectural innovations will be fully tested for pre-training, instructional mitigation and reinforcement learning in a unified process, rather than confined to local experiments or single indicators.
ArchSpace Platform: Intuitive visualization architecture is evolving
In order to implement this "fully open" model, the Shanghai AI Lab will focus on innovation in Foundation Model Architecture, including but not limited to:
It's the first time I've ever heard of this kind of thing.
Latent Space Models, also known as Latent Space Models
This is the first time I've been able to do this, and it's been a long time.
At the same time, the supporting open-source visualization platform ArchSpace was officially launched.
On ArchSpace, the Shanghai AI Lab will be the starting point for a classic Decoder-only Transformer that fully visualizes all subsequent single-point architectural innovation and iteration processes from that starting point.
As an example of a new Attention program, users can see clearly on the website:
How the laboratory gradually grew from small-scale experiments;
Specific experimental data and comparative results for each stage;
The size of the model and the key indicators to be verified next;
The ultimate reason for the decision whether to integrate into the mainstream of R&D.
Every innovation is verifiable; every success or failure is completely traceable.
ArchSpace architecture proposals and implementation processes: from community submission issues, reviews and voting, to branch implementation, phased experimentation and merger.
Specifically, if the community is interested in the real effects of the "Next Concept Prediction" (NCP) approaches proposed by recent architectural innovation papers, the architecture proposal issue can be submitted to the GitHub repository of ArchSpace.
After the committee's review of the proposal, ArchSpace will advance scaling and verification work in phases, followed by performance evaluations at mainstream model sizes such as 1B, 3B and 8B. The largest verification to date will align the full training process of the open source Olmo 3 model: covering pre-training, continuing training and following instructions in the context, with a training scale of approximately 6.2T tokens.
The committee is composed of experts from academia and industry, and is currently under construction, with current members including Linzhou Han Associate Professor, Shanghai Transport University, Changchun Professor, Changchun Assistant Professor, Changchun Assistant Professor, Shanghai AI Laboratory Youth Research Team, and others.
The Jury will review peer-reviewed proposals for architectural innovation submitted by the community, focusing on the three dimensions of innovation value, technological feasibility and experimental design completeness, and select proposals with verifiable value into the formal verification process of ArchSpace.
ArchSpace architecture verification process: aligned with the Olmo 3 training process, covering model sizes 1B, 3B, 8B.
ArchSpace Training and Evaluation Log Visualization
Currently, ArchSpace has put together more than a dozen architectural innovation proposals such as Next Concept Prediction [1], Depth-Attention [2], Siamese Norm [3], Artificial Hippocampus Networks [4], MorphNorm [5], Mobius [6]. These proposals are awaiting a "complete runway" for the same set of open, complete training and evaluation validations.
According to the Shanghai AI Lab, the ArchSpace Committee remains open and invites more experts from academia and industry to join in discovering and promoting authentic architectural innovation.
The following papers or proposals related to architectural innovation:
[1] Liu, Yuliang, et al. "Next Concept Prediction in Discrete Latent Space Leads to Stronger Language Models." arXiv preprint arXiv:2602.08984 (2026).
[2] Zeng, Boyi, et al. "Depth-Attention: Cross-Layer Value Mixing for Language Models." arXiv preprint arXiv:2606.05014 (2026).
[3] Li, Tianyu, et al. "Siamesenorm: Breaking the barrier to reconciling pre/post-norm." arXiv preprint arXiv:2602.08064 (2026).
[4] Fang, Yunhao, et al. "Artificial hippocampus networks for efficient long-context modeling." arXiv preprint arXiv:2510.07318 (2025).
[5]
[6]
Resources are limited, but exploration is limitless: a new paradigm for scientific research organizations in the Big Model era
Based on existing computational conditions, Shanghai AI Lab will be the first to test and deploy this new development and open source model at the 1B/3B/8B parametric level.
Although the current scale is still lightweight, this indicates that the laboratory is actively exploring and building a new paradigm of scientific research organization to fit the big model era:
The technical details of big model training go from "black box" to "totally open";
Making the impact assessment of specific innovation points move from "channel messages" to "publicly referenced, traceable";
Let every successful and failed experiment be the next innovative and reusable knowledge asset in the community.
This is not only an in-depth test of key modeling architecture innovation by the Shanghai AI Lab team, but also a gift to the global AI research community, looking forward to witnessing a whole new phase of the open source model together.