The pixel tells us how the world is presented, and the physical code explains why the world works this way.
To create a world is not to understand a world.
Today's AI is capable of producing realistic videos and coding complex 3D scenes. But whether it's a pixel or a scene, it's just a mirror image of the world.
In other words, the next step in AI is not just to predict what the world will look like, but to restore why the world will work like this.
Recently, MirroS has proposed Code-as-World: allowing AI to proactively reconstruct the observed reality into an executable, verifiable, and reasoned code world, and to use it as a representation of understanding the physical world. Under this framework, objects, states, physical parameters, and laws of evolution are explicitly written into code, so that an interpretation of the world is no longer merely a linguistic description or metaphorical feature, but a world hypothesis that can be queried, executed, intervened, and verified.
When it comes to predicting the future directly from observation, the answer is: What will we see next?
Code-as-World then asks further: What kind of world is capable of producing these observations and continuing to evolve after the intervention?
Code-as-World writes executable codes for physical composition, dynamic evolution, and visual appearance, and allows intelligent agents to continually propose, operate, test, and modify world hypotheses around observations. These verified executable worlds can be transformed into scalable physical supervision, promoting visual language models to learn physical reasoning.
The main page of the project:https://mirros-lab.github.io/code-as-world
The paper:
The code is:https://github.com/mirros-lab/code-as-world
Full article Blog:
It's not the same as understanding correctly.
Human beings understand the physical world and do not remember every specific observation.
Faced with balls of different sizes, slopes of different angles, and terrain of different materials, humans still use the concepts of objects, mass, speed, gravity, and friction to think.
This also means that reproductive observation does not constitute an understanding of the world. As long as the future generated is visually reasonable enough, pixel prediction itself does not require models to explicitly answer: what exists in the world, how states change, and what mechanisms dominate these changes.
The question then goes further: what is the symbol that really supports understanding and reasoning about the physical world?
Several types of world representations exist, each of which retains some of the world's information.
The pixel retains a wealth of detail, but hides the cause in the results; the 3D reconstruction restores geometry, depth and perspective, but does not automatically explain why objects fall, collide or stop; the natural language can express objects, movements and relationships tightly, but it is difficult to accurately carry continuous trajectories, contact relationships with physical parameters.
Code-as-World hopes to find a new point of contact:
It is as semantic as language, as structured as 3D, and as capable of continuous evolution as video.
Code-as-World provides a form in which objects, states, parameters, and rules can be explicitly written; the world can not only be described, but also executed, intercepted, and verified. Code-as-World does not pursue a complete copy of each pixel, but rather prioritizes maintaining the world's composition, constraints, and evolutionary mechanisms.
What is really important is not just to recreate the world as it is, but to find an expression that can explicitly support the structure and mechanisms of the world.
It's not a more real pixel, it's a world that can be executed
If you want to abstract a physical world with code, what do you need to write?
Code-as-World divides a world into three interrelated aspects:
Composition: What is there in the world that describes physical properties such as objects, geometry, and mass, friction, and gravity.
Evolution: How the world changes, describing the state of objects, movement, interaction with key events, and how the world evolves over time.
Appearance: how the world is observed, how the camera, background, material, lighting, and other visual conditions are described.
Combined, Code-as-World no longer represents a landscape, not just a trajectory, but a world that can actually function.
The different components can also be viewed and modified independently: changing an object's speed, mass, motion direction or camera position, while other structures can remain unchanged, and the world is subsequently re-executed.
The code is not the ultimate answer here.
It's more like a world hypothesis that can be run and falsified.
03 Let the Agent Discover the World in Simulation and Verification
Retrieving the underlying physical world from a video or text is essentially a complex reverse problem.
Therefore, Code-as-World does not view the world description as a one-time code-generating task, but instead constructs it as an agentic discovery loop: allowing the Agent to gradually approach a world that can explain observations in the process of constantly proposing hypotheses, simulations and validations.
This process is based on the trace factor reasoning of scientific discoveries: never incomplete, noisy evidence emerges, constantly looking for the best explanation of observations, while maintaining a concise mechanism hypothesis.
First, different forms of input are converted into the corresponding evidence.
For text, the system extracts objects, spatial relationships, physical events and expected results; for real video, further extraction of depth, instance splitting, object trajectories and three-dimensional geometry, together binds to executable world representations.
After that, Agent went through five successive phases:
Propose a world hypothesis: to generate or modify world characteristics based on current evidence and feedback.
Instantiate for instance: Compile an abstract world hypothesis into an executable program.
Execute to execute: run a simulation to get a complete world state trajectory.
R