Introduction by the author
The first author of this article is Li Dong, a first-year PhD student at the Department of Computer Science at the Chinese University of Hong Kong, who was selected for the LongCat Beijing Program, the Tencent Mixed Cloud Program, taught by Prof. Pheng-Ann Heng; the author of this communication is Professor Pheng-Ann Heng, Professor of Computer Science and Engineering at the Chinese University of Hong Kong.
Making the video look real is not the same as making it sound like common sense.
After the oil falls into the hot tub, the sealed plastic bottles are drawn out of the air, but the seals remain unchanged, and the dry ice at room temperature is always the same. The images may be sufficiently refined to make a joke. For video production, this means that the lens lacks credibility; for physical and physical simulation, the problem is more serious, because the model needs to understand not how the object smoothly moves, but why the event happens and how it will continue.
Existing text-based video models usually generate images directly from a highly compressed pronoun. The pronoun tells the model what's going on, but rarely alternates with the speed of change, the sequence, the interaction between the objects, and the state that should be presented at the end of the process. The model has to fill in the implicit information and synthesize the video. Researchers from the Chinese University of Science and Technology of Hong Kong and the Chinese University of Technology and Technology of China summarized this gap as a cause-and-effect opacity gap: human intentions are compressed in short sentences, but the video needs to unfold into a complete, continuous time-space process.
VideoCoCo's entry point is very straightforward: instead of allowing the model to be generated in the pixel space with experimental guesses, instead of using a piece of code that actually works, writing out physical processes that are omitted from the prompts, and then generating video accordingly. The code here is not an add-on tool, but an executable, verifiable chain of thought.
Comparison of the traditional video paradigm with the VideoCoCo paradigm
VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation through an Agentic Dual-Engine System
Link to the paper:https://arxiv.org/abs/2607.27380
The code link:https://github.com/micky-li-hd/VideoCoCo
From CoCo to VideoCoCo:
Expand a preview into a process
The idea for this work did not come out of nowhere. The team's previously proposed CoCo ((Code-as-CoT for Text-to-Image Preview and Rare Concept Generation) has been embraced by ECCV 2026. Faced with complex spatial relationships or rare concepts, CoCo is not in a hurry to generate the final image, but instead writes an executable program, rendering a preview map containing layout and object relationships, and then adding details to the image model around the preview map.
The value of this step is that the reasoning that was originally left at the language level is translated into a visual result that can be run, viewed and modified. Where it is wrong, whether the relationship is established, it is not necessary to wait until the final picture is finished and guess the reason, but can go back to the program and the preview map.
VideoCoCo continued this idea, but video has more of a time axis than an image. A picture only needs to solve a momentary spatial layout, and a video needs to alternate continuous state changes. So the code can not only describe where the object boils, but also how it boils down: when it starts to melt after being heated, how the shape unfolds, and how the container changes in internal and external pressure.
Map of the CoCo pipeline
It's a two-way street.
It's a great way to get a good look at the chicken and the chicken.
VideoCoCo uses a two-engine framework that breaks down process reasoning and visual presentation into two relatively independent nodes. The coding agent is responsible for determining how the event occurs, while the visual agent is responsible for transforming the process into a realistic image. This division is not complicated, but avoids the pressure of having a model simultaneously undergo physical projection and visual synthesis.
For example, Mr. Coding Agent would create a self-contained bl.