It's not the hardest thing to have a video-generation model explore a scene continuously along the camera's trajectory. The real difficulty is: when the lens goes around again, is the layout of the room, the location of the furniture and the texture of the walls the same as before?
Existing video world models can produce excellent visual effects in short films. However, as the production process continues to extend, early images will gradually leave the model in a limited time context. The model may forget the space it has seen: the sofa changes materials, the doors and windows move, and even the geometric structure of the room changes.
One direct solution is to preserve all of what has been observed in the past. However, whether it is the preservation of RGB images, three-dimensional dots, or the potentialities of the scattering model, memory will continue to grow as the length of the video continues. The further the model goes, the more history needs to be preserved and processed.
A team of researchers from Harvard University, MIT, MIT, MIT, and MIT's IBM Watson AI Lab proposed the Honeycomb in an attempt to answer a more fundamental question: Can video world models maintain a fixed size of feature memory while continuously absorbing new scene information?
Honeycomb's answer is HexMemory. It compresses scenes into six fixed-sized two-dimensional feature planes and continually updates the same memory through a loop of reading, generating, and writing feedback.
Figure 1: Closed-loop re-visit dynamic contrast. From left to right, Honeycomb, Spatia and LSM World respectively. After the camera leaves and returns to the starting point, the Honeycomb keeps the room structure and appearance of the object better.
Title and author information: Jack Wei Lun Shi and Kaichen Zhou co-authored the first paper.
Title: Honeycomb: Constant-Size Scene Memory Representation for Video World Models
It was written by Jack Wei Lun Shi (Co-first), Kaichen Zhou (Co-first), Haoyu Chen, Yufeng Weng, Keane Ong, Ruojin Cai, Hang Hua, Justin K.W. Yeoh, Mengyu Wang.
The paper:
The project's main page is: jackswl.github.io/honeycomb
This is the first time I've ever heard of it.
Why does a long video need a piece of memory that doesn't expand?
Video generation models typically only deal directly with the context of a limited length. For shorter shots, the last few shots have provided enough information; but in long-term, open-scenes exploration, the local context cannot answer an important question: What was the scene like long ago?
The existing methods largely use two types of spatial memory. Spatia re-projects the observed RGB content into three-dimensional space, continuously establishing a color point cloud; LSM World associates the scattered model substrate with three-dimensional points, forming substrate caches. These methods allow the model to retrieve historical scenes, but also have one common feature: new observations produce new points or features, and memory size continues to grow as the generation process progresses.
If video is only a few seconds, this growth may be acceptable. But for models of the world that require long-term exploration, interactive environments or intelligent systems, the continuous expansion of memory will eventually become a burden of storage and computation. Honeycomb therefore no longer adds to history but compresses new and old observations repeatedly into a set of fixed structures.
Figure 2: Three spatial memory designs. Spatia accumulates RGB point clouds, LSM World accumulates subtractive features; Honeycomb writes scenes into fixed-sized spatial feature fields. Source: Home page of the Honeycomb project, corresponding paper Figure 2.
Core Design: Remembering an expanding world with six planes
Figure 3: Honeycomb overall framework. The system initializes HexMemory from the beginning, reads memory to generate the next video, and then writes new observations back to the same set of fixed-sized planes. Source: Homepage of the Honeycomb project, corresponding paper Figure 3.
Honeycomb maintains a block consisting of six fixed-sized characteristic planes. The system first enters the initial memory; each time a new video clip is generated, the condition is read from memory and the new observations in the clip are rewritten. Unlike continuous additional potential characteristic points, Honeycomb is always updated on the same set of planes.
1. HexMemory: three space planes, three time and space planes
HexMemory uses three space planes XY, XZ, YZ, and three time-space planes ZT, YT,XT. Here T represents the time at which the content was written into memory.
In order to query the content of a three-dimensional location at the time of writing, the system takes a bi-linear sampling on the corresponding plane and combines these characteristics respectively. The spatial plane preserves the time-shared scene structure, while the time-space plane helps express the content that changes over time. This is equivalent to lower-order decomposition of the field of four-dimensional scene characteristics, which is more compact than directly maintaining a complete four-dimensional grid or growing point caching.
Forward-looking typewriter: only deals with new observations that have just arrived
The system first uses VAE to encode the image as a substrate, then combines the depth with the camera parameters to elevate the substrate to a three-dimensional world coordinate. Each three-dimensional point carries its own location, substrate, camera center and observation radius direction.
In addition to characteristics, each plane maintains a confidence chart, recording how much effective observation the different units received. These confidence levels will determine how much weight old and new information should each have in subsequent updates.
Figure 4: Forward-facing typewriters project three-dimensional dots onto a two-dimensional plane and assign characteristics to four adjacent units with binary weight. The operation is performed on six planes respectively. Source: Homepage of the Honeycomb project, corresponding paper Figure 4.
3. Cyclical updates: extend the coordinate range but not the volume
When new segments cover a larger space or time span, the Honeycomb does not increase the size of the plane, but instead re-mapped the old plane to a new boundary. The fixed grid can therefore cover a larger world; the cost is that the effective resolution of the unit space will decrease after the scope is expanded.
After the change, the old features are merged with the new features of the current sections in a unit-by-unit manner, and then corrected by a small network prediction remainder. The writers only process the new sections and do not need to re-read the entire history, nor do they need to perform gradient optimization for each scene.
Before producing the next video, the system projected three-dimensional points into the perspective of the target camera, queried HexMemory, and reconstructed potential features. These potential features and visibility masks entered the video spread model via conditional branches similar to ControlNet, allowing the generator to re-access scene information that had left the context over time.
Experiment See the Truth: Returning from the Scene
Figure 5: Closed-loop retrospective qualitative results. The Honeycomb better restores the furniture, textures and spatial layout in the input perspective, while the contrasting approaches have different degrees of structure or appearance drift. Source: Homepage of the Honeycomb project, corresponding papers Figure 1 and Figure 5.
To test whether the model actually remembers a scene, the team used a closed-loop retrospective setting: the model starts from an input image, leaves the initial position along the camera's trajectory, explores other areas, and finally returns to the original perspective. The closer the final image gets to the input image, the more reliable the long-term memory is.
In the WorldScore closed-loop test, Honeycomb achieved the best results on all reporting indicators: closed-loop PSNR reached 17.22 dB, 1.23 dB higher than the second method; SSIM was 0.504, LPIPS was 0.311. The optical flux error between the input image and the final return image was 3.00 pixels, while Spatia was 6.64 pixels.
Table 1: WorldScore closed-circuit return results. ↑ indicates the higher the better, and ↓ indicates the lower the better.
It's not just about remembering the past, it's about creating a more accurate new perspective.
In the RealEstate 10K new perspective synthesis experiment, the model started from scratch and generated three consecutive video clips along the trajectory of the real camera. Honeycomb performed best on all three indicators: PSNR, SSIM and LPIPS.
Figure 6: Honeycomb starts from a single input image and generates three self-returning footage sequentially along the RealEstate 10K camera trajectory.
Table 2: RealEstate 10K New perspective synthesis results
In terms of overall video generation quality, Honeycomb scored an average score of 65.52 on WorldScore, higher than Spatia's 63.21 and LSM World's 61.20.[6] It should be emphasized that it is not the leader in every sub-indicator; its outstanding advantages are fixed feature storage while maintaining a stronger generation quality, new perspective synthesis and repeat consistency.
Fixed sizes don't mean just expensive configurations
Honeycomb has a default resolution of 512 and a characteristic memory plane of 48, corresponding to characteristic storage of about 73.9 MB. Since the capacity is determined by the resolution of the plane and the characteristic dimension rather than the length of the video, applications can choose different configurations based on device resources and quality requirements.
After reducing the flat resolution from 512 to 256, feature storage dropped from 73.9 MB to 19.8 MB, a reduction of about 73%.
In the same setting, closed-loop PSNR decreased from 17.22 dB to 17.10 dB and SSIM from 0.504 to 0.500.
When it continued to fall to 128 resolutions, the memory was only 5.8 MB, but the decline in quality began to become more apparent.
Table 3: Difference between HexMemory resolution and storage mass.
Permanent is not only storage but also writing costs.
The team also compared three methods of updating memory. Direct optimization can achieve a slightly higher rate of return visits, but it takes about 3.2 seconds to write each time; replacement writing re-processes all historical observations, increasing the number of bits of time from 11.2 milliseconds to 40.1 milliseconds.
Honeycomb's loopwriter only processes the latest footage and merges with the old memory. Whether the test is done to the 2nd, 5th or 9th video footage, the writing time is stable at about 13 milliseconds. Compared to direct optimization, it is updated about 240 times faster each time; its writing cost does not increase as the video length continues compared to the complete history of reconstruction.
Table 4: Time and quality of return visits for different memories.
Think about it: the world model doesn't just need the next one.
The meaning of Honeycomb is not merely to propose a smaller cache. It demonstrates a new way of remembering scenes: past information does not have to exist with ever-increasing images, three-dimensional dots or subtractive entries, but can be continuously compressed into a set of fixed structures. The system only processes the latest observations each time, but can still retrieve earlier scene information in a future perspective.
This design also has clear boundaries. As the plane size remains unchanged and the space and time span of camera exploration continues to expand, the fixed grid must represent increasingly larger areas and the effective resolution of unit space will gradually decrease.
Furthermore, Honeycomb still relies on the ability of deep estimates, camera trajectories, and basic video generation models. If early geometric estimates or generated results are wrong, memory updates can also lead to errors in a follow-up state. How to distinguish durable structures from short-lived changes in dynamic scenarios, and how to manage limited resolutions over longer trajectories, are still questions worth studying further.
But this work has shown that long-term video production does not necessarily require a continuously expanding scene cache. Future video world models need to know where they've been, understand which scene structures should remain unchanged for the long term, and restore the same world when they return.
Honeycomb uses six fixed-sized feature planes to provide a compact and efficient implementation of this capability. When the model returns to its original location, it not only generates an image that looks reasonable, but struggles to remember: here, it should still be here.