Over the past few years, Linux has radically changed natural language processing and computer vision.
Can the physical world have a similar basic model?
There are complex physical laws behind climate change, airflow, material deformation and even electromagnetic phenomena. Traditional scientific machine learning often trains a model for a problem: changing a physical system, changing a space scale, or even changing several variables. The basic model tries to change this pattern: first learning general representations from a large amount of different physical data, and then moving quickly to new problems.
But when subjects come to the real world of 3D physics, it's not that simple. The data generated by 3D simulations is usually enormous. A high-resolution fluid simulation can take up a large amount of storage space, while the data scale required to train basic models is even more surprising. At the same time, different physical systems have different numbers of variables, spatial resolution and evolutionary laws, making it not easy for the same model to really learn the transferable knowledge.
Tadpole put it in this context.
Tadpole: Autoencoders as Foundation Models for 3D PDEs with Online Learning
The paper links to:
Github code:https://github.com/tum-pbs/tadpole
It's not about learning how to live in the future, it's about learning how to live in the world
Many basic physical models use an intuitive training method: to give a state of the past so that the model can predict the future. This is similar to predicting the next word in a language model. If the model can see enough physical evolutionary processes, it may be able to gradually master the laws common to them.
Tadpole chose a different path. It was first trained as a self-encoder: after seeing a three-dimensional physical field at some point, the data was compressed to a more compact potential space, and then attempted to restore the original data. In other words, Tadpole's first lesson in the pre-training phase was not: What happens the next second?
The question is: what is the nature of these complex three-dimensional physical states?
This distinction is very important.
How a physical state changes in the future often depends on specific equations, boundary conditions, and physical parameters. Similarly, the current state may go to a completely different future in different systems, so direct study of dynamics itself is a highly complex task. While rebuilding focuses on the spatial structure of the physical field itself. Since physical systems typically have strong spatial relevance and continuity, autocoders can first learn these more general characteristics and then migrate these knowledge to specific tasks. In this sense, Tadpole's thinking is similar to teaching a model to understand the world: from its beginning it is required to predict what will happen in the next few seconds, rather than allowing it to understand what structures make up the world.
202 TB of data, but not 202 TB of hard drive
The three-dimensional basic physical model also has a much more realistic problem: the data is too big. Traditional methods usually require a large number of numerical simulations to be run in advance, save the results to the disk, and then read them repeatedly during training. But when it comes to the three-dimensional high-resolution problem, this method quickly encounters two bottlenecks: one is storage space and the other is the speed of data reading.
Tadpole did not save all the training data in advance, but instead used online learning (Fig. 1a). In the course of training, the numerical solvers on GPUs continuously generate new physical data in real time, which is then sent directly to the model training, rather than forming a huge static dataset. To avoid simulating the delay in neural network training, the authors also designed a multi-level caching mechanism to allow data generation, communication and model training to be carried out in parallel. Ultimately, Tadpole's training process uses the cumulative use of about 202 TB of training data without the need to store such a large data set locally.
Figure 1:Tadpole's overall framework. On the left, it shows online pre-training processes: GPU solvers continue to generate 3D physical data for self-coding training after pre-processing and caching; on the right, it shows how pre-training models migrate to different tasks such as reconstruction, dynamic prediction, and modeling.
The same model, facing a completely different 3D world.
In reality, three-dimensional physical data is not uniform in size. Some systems have only a few variables, some may contain more than a dozen variables; some data have dozens of grid points, and some may have a spatial resolution of 10243.
Tadpole's solution is to learn how to disassemble complex three-dimensional data into a single local area. During pre-training, different physical variables are first disassembled and then randomly cut the local area from three-dimensional space as a training sample. Thus, the number of different variables, different space sizes of data can be unified into the same training framework.
Figure 2: Tadpole-B results in zero-sample reconstructions of various non-pre-trained three-dimensional physical systems, including homogeneous flow, hydrogen flow, transition boundary layer, and magnetic flux mechanics, with a maximum spatial resolution of 10243.
In the experiments, Tadpole's test data came not only from different physical problems, but also from much higher resolutions than those seen in model pre-training. The experiments also showed two clear trends: the larger the model, the better the zero-sample reconstruction capability; the more physical systems exposed during pre-training, the greater the generalization capability in the face of new problems. In other words, what Tadpole learned was not just a technique for simulating a task, but closer to a three-dimensional physical representation that could be moved.
Why would a model that only learns to rebuild a dam predict the future?
To solve this problem, the authors further propose the Tadpole-DFT (Fig. 1b). It does not roll back the already-trained model, but adds a small amount of new learning ability to the already-represented one. First, a lightweight subnetwork learns the interactions and time changes between different physical variables in the potential space; second, the model reintroduces the skip connection, passing precise spatial information directly to the decoder; and finally, through LoRA, only a small number of parameters, rather than re-training the entire network.
Figure 3: Tadpole's performance on three-dimensional dynamic prediction tasks. Through Tadpole-DFT, the pre-training model can be adapted to complex three-dimensional flow prediction tasks using fewer trainable parameters and compared with multiple existing physical base models.
In several three-dimensional fluid flow prediction experiments, Tadpole demonstrated competitive performance. Particularly in fluid flow boundary level missions, some Tadpole models still achieved lower prediction errors with far fewer traceable parameters than in large baselines. The experiment also showed that Tadpole-DFT, a specially designed model, could more efficiently use the already learned representations to fine-tune the full parameters of the entire pre-training model than simply.
Not only does it predict, it can even create new physical fields.
The potential space Tadpole learned had another use: generating new three-dimensional physical data. The authors further trained a generating model in this post-compression potential space to make the model master a new potential representation and then restore it to a complete three-dimensional flow field through Tadpole's decoder. The biggest benefit of doing so was that modeling was not done directly on an extremely large three-dimensional grid, but that the main calculations were done in much more compact space. In three-dimensional flow generation experiments, the generating model built by Tadpole based on micro-modulation achieved good results on multiple statistical indicators. Thus, the same training model began to demonstrate a very interesting multifunctional model: it could compress tasks and reconstruct the physical world, learn how to physicise the world, and also indicate how to generate new physics.
From a grid of rules to a real geometry of complexity
Tadpole initially studied the main rules of physics data on 3D grids, but two of the core concepts - re-constructing grids as a pre-training goal, and generating synthetic physical data online during training - did not stop in this scenario. In another follow-up work from the same subject group also received by NeurIPS 2026, Geometry Meets Physics: Data-Efficient Pre-Training For Unstructured Neural PDE Solvers, this idea was further extended to non-structural grids and complex 3D geometry.
Figure 4: Tadpole's follow-up work on the predictive performance of GMP pumps in vehicle and aircraft outflow problems. Pre-training models (PT) significantly reduce predictive errors in complex geometry.
Such problems are closer to real engineering scenarios: geometric shapes such as automobiles, aircraft, turbine machines are difficult to arrange neatly in a rule grid, often requiring the use of irregular computing grids. This follow-up is pre-trained from both geometric and physical perspectives, respectively: for static problems, the geometric shape itself learns the representation of complex three-dimensional regions; for instantaneous problems, Tadpole's online learning theory continues to generate synthetic physical data in real time during training, thus reducing reliance on expensive precomputing datasets. Experiments have shown that this pre-training method can lead to faster convergence, higher data efficiency, and better fine tuning in data-limited scenarios.
From predicting the future to understanding the world
Tadpole explored a slightly different path from the traditional basic model of physics. It did not require pre-training models to initially master the time evolution of all complex physical systems, but to first learn a common physical representation by reconstructing it; data does not have to be generated and stored on a disk in advance, but can be generated in real time with training. This representation can then be used to reconstruct, predict dynamics and generate modeling, and extend further to non-structural grid and complex three-dimensional geometry.
If AI really wants to be the basic model for understanding the physical world, does it have to be able to predict the future?
Perhaps there is another possibility: first let it learn how to represent the world and then teach it how to change the world.
Author's note
The first author: Zhang Zheng, a fourth-year PhD student at the Technical University of Munich, whose main research focuses on the production of physical base models and physical constraints, is currently actively looking for positions in the domestic industry.
Communication author: Nils Thuerey, Professor for Physics-based Simulation at the Technical University of Munich, winner of the Academy of Science and Technology Award for AI and deep learning approaches aimed at PDE simulation.
© THE