The first author and correspondent for the paper are from Beijing University's Wang Choi Computer Institute, the first author is Ph.D. student Liu Xiaobo, and the correspondent is Ph.D. student mentor Liu Yi. In recent years, the team has published several representative results at summits such as TPAMI, IJCV, CVPR, ICML, won multiple national and international multi-modal understanding and generation competitions, and has extensively collaborated with renowned universities and research institutions at home and abroad.
This article mainly introduces the latest paper from the team Music-to-Dance Generation via Atomic Movements.
The work focused on music-driven dance-generation tasks, first proposed as a basic unit of dance with atomic movements, established dance representations with clear semantics and structure, and designed a two-stage framework for "action planning, choreography, dance generation" to make the dances generated not only more in line with musical rhythm and overall choreography logic, but also with good explainability and editability, providing new research ideas for intelligent choreography.
Currently, the work has been officially received by ECCV 2026, and the corresponding code and model are all open-source.
The title of the paper is Music-to-Dance Generation via Atomic Movements.
Link to the paper:https://arxiv.org/abs/2607.13978
The code link:https://github.com/oceanflowlab/AtomicDance
The main page of the project:https://cxhcmhhh.github.io/AtomicDanceProject/
Why can AI dance and not dance?
In recent years, the development of scattered models has led to rapid advances in music-driven dance production. Injecting a piece of music, the model can produce corresponding three-dimensional human dance movements. However, if you look closely at real dance, you will find that the dance is not a simple spelling of continuous movements, but has a distinct structural structure. Whether it is street dance, jazz or modern dance, a complete dance usually revolves around a number of clearly defined movements: the movements of the wrists may be repeated several times with different music stages; a rotating movement may be constantly changing speed, pitch and direction; multiple basic movements are continuously combined in a certain order to form a complete dance.
However, most current mainstream methods view music-to-dance generation as end-to-end sequential generation problems, directly predicting each human gesture. Although this method is able to learn local rhythmic correspondence, it is not possible to understand the structural organization behind the dance, so it is easy to cause a long-term lack of overall consistency, repetition of movements but lack of change, difficulty in understanding why the model produces a certain movement, and user difficulty in editing the already generated dance, etc.
Therefore, the question of whether AI should first learn to "dance" and then learn to "dance" is the starting point for this paper.
It's about breaking dance down into "atomic action".
To this end, the team proposed a new concept called the Atomic Movement. The team believes that a movement that is truly capable of acting as a dance basic unit should meet three characteristics:
(1) The complete process of movement, rather than a single gesture. For example, a calf movement involves not only the moment of lifting the calf, but also the entire process of preparation, strengthening and tightening the calf, so as to express the meaning of the entire movement.
The same movement can be repeated over and over again throughout the dance as the music changes in speed, direction, breadth and intensity.
Each movement can be described in natural language, such as "lifting up your right hand", "stretching out your arms", "spinning your body", and so on, so that the movement can be understood, retrieved, and edited.
Figure 1: Extract atomic movements from existing dance data.
To automatically detect these atomic movements, the team designed a three-phase build process, as shown in Figure 1:
Movement segmentation: as shown in Figure 1 (1), the sequential dance is first divided into complete movement events rather than fixed length segments;
Clustering of movements: using motion characteristics to detect a large number of repetitive basic movement patterns, as shown in Figure 1 (2).
LLM Semantic Re-clustering: In-group Re-clustering: as shown in Figure 1 (3), further uses large language models to analyze the particle semantics within each cluster to give explanatory natural language labels for atomic motion.
Eventually, the model obtained a set of repeatable, combined, and explainable atomic movements that provided the basis for subsequent choreography.
Simulating a real dance process: planning first, then completing the dance
Based on atomic motion, the team further proposed a two-stage structured generation framework, as shown in Figure 2.
Figure 2: A complete overview of the method.
The first stage: movement planning. As shown in Figure 2 (1), the model first fully understands the entire piece of music and predicts the location, type, and duration of each atomic movement.
The second stage: Dance complementation. As shown in Figure 2 (2), the model selects prototypes of movements from the corresponding atomic movements according to the planned results, and uses the natural welding between the spread models to generate movements, while making appropriate changes to the repetitive movements, so that the entire dance is both holistic and performance-rich.
This design perfectly simulates the creative process in reality: the choreographer is responsible for planning the movement structure and the dancer is responsible for completing the specific performance. Compared to traditional end-to-end production methods, the model is able to make full use of the entire musical information, generate a more consistent musical structure, more coherent natural dance, while supporting users to directly modify the atomic movement, achieving true editable generation.
The results of the experiment: creating dances that are more natural, more in tune with the musical structure
The research team made a comprehensive comparison on the AIST++ data set with current mainstream music dance generation methods.
Table 1: AtomicDance's quantitative experimental results show the best performance of rough bodies.
The results of the experiment showed that the method had a leading performance in a number of indicators, such as dance authenticity (FID), rhythm consistency (Beat Alignment Score) and structural consistency (R-precision). It was seen that the proposed atomic movement represented not only a significant improvement in the overall structure of the long-term dance, but also brought the results of the production closer to the actual choreography process. At the same time, the study also conducted dilution experiments, such as Table 2, which verified the effectiveness of atomic movement recreation on dance quality, structural consistency and musical consistency; Table 3, which verified the performance differences of the various strategies proposed in the study when selecting atomic movements, and confirmed the optimal overall scheme of selection based on the length of atomic movements.
Table 2: Dissolution experiments for the re-creation of atomic motion.
Table 1: Dissolution experiments for the selection of atomic motion.
Figure 3: Comparison of qualitative results.
In addition, qualitative results such as Figure 3 and the video below further validate the superiority of this method over existing methods.
From "generate action" to "understanding dance"
The work further explores whether AI can learn the structural laws of human choreography rather than the existing methods of choreography itself. By introducing the intermediate of atomic movement, the model can explicitly learn how dance is composed, allowing dance production to evolve from mere movement prediction into a choreographic process with structural planning capabilities, and allowing for better explanation, editability, and control of the results of choreography.
The team believes that this work provides new technological pathways for the future of digital performance, intelligent choreography, collaborative art creation, and new ideas for structured motion generation and explanatory model research.