MiniMax H3 on vLLM-Omni: from system-level optimization to real-time service based on FastVideo Fast

The service of MiniMax H3 is a system problem. One request is to go through a huge Qwen3-VL encoder, a long sequence of audio video DiT, independent video and audio VAE, the boundaries of the device and process, and finally to complete the packaging of H.264/AAC.

So vLLM-Omni starts with a complete live stream: attention to communication, integrated DiT calculators, parallel VAE decoding, compact output transmission, and parallel MP4 construction. FastVideo's FastH3 then handles the remaining commands and switches the 49 DiTs forward to four.

In the eight-card B300 configuration tested, FastH3 produces a full MP4 of 10.125 seconds with a running time of 8.678 to 8.710 seconds. Real-time refers to the full response time faster than its playing time, not to streaming delivery or first-time.

1. Why MiniMax H3 service is a system-level issue

The MiniMax H3 uses text, images, video, and audio as inputs, co-generating video and synchronized audio. Its various components have different requirements for computation, display, and placement:

request -> encoder -> joint audio/video DiT -> video + audio VAEs-> GPU output preparation -> D2H/IPC -> H.264/AAC MP4

Figure 1: The text goes through the H3/Qwen3-VL encoder; the visual and audio conditions also pass through their respective VAE respectively. The condition sub-variable and the noise-bearing target sub-variable form a packing sequence, where the combined audio video is sounded, then decoded and built into MP4 separately.

The checkpoint has been released covering three service missions:

DiT dominates basic scheduling, but it's not the only bottleneck. The encoder's residency will affect capacity; the VAE decoding will appear after the noise is shortened; and the original encoder will still have to cross the process boundary and be encapsulated into MP4.

2. The boundaries between calibration and evidence for benchmarking tests

This article separates the two lines of evidence:

The two lines are both valid and frozen experiments, but they do not share the same source code SHA, prompt, seed and product. Therefore, we do not infer the acceleration ratio from the basic version to FastH3. This article only reports an absolute delay in FastH3 until a strictly aligned set of A/B experiments.

2.1 Control variables of freezing

Both lines are counted from the time of simultaneous submission of the request to the time when the full MP4 is received. Download, start, compile and preheat are excluded from this range. Each accepted output must be decoded as H.264 video plus 32 kHz AAC, containing the expected frequency and frequency, the video bandwidth and audio RMS are non-zero, and subject to consistent human inspection by prompt.

For FastH3, we retain verified video and audio streams and define T_media = max (T_video, T_audio), which is the effective playback time of the full MP4:

RTF_client = T_client / T_media

RTF_client <= 1.0 is a real-time determination of the complete response. Once a media check failure, audio failure, OOM, accelerator error or unexpected regression occurs, the configuration is terminated before repeated measurement.

For the rest of the hardware, we intend to focus on what to do instead of stacking a comparison table: H200 and data center CUDA, RTX PRO 5000, RTX 4090, RTX 5090, GB10 and ROCm.

3. System-level optimization of vLLM-Omni

Basic H3 This line retains the released BF16 weights, 50 sigma points and tight attention coverage. Optimization runs along the execution path, rather than by function list rows.

3.1 Long sequence attention and communication

H3 uses text, audio, and video tokens as a long sequence of packages. Under this typical load, 58,758 valid tokens occupy a coordinated buffer zone of 58,816 tokens.

• TRTLLM_ATTN receives the effective sequence length and further processing of the packing sequence removes structural tail padding.

• Rank local boundaries only construct local embedding/RoPE lines, and gather compact projections of 128 channels, rather than the hidden state of 5,376 channels.

• Fast Ulysses uses NCCL SymmetricMemory to directly swap pieces with the desired layout, saving all-to-all a separate rearrangement before and after.

3.2 Integrated DiT algorithms

The 49-fold forward rotation repeatedly imposes small algorithms around the matrix. vLLM-Omni combines Q/K RMSNorm with RoPE.#5990The FP32 modulation, integration and residual calculations are combined.#6281、#6878It replaced the separate SiLU with a dual launch multiplied by the combined SwiGLU.#6283）。

3.3 Parallel and integrated VAE decoding

After the noise was removed, the H3 decoded video and audio separately. The VAE patch parallelly distributed the tile-based video decoder across eight GPUs. The precise VAE algorithm path speeds up understanding the materialization of the block of code, integrated Q/K integration with RoPE, integrated SwiGLU, and scaled residual updates, and retains eager regression for unsupported layouts.

3.4 Preparation of GPU output, transmission and MP4

A request is not completed before hundreds of tons leave the GPU. Optimized paths allow the conversion to be done only once:

1. The GPU output is ready to convert the decoded FP32 BCTHW to a continuous uint8 BTHWC, reducing the video load by 75% before transmission.

2. This compact load is transmitted from pinned D2H and the IPC from the worker to the engine.

3. Direct planar encoding, a permanent parallel converter, and support for a post-transmission strided RGB plane allow H.264 to directly extract data without having to construct a complete switching RGB buffer zone.

FP32 BCTHW -> uint8 BTHWC -> pinned D2H/IPC -> planar frames -> H.264/AAC MP4

3.5 Baseline H3 test results

Both run with eight B300s, the same prompt as seed, 50 sigma dots, and the same full MP4 boundary. Diffusers use copy weight and parallel native context; vLLM-Omni uses the TP8 encoder; DiT USP8/Ring1 with Fast Ulysses; VAE PP8 tile decoding, and TRTLLM_ATTN.

Examples of MiniMax-H3 model-card

VLLM-Omni baseline

With lossless optimization, vLLM-Omni reduces the total response time relative to Diffusers by 30.8%, equivalent to 1,445 times the acceleration. Here, lossless acceleration does not depend on quantization, scarcity of attention, caching repetition or reducing the number of noise steps. It does not mean that the output is consistent: different kernel implementations with float return sequences can still disrupt the distribution trajectory.

These improvements reduce the expense of de-noise. FastH3 deals with the remaining dominant items, reducing the de-noise cycle itself from 49 down to 4 down.

4. Expanding the general H3 service architecture

The general H3 line contains two different types of production controls. The difference between DLO and encoder is the change in capacity and placement; the preferred quantitative weighting and approximate attention are the change in storage or timing with numerical accuracy. These paths explain how to install, how to expand, and how to accelerate the broader architecture. They are not involved in the generation of the FastH3 numbers in Section 6.

4.1 Distributed layered unloading

DLO (Distributed Layerwise Offload, distributed layer-by-layer unloading) retains a bounded DiT layer window in the HBM, with the remainder drawn from the host's memory flow. AllGather mode builds an active layer of weight from the host's side; rank local mode takes the volume generated by each rank from the regular loader.

Figure 2: DLO prepares the next layer at the same time as the current level of computation. Mechanism and deployment weights see DLO column.

8×B300 BF16 DLO Front of the Pareto

At the official BF16 MiniMax-H3 FL2VA checkpoint ((5,175 seconds, 1344×768, SP8/Ulysses8/Ring1/DP1/TP1 AllGather, CUDNNN attention), the first request for CUDA/cuDNN/JIT work loaded with hydrogen is excluded, while the other two requests are taken.

Figure 3: Time extension with the surviving Pareto frontier. <em>r</em> for the number of permanent DiT blocks. The true center point is the non-dominant measurement value and the empty center point is the controlled measurement value. At <em>r</em> = 35, the DLO decreased the HBM reported by 37.5% at a 5.1% time delay cost; <em>r</em> = 0 is the smallest surviving endpoint.

4.2 Coder separation

H3 requires a Qwen3-VL encoder weight of about 51.5 GB under BF16. The encoder separates the path to move this one-time encoder into a separate vLLM phase, which has its own placement, volume parallel, copy, queue, kernel and prefix cache. The editor assigns the 50th layer of hidden status and token character labels that it produces to the DiT/VAE phase, along with the original media.

Figure 4: Encoder capacity and scattering capacity can be independently expanded. The combined single-node recipe returns condition information through the coder and keeps the scattering phase internally connected; it does not configure OmniConnector. SHM/RDMA remains a future cross-node option in RFC #5707.

4.3 Selective quantification and attention acceleration

Section 3 uses the intentional attention of the BF16 and the accuracy of the published checkpoints. The general H3 deployment can also choose the following paths, but each is an independent quality and performance image rather than a lossless operating benefit.

Weight and activation quantification

• Online FP8. The combined global FP8 path starts from the BF16 checkpoint and quantifies the linear layers of the qualifying DiT and Qwen3-VL text decoder when loaded. Embedding, unification, RoPE, visual towers, two VAE and the precision-sensitive projection layers maintain their declaration accuracy.

• SVDQuant NVFP4 W4A4. The combined off-line loader combines the basic GEMM of NVFP4 W4A4 with a BF16 lower-order modification. Current evidence establishes that the checkpoint is compatible with correctness; the inherent fusion residual GEMM performance path is still a work in progress.

Figure 5: Online FP8 creates the FP8 weights and scales when loaded, and then quantifies the eligible activation online. Offline SVDQuant combines the basic branches of NVFP4 W4A4 with a BF16 lower-order modification. Source: vLLM-Omni #5910 and #6162, and two readings of online FP8 and SVDQuant in the cookbook.

A quantized image must report peak HBM, start-up host memory, checkpoint storage, latency, and video and audio quality under the same seed. Capacity gains are not automatically equal to latency gains, and loader accuracy does not constitute evidence of convergence kernel gains.

Online FP8 on the B300: Capacity and Delay

Below, the full-residency results compare the online FP8 with the released BF16 checkpoint separately. Both lines use 8 B300s, Ulysses8/Ring1 with Fast Ulysses, TP8 encoder, VAE PP8 tile decoding, CUDNNN attention, and 10 seconds 1344×768/24 FPS, requiring a load of 50 sigma points (49 DiT forward).

Each time the inputs were requested, RGB images of 243 tons 1344×768 were returned with a 32 kHz stereo sound frequency. Three repetitions of different seeds were used, so they established the output shape and generated success, rather than the pixel-by-pixel equivalent of BF16.

Quantitative and sparse attention in TRTLLM_ATTN

TRTLLM_ATTN offers two options for loss-accelerating modes:

• SAGE quantifies both QK and PV paths to FP8.

• Skip-Softmax uses the results of QK to dynamically skip the non-essential Softmax and P×V calculations.

Figure 6: SAGE quantifies Q, K, P, V to FP8 for Q×K and P×V, while Skip-Softmax is determined by BLASST's tile level, bypassing the selected Softmax and P×V tiles.

The following table summarizes the video quality and acceleration ratio in terms of density and non-quantitative attention:

TRTLLM Baseline

SAGE

Skip-Softmax

SAGE+Skip-Softmax

The Skip-Softmax configuration used in the above measurements is conservative in maintaining video quality. Users can choose a higher threshold, or enable Skip-Softmax at a more noisy pace, in order to exchange for more speed.

Cache-DiT

Cache-DiT is a request-level caching strategy, rather than a attention back-end. For H3, quality=high enables step-by-step dynamic replication and quality=lossless restores the reference path. Its fate and non-fate behavior depend on the specific deployment, and therefore requires independent delay and quality verification, not including the above attention A/B.

4.4 Compatibility boundaries

Step-by-step execution of supplementary instructions. H3 can accept and suspend requests between noise steps#5810In issue #5700, the request pattern is still recommended during cancellation and recycling mechanisms and small-scale low utilization loads.

5. From system optimization to FastH3

FastH3 is a four-step DMD2 student model developed by FastVideo based on the MiniMax H3 distillation. It duplicates H3's encoder, video VAE, audio VAE, tokenizer and timer, but reduces the noise cycle to four transformer forward directions at five sigma positions. vLLM-Omni also supports Dense/Data-Free products and recommended VSA/Data-Free products.

This integration is a two-level collaboration:

• FastVideo develops and releases distilled student models and adapter products.

• vLLM-Omni validates the product, completes the merger at the same time as the checkpoint stream is loaded, distributes the load after the merger, and serves through optimized attention, VAE, transmission and MP4 pathways.

FastH3 is not a normal LoRA that can be switched on request. In addition to the low-order factor, its product also carries a full-order delta with the substitution weight, which is not expressed by the normal LoRA layer. Therefore, vLLM-Omni chose to merge the product before splitting rather than activating it on request.

Figure 7: Turbo keeps the base weight unchanged and applies A/B sideways as requested. FastH3 integrates low-order and full-order modifications into a dedicated student model before splitting. Source: Turbo#6476DLO support#6550It's an integrated version of FastH3.#6714 → Turbo #6476DLO support#6550FastH3 is integrated.#6714It's the same thing.#6909 inVSA and Ulysses support

FastH3 v1 only accepts T2VA, requiring its own four-step tuning with the flow shift of the checkpoint, refusing offload, and no longer accepting LoRA when another request is made. Its VSA variant also additionally requires CUDA, external FastVideo kernel packs, and local or pure Ulysses sequence parallels. These are service contracts, not tuning recommendations.

6. FastH3 live service on the B300

This section reports the absolute results of FastH3 on vLLM-Omni 86b85c07. It has not since removed the basic H3 results from different source, prompt and seed.

6.1 Fixed product version

The Dense/Data-Free product of this measurement is fixed in Hugging Face revised edition bcf40ca6f457ed66f8badf13514943e390205fca:

FASTH3_REV=bcf40ca6f457ed66f8badf13514943e390205fcaFASTH3_DIR=/models/FastH3-LoRAhf download FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA \  dense-datafree/adapter_model.safetensors \  --revision "$FASTH3_REV" \  --local-dir "$FASTH3_DIR"echo "4ce198c83132251b7fd0de2503823aa49c53983f068318f66cb19eaefb7fcc12  $FASTH3_DIR/dense-datafree/adapter_model.safetensors" \  | sha256sum -c -

The size of the adapter is 1,485,626,152 bytes. Please fix the revised version of the warehouse and the file verification at the same time; the name of the warehouse still contains Preview-v1, and the matching vLLM-Omni integration has been merged.

6.2 Deployment and request

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \VLLM_WORKER_MULTIPROC_METHOD=spawn \VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \vllm serve "$H3_MODEL" --omni \  --host 127.0.0.1 --port 8095 --trust-remote-code \  --task-type fl2va --served-model-name MiniMaxAI/MiniMax-H3 \  --num-gpus 8 --usp 8 --ring 1 --ulysses-a2a-permute \  --text-encoder-tp-size 8 \  --vae-patch-parallel-size 8 --vae-parallel-mode tile --vae-use-tiling \  --diffusion-attention-back