You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
perf: reduce CUDA launch overhead in H3 VAE encoding - #2369
H3 VAE encoding spends much of its time launching CUDA kernels for spatial reflection padding. CONCAT launches once per channel with the current tensor layout.
Fold frames and channels into one dimension while padding width and height in both CausalConv3d and Downsample3D, then restore the original shape before the existing temporal padding and convolution. This lets CONCAT process all channels in one launch. The reflected edge values are unchanged.
Related Issue / Discussion
N/A
Additional Information
Tested MiniMax-H3-Image reference-image editing on Windows with an RTX 5070 Ti Laptop GPU (12 GB) and CUDA 13.4, against 532d2c15c430a785a712b2d94281f04e733008a4.
Settings: INT8 ConvRot diffusion and P-H3 text encoder, FP16 H3 Image VAE, 1024 × 1152 reference, 1024 × 1024 output, 6 Euler steps, CFG 1, seed 42, CPU offload, 8 CPU threads, VAE tiles 256 with 25% overlap.
Mean of two runs per version, each in a fresh process without profiling:
Before
After
Reference-image encoding
19.15 s
3.04 s
Total time, including startup and saving
80.17 s
62.44 s
Encoding is 6.3× faster; total time drops by 22%. Both optimized runs produce the same pixels as the baseline. A separate Nsight Systems capture shows encoder CONCAT launches dropping from 1,375,080 to 72,240.
Also checked CausalConv3d with one- and five-frame inputs on CPU and CUDA. On each backend, the original and modified code produce exactly the same values. The CUDA Release build, changed-line formatting check, and git diff --check pass.
The change is in the shared H3 encoder, also used for video and by Sol-H3-VAE. Full video generation, Metal, and Vulkan were not tested.
Downsampling padding
The initial change above optimizes CausalConv3d. Downsample3D also pads the right and bottom edges, so this PR now applies the same layout change there.
Measured separately against 3fdcce8, which already includes the CausalConv3d optimization, using the same model, reference image, output size, steps, CFG, and seed listed above, with CPU RNG and a 7.5 GiB VRAM budget. Two runs per version were interleaved in separate processes:
Before
After
Reference-image encoding
2.94 s
2.12 s
This further reduces encoding time by 27.9%, saving 0.82 s. All four generated images are pixel-identical. These are separate measurements from the initial comparison above. End-to-end time varied with offload and memory pressure, so no additional end-to-end speedup is claimed.
Downsample3D checks with single- and multi-frame inputs, including non-contiguous inputs, produce exactly the same values before and after the change on each backend separately (CPU and CUDA). The CUDA Release build, changed-line formatting checks, and git diff --check pass.
If this PR adds or changes model support, I have added or updated its usage guide and relevant model index/README entries following the model documentation requirements. (N/A: this PR optimizes the existing encoder without changing model support.)
Summary by CodeRabbit
Bug Fixes
Improved spatial padding and edge handling during video processing, including downsampling. This helps ensure frames are handled consistently at image boundaries and supports reliable spatial operations across frames. Temporal padding and convolution behavior are unchanged.
No actionable comments were generated in the recent review.
Walkthrough
CausalConv3d::forward and Downsample3D::forward now make their inputs contiguous and flatten frames and channels around spatial padding operations. Both methods restore the 4D tensor shape afterward.
CausalConv3d::forward flattens frames and channels while applying reflect padding. When spatial_stride == 2, Downsample3D::forward uses the same reshape pattern for right and bottom edge extensions. Both methods restore the 4D shape.
The change batches spatial padding in the H3 VAE encoder to cut CUDA kernel launches, and no concrete correctness or stability defect was found. The remaining unknowns are the CPU, Metal and Vulkan backends, which the author did not test. Merge risk is minimal.
Pre-merge checks: 4
Check name
Status
Explanation
Title check
The title "perf: reduce CUDA launch overhead in H3 VAE encoding" matches the main change. The raw summary and objectives describe batching spatial reflect-padding in CausalConv3d and Downsample3D so t…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Description check
The description follows the repository template closely. It has a Summary that explains the change and its purpose (folding frames and channels so CONCAT runs in one launch, with unchanged edge values…
Full details: Title check
Explanation
The title "perf: reduce CUDA launch overhead in H3 VAE encoding" matches the main change. The raw summary and objectives describe batching spatial reflect-padding in CausalConv3d and Downsample3D so that CUDA launches drop during H3 VAE encoding. The title is specific, concise, and names the performance goal and the affected area.
Full details: Description check
Explanation
The description follows the repository template closely. It has a Summary that explains the change and its purpose (folding frames and channels so CONCAT runs in one launch, with unchanged edge values). It states the Related Issue as N/A. Additional Information gives detailed verification: before/after timings, pixel-identical output, CPU and CUDA value checks, build and formatting checks, and untested backends. The Checklist is present and checked, with a N/A note for the model-support item. The description is specific and complete. Minor note: the Related Issue section says N/A, which is acceptable, and the PR objectives reference PR #2369 which the description does not mention, but this is not a required template field.
Finishing Touches
Generate unit tests (beta)
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Repository UI
Review profile: CHILL
Plan: Advanced
Run ID: be1b7975-9b35-4ce9-9c3b-8a38b6937e03
Commits
Reviewing files that changed from the base of the PR and between 3fdcce8 and f1d8c9d.
Files selected for processing: 1
src/model/vae/minimax_h3_vae.hpp
Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
H3 VAE encoding spends much of its time launching CUDA kernels for spatial reflection padding. CONCAT launches once per channel with the current tensor layout.
Fold frames and channels into one dimension while padding width and height in both CausalConv3d and Downsample3D, then restore the original shape before the existing temporal padding and convolution. This lets CONCAT process all channels in one launch. The reflected edge values are unchanged.
Related Issue / Discussion
N/A
Additional Information
Tested MiniMax-H3-Image reference-image editing on Windows with an RTX 5070 Ti Laptop GPU (12 GB) and CUDA 13.4, against
532d2c15c430a785a712b2d94281f04e733008a4.Settings: INT8 ConvRot diffusion and P-H3 text encoder, FP16 H3 Image VAE, 1024 × 1152 reference, 1024 × 1024 output, 6 Euler steps, CFG 1, seed 42, CPU offload, 8 CPU threads, VAE tiles 256 with 25% overlap.
Mean of two runs per version, each in a fresh process without profiling:
Encoding is 6.3× faster; total time drops by 22%. Both optimized runs produce the same pixels as the baseline. A separate Nsight Systems capture shows encoder CONCAT launches dropping from 1,375,080 to 72,240.
Also checked CausalConv3d with one- and five-frame inputs on CPU and CUDA. On each backend, the original and modified code produce exactly the same values. The CUDA Release build, changed-line formatting check, and
git diff --checkpass.The change is in the shared H3 encoder, also used for video and by Sol-H3-VAE. Full video generation, Metal, and Vulkan were not tested.
Downsampling padding
The initial change above optimizes CausalConv3d. Downsample3D also pads the right and bottom edges, so this PR now applies the same layout change there.
Measured separately against
3fdcce8, which already includes the CausalConv3d optimization, using the same model, reference image, output size, steps, CFG, and seed listed above, with CPU RNG and a 7.5 GiB VRAM budget. Two runs per version were interleaved in separate processes:This further reduces encoding time by 27.9%, saving 0.82 s. All four generated images are pixel-identical. These are separate measurements from the initial comparison above. End-to-end time varied with offload and memory pressure, so no additional end-to-end speedup is claimed.
Downsample3D checks with single- and multi-frame inputs, including non-contiguous inputs, produce exactly the same values before and after the change on each backend separately (CPU and CUDA). The CUDA Release build, changed-line formatting checks, and
git diff --checkpass.Checklist
Summary by CodeRabbit