Skip to content

perf: reduce CUDA launch overhead in H3 VAE encoding - #2369

Open
losewayy wants to merge 2 commits into
leejet:masterfrom
losewayy:perf/h3-vae-spatial-padding
Open

losewayy wants to merge 2 commits into
leejet:masterfrom
losewayy:perf/h3-vae-spatial-padding

Conversation

@losewayy

@losewayy losewayy commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

H3 VAE encoding spends much of its time launching CUDA kernels for spatial reflection padding. CONCAT launches once per channel with the current tensor layout.

Fold frames and channels into one dimension while padding width and height in both CausalConv3d and Downsample3D, then restore the original shape before the existing temporal padding and convolution. This lets CONCAT process all channels in one launch. The reflected edge values are unchanged.

Related Issue / Discussion

N/A

Additional Information

Tested MiniMax-H3-Image reference-image editing on Windows with an RTX 5070 Ti Laptop GPU (12 GB) and CUDA 13.4, against 532d2c15c430a785a712b2d94281f04e733008a4.

Settings: INT8 ConvRot diffusion and P-H3 text encoder, FP16 H3 Image VAE, 1024 × 1152 reference, 1024 × 1024 output, 6 Euler steps, CFG 1, seed 42, CPU offload, 8 CPU threads, VAE tiles 256 with 25% overlap.

Mean of two runs per version, each in a fresh process without profiling:

Before After
Reference-image encoding 19.15 s 3.04 s
Total time, including startup and saving 80.17 s 62.44 s

Encoding is 6.3× faster; total time drops by 22%. Both optimized runs produce the same pixels as the baseline. A separate Nsight Systems capture shows encoder CONCAT launches dropping from 1,375,080 to 72,240.

Also checked CausalConv3d with one- and five-frame inputs on CPU and CUDA. On each backend, the original and modified code produce exactly the same values. The CUDA Release build, changed-line formatting check, and git diff --check pass.

The change is in the shared H3 encoder, also used for video and by Sol-H3-VAE. Full video generation, Metal, and Vulkan were not tested.

Downsampling padding

The initial change above optimizes CausalConv3d. Downsample3D also pads the right and bottom edges, so this PR now applies the same layout change there.

Measured separately against 3fdcce8, which already includes the CausalConv3d optimization, using the same model, reference image, output size, steps, CFG, and seed listed above, with CPU RNG and a 7.5 GiB VRAM budget. Two runs per version were interleaved in separate processes:

Before After
Reference-image encoding 2.94 s 2.12 s

This further reduces encoding time by 27.9%, saving 0.82 s. All four generated images are pixel-identical. These are separate measurements from the initial comparison above. End-to-end time varied with offload and memory pressure, so no additional end-to-end speedup is claimed.

Downsample3D checks with single- and multi-frame inputs, including non-contiguous inputs, produce exactly the same values before and after the change on each backend separately (CPU and CUDA). The CUDA Release build, changed-line formatting checks, and git diff --check pass.

Checklist

  • I have read and confirmed this PR follows the contribution guidelines.
  • If this PR adds or changes model support, I have added or updated its usage guide and relevant model index/README entries following the model documentation requirements. (N/A: this PR optimizes the existing encoder without changing model support.)

Summary by CodeRabbit

  • Bug Fixes
    • Improved spatial padding and edge handling during video processing, including downsampling. This helps ensure frames are handled consistently at image boundaries and supports reliable spatial operations across frames. Temporal padding and convolution behavior are unchanged.

@coderabbitai

coderabbitai Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

CausalConv3d::forward and Downsample3D::forward now make their inputs contiguous and flatten frames and channels around spatial padding operations. Both methods restore the 4D tensor shape afterward.

Changes: Spatial padding

Layer / File(s) Summary
Spatial padding reshape
src/model/vae/minimax_h3_vae.hpp
CausalConv3d::forward flattens frames and channels while applying reflect padding. When spatial_stride == 2, Downsample3D::forward uses the same reshape pattern for right and bottom edge extensions. Both methods restore the 4D shape.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Refactor

Suggested reviewers: leejet

Merge Risk: Minimal · up to f1d8c

The change batches spatial padding in the H3 VAE encoder to cut CUDA kernel launches, and no concrete correctness or stability defect was found. The remaining unknowns are the CPU, Metal and Vulkan backends, which the author did not test. Merge risk is minimal.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title "perf: reduce CUDA launch overhead in H3 VAE encoding" matches the main change. The raw summary and objectives describe batching spatial reflect-padding in CausalConv3d and Downsample3D so t…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description follows the repository template closely. It has a Summary that explains the change and its purpose (folding frames and channels so CONCAT runs in one launch, with unchanged edge values…
Full details: Title check

Explanation

The title "perf: reduce CUDA launch overhead in H3 VAE encoding" matches the main change. The raw summary and objectives describe batching spatial reflect-padding in CausalConv3d and Downsample3D so that CUDA launches drop during H3 VAE encoding. The title is specific, concise, and names the performance goal and the affected area.

Full details: Description check

Explanation

The description follows the repository template closely. It has a Summary that explains the change and its purpose (folding frames and channels so CONCAT runs in one launch, with unchanged edge values). It states the Related Issue as N/A. Additional Information gives detailed verification: before/after timings, pixel-identical output, CPU and CUDA value checks, build and formatting checks, and untested backends. The Checklist is present and checked, with a N/A note for the model-support item. The description is specific and complete. Minor note: the Related Issue section says N/A, which is acceptable, and the PR objectives reference PR #2369 which the description does not mention, but this is not a required template field.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: be1b7975-9b35-4ce9-9c3b-8a38b6937e03
Commits

Reviewing files that changed from the base of the PR and between 3fdcce8 and f1d8c9d.

Files selected for processing: 1

  • src/model/vae/minimax_h3_vae.hpp

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant