Skip to content

perf(vae): batch spatial padding in Hunyuan causal convolutions - #2374

Open
losewayy wants to merge 1 commit into
leejet:masterfrom
losewayy:perf/hunyuan-vae-padding
Open

losewayy wants to merge 1 commit into
leejet:masterfrom
losewayy:perf/hunyuan-vae-padding

Conversation

@losewayy

@losewayy losewayy commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fold frames and channels into one dimension ([W, H, T * C]) via ggml_ext_cont and ggml_reshape_3d during spatial replicate_pad in Hunyuan's shared CausalConv3d, then restore the 4D shape before convolution. This avoids per-channel CUDA CONCAT kernel launches while keeping temporal padding and carry unchanged.

Related Issue / Discussion

N/A

Additional Information

Tested on RTX 5070 Ti Laptop (sm_120, CUDA 13.4, Vulkan 1.4, Windows Release) against 532d2c1. Standalone CausalConv3d checks (10 edge cases including non-contiguous views and temporal carry, plus 4 larger spatial shapes) are byte-identical on CPU, CUDA, and Vulkan separately. FP16 HunyuanVideo 1.5 VAE runs (512x288, 5 frames; 1 warmup + mean of 2 runs) produce byte-identical latents and decoded pixels on both CUDA and Vulkan.

backend / stage (512x288, 5 frames, FP16) baseline batched diff
CUDA encode 2.835 s 1.925 s -32.1%
CUDA decode 5.499 s 3.172 s -42.3%
CUDA combined 8.334 s 5.098 s -38.8%
Vulkan encode 27.002 s 26.907 s -0.4%
Vulkan decode 43.110 s 42.993 s -0.3%
Vulkan combined 70.112 s 69.900 s -0.3%

Full video generation and Metal were not tested.

Checklist

Summary by CodeRabbit

  • Bug Fixes
    • Improved handling of spatial padding during video processing.

@coderabbitai

coderabbitai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

CausalConv3d::forward now makes its input contiguous and combines frame and channel dimensions before spatial replicate padding. It restores the tensor shape after padding. Temporal padding and convolution behavior remain unchanged.

Changes: Hunyuan VAE spatial padding

Layer / File(s) Summary
Prepare the tensor for spatial padding
src/model/vae/hunyuan_vae.hpp
CausalConv3d::forward combines the frame and channel dimensions before spatial replicate padding, then restores the four-dimensional shape.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Refactor

Possibly related PRs

Suggested reviewers: leejet

Merge Risk: Minimal · up to 021d3

This change batches spatial padding in the Hunyuan VAE convolutions to speed up CUDA encode and decode. No concrete defect was found, and the author reports identical outputs on the platforms tested. Merge risk is minimal, though Metal was not tested.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the main change: batching spatial padding in the Hunyuan causal convolutions for performance. The raw summary confirms that CausalConv3d::forward now folds frames and channels into one…
Description check Passed The description follows the template. It has a Summary that explains the change and why it was made. It has a Related Issue / Discussion section marked N/A. It has an Additional Information section wi…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: batching spatial padding in the Hunyuan causal convolutions for performance. The raw summary confirms that CausalConv3d::forward now folds frames and channels into one dimension for spatial replicate padding, then restores the 4D shape. The title follows the conventional commit format and is specific enough for a scan of the history.

Full details: Description check

Explanation

The description follows the template. It has a Summary that explains the change and why it was made. It has a Related Issue / Discussion section marked N/A. It has an Additional Information section with detailed test hardware, backends, edge-case coverage, and a timing table. The Checklist is complete and both items are checked. The description does not include an HTML comment placeholder, and the template's comments were removed, which is expected.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0ae6cb56-f70c-4471-9711-bbca8670a706
Commits

Reviewing files that changed from the base of the PR and between 532d2c1 and 021d382.

Files selected for processing: 1

  • src/model/vae/hunyuan_vae.hpp

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 6 remain after this review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant