[PyTorch] Add no-load-balance THD all-gather CP - #3221

Open

sudhakarsingh27 wants to merge 13 commits into

NVIDIA:mainfrom

sudhakarsingh27:sudhakars/relaxed-load-balancing-all-gather

Commits on Aug 18, 2026

Add packed THD partitioning for all-gather CP
sudhakarsingh27
committed
bf31706
Refine THD super-sequence partitioning
sudhakarsingh27
committed
34f009a
Add contiguous THD all-gather partition
sudhakarsingh27
committed
63d0215
Remove mirrored packed THD partition
sudhakarsingh27
committed
b360584
Gate packed THD partition with env flag
sudhakarsingh27
committed
1fef4e5
Remove experimental packed THD tests from PR
sudhakarsingh27
committed
57f3cf8

Commits on Aug 19, 2026

Rename experimental THD policy and add coverage
sudhakarsingh27
committed
e8a1f70

Commits on Aug 20, 2026

Move THD no-load-balance checks to CP dispatch
sudhakarsingh27
committed
fb66611

Commits on Aug 21, 2026

Merge origin/main into no-load-balance THD branch
sudhakarsingh27
committed
eb543ee
Expose CP attention load-balancing strategy
sudhakarsingh27
committed
dfa0fd9
Rename CP load-balancing strategy
sudhakarsingh27
committed
90e906a
Mark no-load-balance CP strategy experimental
sudhakarsingh27
committed
88998d4
Skip known sm90 deterministic THD OOM
sudhakarsingh27
committed
ac516e8