[Pytorch] MOE Sequential Block - #3350

Open

vthumbe1503 wants to merge 73 commits into

NVIDIA:mainfrom

vthumbe1503:dispatch_combine_op

Commits on Jul 28, 2026

produce/consume extra output
vthumbe1503
committed
b1574f0

Commits on Aug 4, 2026

allow for fusions with producer/consumer being part of same fuser with error handling tests
vthumbe1503
committed
63192ab
cleanup
vthumbe1503
committed
3b4b523
minor cleanup
vthumbe1503
committed
de38ed8
dispatch combine impl
vthumbe1503
committed
385b0d5

Commits on Aug 5, 2026

fusible ops test
vthumbe1503
committed
ad3b044
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed
5fb0d3a
Merge remote-tracking branch 'nvidia_origin/main' into enable_extra_out_consumption
vthumbe1503
committed
2ba4f6a
keep just ops infra changes
vthumbe1503
committed
3af2ecc
cleanup with residual tests
vthumbe1503
committed
d7d6380
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed
74f563a

Commits on Aug 6, 2026

Merge branch 'main' into enable_extra_out_consumption
vthumbe1503
authored
29d23f2
address review comment
vthumbe1503
committed
87e2b36

Commits on Aug 7, 2026

update to cleaner documentation
vthumbe1503
committed
80601dc
address review comments
vthumbe1503
committed
5070e34
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed
ae41ad3

Commits on Aug 9, 2026

some cleanup
vthumbe1503
committed
f82cbed
update docs
vthumbe1503
committed
5a4e1ec
pin channels through channel version
vthumbe1503
committed
0a479c7
unecessary handling removal
vthumbe1503
committed
d679998
simplify
vthumbe1503
committed
8f7ba95
doc update + extra_grad = None case
vthumbe1503
committed
c62bb15
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed
a93b820
test cleanup
vthumbe1503
committed
35b73b1
no need to check staleness in every forward call
vthumbe1503
committed
6801a6d
remove redundant tests
vthumbe1503
committed
6688e8a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed
12430c2
revert from bad names
vthumbe1503
committed
b189550
keep simple
vthumbe1503
committed
a4cc112
Merge branch 'enable_extra_out_consumption' of github.com:vthumbe1503/TransformerEngine into enable_extra_out_consumption
vthumbe1503
committed
7edaf89
unecessary checks
vthumbe1503
committed
5ba6055
minor doc
vthumbe1503
committed
76826dc

Commits on Aug 10, 2026

Commits on Aug 11, 2026

Update transformer_engine/pytorch/ops/fuser.py

vthumbe1503
and
timmoon10
authored
87c1cf6
Update docs/examples/op_fuser/op_fuser.rst

vthumbe1503
and
timmoon10
authored
307ab15
Update transformer_engine/pytorch/ops/fuser.py

vthumbe1503
and
timmoon10
authored
97a91cf
address review comments + extra output being configurable to be outputted
vthumbe1503
committed
6468a14
cleanup
vthumbe1503
committed
2eb21ed
Merge remote-tracking branch 'origin/enable_extra_out_consumption' into dispatch_combine_op
vthumbe1503
committed
1b0a4b1
Restore ep_reference to its originally created content.
vthumbe1503
committed
212a460
Support BF16 grouped MLP in ep_reference and keep MOE extras internal.
vthumbe1503
committed
c7dacdf
update test to have internal extra_out
vthumbe1503
committed
00b48cf
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed
db24495

Commits on Aug 12, 2026

extra output grad can be None, equivalent to zero
vthumbe1503
committed
3c37643

Commits on Aug 17, 2026

Commits on Aug 19, 2026

cudnn kernel
vthumbe1503
committed
e40eb74
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed
c7eceaa
remove the previous artifact
vthumbe1503
committed
fb8fc83
minor change
vthumbe1503
committed
76d7fdd
minor bug fix
vthumbe1503
committed
c946162

Commits on Aug 20, 2026

single_groupe_weight + remove copies
vthumbe1503
committed
80f6063
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed
a779c6c

Commits on Aug 21, 2026

cpu init for megatron needs
vthumbe1503
committed
f197433
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed
075fc44

Commits on Aug 22, 2026

Commits on Aug 23, 2026

cleanups
vthumbe1503
committed
31407af
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed
1d5d2ed
more cleanup
vthumbe1503
committed
d13c17f
Merge branch 'dispatch_combine_op' of github.com:vthumbe1503/TransformerEngine into dispatch_combine_op
vthumbe1503
committed
bfb7eec
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed
8e18c46
refactor tests
vthumbe1503
committed
895d1a1
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed
a4279c2

Commits on Aug 24, 2026

some fix
vthumbe1503
committed
ffb4d62
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed
d449da9
update fused kernel import issue and view issue in delay wgrad
vthumbe1503
committed
f888050