Skip to content

Insert gradient reduction during pass expansion when multiple consumers use the same input - #1666

Open
elliottslaughter wants to merge 8 commits into
flexflow:masterfrom
elliottslaughter:fix-bwd-multi-use
Open

Insert gradient reduction during pass expansion when multiple consumers use the same input#1666
elliottslaughter wants to merge 8 commits into
flexflow:masterfrom
elliottslaughter:fix-bwd-multi-use

Conversation

@elliottslaughter

@elliottslaughter elliottslaughter commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

In the backward pass, a value used multiple times by different nodes results in gradients that must be summed together to produce the final result. This PR adds a new GradientReduction node type to implement this, along with a subgradient_id_t on DynamicValueAttrs to track the distinct sub-gradients that contribute to the final result, and then updates pass expansion to detect the duplicate values in the backward pass and insert GradientReductions for them.


This change is Reviewable

@elliottslaughter elliottslaughter changed the title Insert gradient reduction during pass expansion when multiple consumer use the same input Insert gradient reduction during pass expansion when multiple consumers use the same input Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant