Qualcomm AI Engine Direct - Add QNN HadamardTransform op support by chenweng-quic · Pull Request #21729 · pytorch/executorch

chenweng-quic · 2026-08-11T03:55:35Z

Summary

Add support for the QNN HadamardTransform op for the HTP backend.

When a constant weight equals a scaled Hadamard matrix (scipy.linalg.hadamard(dim) * s), the op is lowered to a single QNN HadamardTransform (optionally scaled) instead of the generic op.

A first-class custom op torch.ops.qnn_custom.hadamard_transform is registered, and a pre-quantization RecomposeHadamard pass detects the matching nodes and rewrites them into the custom op, so it is annotated and lowered as a HadamardTransform from the start rather than being detected late in the builder and validated as FullyConnected / MatMul / Conv. A shared matcher (match_hadamard_weight in builders/utils.py) inspects the real constant weight during the annotation pass. Three patterns are covered:

linear — bias-less aten.linear.
matmul — x @ W, matching the constant right-hand operand (last-dim transform); this is the R3-rotation pattern used by static LLaMA.
conv — bias-less pointwise 1x1 conv with groups=1, which mixes only the channel dim.

Since hadamard_transform operates on the last dim, linear and matmul rewrite directly, while conv is wrapped in permutes that move the channel dim to the last dim and back. A dedicated htp_rules.py annotator and op_hadamard.py builder handle quantization and lowering. The rewrite is gated on QNN SDK >= 2.47 for backward compatibility.

Test plan

python backends/qualcomm/tests/test_qnn_delegate.py -k "test_qnn_backend_hadamard_transform" --device ef5e4029 --host localhost --soc_model SM8850 --build_folder build-android --executorch_root . --artifact test_qnn_delegate_artifact

pytest backends/qualcomm/tests/rework/htp/op/v68/test.py --build_folder build-android --soc_model SM8850 --device ef5e4029 -k test_hadamard

pytest backends/qualcomm/tests/rework/passes/test.py -k "test_recompose_hadamard"

The tests assert that the graph is rewritten to qnn_custom.hadamard_transform before quantization, and inspect the optrace QHAS op types to assert HadamardTransform appears after lowering.

Performance

CTX_LEN = 1024
AR_LEN = 128

Model	w/o		w/
	Prefill Graph execute time	Decode Graph execute time	Prefill Graph execute time	Decode Graph execute time
qwen2_5-0_5b	58792	5295	16361	5321
qwen2_5-1_5b	69230	15451	69329	15495
qwen3-1_7b	68005	19643	39825	19466
smollm2_135m	10001	2894	6688	2804

pytorch-bot · 2026-08-11T03:55:39Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21729

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2026-08-11T03:56:27Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

psiddh · 2026-08-11T23:09:26Z

+
+import numpy as np
+
+import scipy.linalg


make it lazy , move the import inside _match_hadamard_weight) so merely importing the pass package doesn't require it ?

Could we drop scipy here and reuse the _hadamard_matrix from
builders/custom_ops.py? It's not in setup.py/requirements and _passes/init.py imports this eagerly, so every QNN export would need scipy or hit ModuleNotFoundError. am I reading that right?

psiddh · 2026-08-11T23:11:32Z

I have one comment / clarification, but o/w lgtm

The URL, xref, and file size linters diffed base..head, where base is the tip of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions and the branch is blamed for links someone else already repaired. pytorch#21729 failed exactly that way, on eight links pytorch#21694 had already fixed or ignored. The workflow now resolves the merge base and passes it down. That is the half that matters for branches already open: on a pull request the reusable workflow resolves from the merge commit, so it carries this fix even though scripts/ still comes from the branch itself. The scripts switch to three dot ranges so running them by hand behaves the same way, and the checkouts fetch real history, without which no merge base exists to compute. Authored with Claude Code (Claude Opus 5).

chenweng-quic · 2026-08-12T02:07:13Z

I have one comment / clarification, but o/w lgtm

Thanks for review, will update.

psiddh · 2026-08-12T05:18:44Z

@chenweng-quic Linters fixes are needed now

The URL, xref, and file size linters diffed base..head, where base is the tip of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions and the branch is blamed for links someone else already repaired. pytorch#21729 failed exactly that way, on nine links pytorch#21694 had already fixed or ignored. The workflow resolves the merge base and passes it down. That is the half that matters for branches already open: on a pull request the reusable workflow resolves from the merge commit, so it carries this fix even though scripts/ still comes from the branch itself. The scripts switch to three dot ranges so running them by hand behaves the same way. Computing a merge base needs history, so the pull request path checks out in full, restoring what pytorch#17682 traded away for speed at a measured 25s of wall clock. Pushes and the nightly whole tree scan cannot use a merge base and stay shallow. A missing merge base leaves the range unset for a whole tree scan rather than substituting the base tip, which two of the three scripts accept and then quietly pass. Authored with Claude Code (Claude Opus 5).

…21765) ### Summary The URL, xref, and file-size linters diffed `base..head`, where `base` is the **tip** of the base branch rather than the point the branch left it. A branch cut before recent commits still carries the lines those commits replaced, so against the newer tip its old copies read as additions, and the branch gets blamed for links someone else already repaired. #21729 failed exactly this way. It touches 18 files and adds no URLs at all, but the two-dot diff scoped the lint to 143 files and flagged nine links that #21694 had already fixed or `@lint-ignore`d. #21707 is starker: one Python file with no URLs in it, 199 files linted, the same nine failures. ``` base..head 143 files <- what CI linted base...head 18 files <- what the PR actually changes ``` The stray `jq: parse error` lines in #21729's log are the same symptom from the other direction: the job runs the branch's own pre-#21694 copy of `lint_urls.sh`. ### Fix The workflow resolves the merge base and passes it down. That is the half that matters for branches already open: on a `pull_request` the reusable workflow resolves from the merge commit, so it carries this fix even though `scripts/` still comes from the branch itself. The scripts also switch to three-dot ranges so `./scripts/lint_urls.sh main HEAD` by hand behaves the same. `lint_xrefs.sh` and `lint_file_size.sh` had the identical bug and get the identical change. **Cost, stated plainly.** A merge base needs real history, so this restores `fetch-depth: 0` on the `pull_request` path — line for line what #17682 removed in February for speed. Measured on this branch, that is ~25s of added wall clock and ~79s of runner time across the three concurrent jobs. #17682's 6min → 10s was mostly the runner and Docker change rather than the fetch depth, though the commit changes both at once and I can't fully separate them. Pushes and the nightly whole-tree scan cannot use a merge base and stay shallow: ```yaml fetch-depth: ${{ github.event_name == 'pull_request' && '0' || '1' }} ``` The quotes matter — bare `0` is falsy in GitHub expressions, so `&& 0 || 1` always yields `1` and would silently disable the fix. **No silent fallback.** When there is no merge base there is no usable range, so the range is left unset and the linters scan the whole tree. Substituting the base tip instead produces a range the scripts fail on quietly: verified against unrelated histories, `lint_urls.sh` and `lint_xrefs.sh` both exit **0** having checked nothing, while `lint_file_size.sh` exits 128 and the wrapper reports "some files exceed the 1 MB limit", which is not what happened. Unset args are loud instead — verified rc=1 on a tree containing a dead link. **`--no-color`.** Both `git diff` calls now pass it, matching the `git grep --no-color` two lines below. With `color.ui = always` in a developer's config, added lines arrive wrapped in escape sequences, `grep -E '^\+'` matches nothing, and the check passes having found nothing (measured: 2 matches → 0). ### Test plan `.ci/scripts/tests/test_link_check_diff_selection.py` builds a diverged history where `main` repairs bad links and shrinks an oversized file while the feature branch simply predates all of it. Four fixture files each pin a different part, and `curl` is stubbed so there is no network: | Fixture | Pins | |---|---| | `both_sides.md` — edited on both branches | the per-file diff range | | `big.bin` — oversized at the branch point, shrunk on main | `lint_file_size.sh`'s range | | `colorful.gitconfig` — `color.ui = always` | `--no-color` | | `base_only.md` / `feature_only.md` | that main-only changes stay invisible and the branch's own additions are still checked | ``` pytest .ci/scripts/tests/test_link_check_diff_selection.py # 4 passed ``` Mutating each changed line individually: ``` inner per-file range -> .. 3 failed caught lint_file_size range -> .. 1 failed caught drop --no-color 1 failed caught file-selection range -> .. 4 passed equivalent mutant, see below ``` The file-selection range is not pinned because it cannot be: with the per-file diff at three dots, the extra files it selects produce empty diffs. Replayed against #21707's real 199-file range, both variants emit byte-identical output. It is changed for consistency, not behaviour. **Narrowing the scope must not blunt the check**, so each linter also has a positive control where the branch itself adds the bad thing: ``` lint_urls adds a dead URL -> rc=1, reports example.invalid/dead lint_xrefs adds a broken reference -> rc=1, reports sub/missing.md lint_file_size adds a 1MB+ file -> rc=1, reports feature_big.bin ``` End to end on real content: a synthetic commit on top of `main` that puts #21694's nine repaired links back, as if a PR had added them, gives `rc=1` with 9 FAIL and 5 OK against the live network. Incidentally `musl.cc` answered this time where CI saw `000`, and `pybind/cmake_example` now 404s where CI saw `301` — which is the retry-then-WARN path earning its keep. Also replayed #21729's and #21707's exact CI refs through the fixed scripts (exit 0 each), and ran all three linters plus `lintrunner` against this PR's own diff. ### Known limitations - The checkout is `head.sha`, so references still resolve against the branch tree rather than the merge result — the mirror image of the bug fixed here. Pre-existing and not worsened by this PR. - The whole-tree branch swallows a `git grep` failure and exits 0, so on a git built without PCRE the scan silently checks nothing. Reachable locally, not on the runners, which is why the nightly scan works. Pre-existing; worth a follow-up rather than widening this PR. ### Rollout This does **not** repair a currently red run. A rerun keeps the original `GITHUB_SHA`, and advancing the base alone does not fire `synchronize`, so an already-open PR picks the fix up only on its next newly triggered pull-request run — any push, or close/reopen. That is still cheaper than a content rebase: no history rewrite, no conflicts, nothing to re-review. Supersedes #21762, which fixes the same bug but leaves the checkout shallow and misses `lint_urls.sh`. Its reviewer's shallow-checkout point is exactly right: a `--depth=1` fetch writes a shallow graft even into an otherwise complete clone, after which `git merge-base` fails and `A...B` is fatal, so the two halves of this change are a pair. The regression test here is adapted from that PR. Authored with Claude Code (Claude Opus 5).

psiddh · 2026-08-12T23:22:03Z

@chenweng-quic can you resolve the conflict ?

Lower a bias-less linear / matmul / pointwise 1x1 conv whose constant weight is a scaled Hadamard matrix into QNN's native HadamardTransform op. A torch.ops.qnn_custom.hadamard_transform op is registered, and a pre-quantization RecomposeHadamard pass detects the matching nodes (via the shared match_hadamard_weight matcher in builders/utils.py) and rewrites them into the custom op. Since hadamard_transform operates on the last dim, linear and matmul rewrite directly, while conv is wrapped in permutes that move the channel dim to the last dim and back. A dedicated htp_rules.py annotator and op_hadamard.py builder handle quantization and lowering. The rewrite is gated on QNN SDK >= 2.47 for backward compatibility. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

_passes/__init__.py imports RecomposeHadamard eagerly, so the module-level `import scipy.linalg` would raise ModuleNotFoundError for every QNN export on hosts without scipy (it isn't a project dependency). Reuse the existing torch-based `_hadamard_matrix` Sylvester construction from builders/custom_ops.py instead, and do the same in the test files that had duplicated the scipy-based construction to build reference Hadamard weights. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Summary: PR pytorch#21729 added HadamardTransform op support, gated on a qnn sdk version. The version check assumed QNN_SDK_ROOT was available, resulting in errors when it is not. This PR matches other usage like op_embedding.py to fallback when SDK is absent. Reviewed By: rascani Differential Revision: D115881389

chenweng-quic requested review from abhinaykukkadapu and psiddh as code owners August 11, 2026 03:55

meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 11, 2026

psiddh reviewed Aug 11, 2026

View reviewed changes

psiddh approved these changes Aug 11, 2026

View reviewed changes

psiddh mentioned this pull request Aug 12, 2026

Lint only what a PR adds: diff the link checks from the merge base #21765

Merged

chenweng-quic force-pushed the dev1/chenweng/hadamard_transform_support branch from 8fe3323 to 87b5243 Compare August 12, 2026 04:01

chenweng-quic and others added 3 commits August 13, 2026 10:20

fix lint

a356d89

chenweng-quic force-pushed the dev1/chenweng/hadamard_transform_support branch from 032ec43 to a356d89 Compare August 13, 2026 02:21

psiddh merged commit 43f89fb into pytorch:main Aug 13, 2026
182 of 183 checks passed

JakeStevens mentioned this pull request Aug 13, 2026

Add check for QNN_SDK_ROOT in hadamard pass (#21815) #21815

Merged

Navigation Menu

Uh oh!

Qualcomm AI Engine Direct - Add QNN HadamardTransform op support - #21729

Qualcomm AI Engine Direct - Add QNN HadamardTransform op support#21729
psiddh merged 3 commits into
pytorch:mainfrom
CodeLinaro:dev1/chenweng/hadamard_transform_support

chenweng-quic commented Aug 11, 2026 •

edited

Loading

Uh oh!

pytorch-bot Bot commented Aug 11, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 11, 2026

Uh oh!

psiddh Aug 11, 2026

Uh oh!

psiddh commented Aug 11, 2026

Uh oh!

chenweng-quic commented Aug 12, 2026

Uh oh!

psiddh commented Aug 12, 2026 •

edited

Loading

Uh oh!

psiddh commented Aug 12, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

chenweng-quic commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Performance

Uh oh!

pytorch-bot Bot commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21729

Uh oh!

github-actions Bot commented Aug 11, 2026

This PR needs a release notes: label

Uh oh!

psiddh Aug 11, 2026

Choose a reason for hiding this comment

Uh oh!

psiddh commented Aug 11, 2026

Uh oh!

chenweng-quic commented Aug 12, 2026

Uh oh!

psiddh commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

psiddh commented Aug 12, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

chenweng-quic commented Aug 11, 2026 •

edited

Loading

pytorch-bot Bot commented Aug 11, 2026 •

edited

Loading

This PR needs a `release notes:` label

psiddh commented Aug 12, 2026 •

edited

Loading