Optimize managed FastTree Sumup to native parity on arm64

[vladimir-aubrecht]

Problem

FastTree builds feature histograms via Sumup, the per-iteration hot loop of tree
training. On x64/x86 this uses the native FastTree library; on arm64 (and any
platform where the native library isn't available) it falls back to the generic
managed IntArray.Sumup, which goes through the IIntArrayForwardIndexer interface
with per-element bounds checks. That fallback is ~1.8x slower than native and
allocates on every call.

Note: the native Sumup (Sumup.h) is itself a plain scalar loop — the only real
SIMD in FastTreeNative is in segment.cpp (one-time segment compression), not in the
per-iteration histogram path. So there is no algorithmic reason managed can't match it.

Change

Add optimized managed Sumup implementations that mirror the native templates
(Sumup.h / SumupNibbles.h / SumupSegment.h) exactly, using fixed pointers and
no bounds checks:

• DenseIntArray: new SumupManagedDense covering 4/8/16/32-bit × weighted/unweighted
  × root (no doc indices)/leaf. Dense8/4/16/32BitIntArray now point their managed
  handler at it instead of the slow base.Sumup.
• SegmentIntArray: new SumupManaged mirroring SumupSegment /
  SumupSegment_noindices for the compressed segment format.

Native remains the default on x64/x86 (UseFastTreeNative is unchanged) — only the
managed fallback path is replaced, so arm64 picks up the fast path automatically.
Because the loops iterate in the same order as native, float accumulation is
bit-identical and existing baselines are unchanged.

Results (Apple M5, arm64)

Real in-repo types (FeatureHistogram.SumupWeighted), Dense8, weighted, 256 bins:

case	N	old fallback	new managed	speedup	identical
root	1,000,000	1.119 ms (894 Melem/s)	0.632 ms (1582 Melem/s)	1.77x	yes
leaf	500,000	0.553 ms (905 Melem/s)	0.363 ms (1378 Melem/s)	1.52x	yes

Native comparison (native Sumup.h scalar loop built for arm64 vs equivalent managed,
N=20M): native 1337 Melem/s, old fallback 746 Melem/s (1.79x slower), new managed
1386 Melem/s (0.96x = parity). Managed allocations per call: 20 B -> 0 B.

Testing

• FastTree/FastForest/FastRank baseline tests in Microsoft.ML.Predictor.Tests pass on
  arm64 (7 passed, 0 failed; the managed path is what runs there, compared against
  native-generated baselines -> confirms numerical identity).
• Standalone verification of the two non-trivial decoders: segment bit-unpacking
  (2000 random trials, sequential + indexed) and the 4-bit nibble formula
  (5000 trials) — 0 mismatches.

Notes for reviewers

• This does not change x64/x86 behavior; native stays the default there.
• UseFastTreeNative is intentionally left unchanged. If we later want to drop the
  native library entirely, this managed path is now fast enough to be the default on
  all architectures — but that is a separate decision.

Fixes #

[vladimir-aubrecht]

@dotnet-policy-service agree company="Microsoft"

[Copilot]

Pull request overview

This PR improves FastTree training performance on arm64 (and other platforms without the native FastTree library) by replacing the slow managed IntArray.Sumup fallback with optimized, bounds-check-free managed implementations that mirror the native scalar Sumup templates to preserve bit-identical histogram accumulation.

Changes:

• Added an optimized managed dense Sumup implementation (SumupManagedDense) and routed Dense 4/8/16/32-bit arrays to use it when native is unavailable.
• Added an optimized managed segment Sumup implementation (SegmentIntArray.SumupManaged) mirroring native segment decoding for both sequential (root) and indexed (leaf) cases.
• Updated handler selection so managed fallbacks use the new optimized implementations instead of base.Sumup.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
src/Microsoft.ML.FastTree/Dataset/SegmentIntArray.cs	Switches managed fallback to a new pointer-based segment Sumup implementation matching native decoding/accumulation order.
src/Microsoft.ML.FastTree/Dataset/DenseIntArray.cs	Introduces pointer-based dense Sumup implementation and wires Dense 4/8/16/32-bit arrays to use it on non-native platforms.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[matouskozak]

Standalone verification of the two non-trivial decoders: segment bit-unpacking
(2000 random trials, sequential + indexed) and the 4-bit nibble formula
(5000 trials) — 0 mismatches.

Would it make sense to add these as a tests? I'm wondering if we have much coverage over the new implementation...

I think ideally we should run both native and the new manage implementation on CI side by side to make sure they are ± equal.

[vladimir-aubrecht]

Good idea — added FastTreeSumupParityTests (946ffe6) that turns the ad-hoc verification into real tests:

• ManagedSumupMatchesReference — managed histogram vs an independent brute-force reference. Covers Dense 4/8/16/32-bit and Segment, root (no doc indices) and leaf (with doc indices), with and without weights. Runs on all platforms including arm64.
• ManagedSumupMatchesNative — the native and managed handlers run side by side on the same random data and the histograms are asserted bit-identical. It's gated with [NativeDependencyTheory("FastTreeNative")], so it executes wherever the native FastTree library is present (i.e. the x64 CI legs) and is skipped where it isn't (arm64).

To let the tests call the managed path directly, the Dense SumupManaged handlers are now internal (Segment's is already public). I deliberately left SumupNative as-is to avoid an unrelated visibility change in this PR.

Note: the current red CI is an unrelated flaky SdcaLogisticRegression (LogLoss 0.5061 vs the 0.5 bound) on the arm64 Release leg — SDCA doesn't touch the FastTree Sumup path, the arm64 Debug leg passed, and it reproduces as borderline/non-deterministic locally.