Skip to content

Preserve object dtype on full reduction (e.g. .sum()) of object arrays - #11468

Open
agu2347 wants to merge 1 commit into
pydata:mainfrom
agu2347:fix-object-dtype-full-reduction
Open

Preserve object dtype on full reduction (e.g. .sum()) of object arrays#11468
agu2347 wants to merge 1 commit into
pydata:mainfrom
agu2347:fix-object-dtype-full-reduction

Conversation

@agu2347

@agu2347 agu2347 commented Jul 21, 2026

Copy link
Copy Markdown

Fixes #5024.

Summing (or otherwise fully reducing over all axes) an object-dtype DataArray/Variable converted the result to a numpy fixed-width string/unicode dtype instead of preserving the original object dtype -- e.g. summing an all-string object array produced dtype <U9 instead of object. Reducing over a subset of axes already preserved the dtype correctly, as noted in the issue.

Root cause: numpy's own full-array reduction (no axis argument) over an object-dtype array returns a raw Python scalar rather than a 0-d ndarray -- e.g. np.sum() on a 3x3 object array of the string 'a' returns the plain str 'aaaaaaaaa', not an array. NamedArray.reduce() passed this raw scalar on to from_array(), whose final fallback path is a bare np.asarray(data) with no explicit dtype. numpy then infers a dtype from the scalar itself (a unicode dtype for a str), losing the fact that it originated from an object-dtype reduction.

from_array() already has a precedent for exactly this class of problem: tuple inputs are wrapped via to_0d_object_array() specifically to avoid this dtype misinference. That special-casing can't happen inside from_array() itself for raw scalars in general, though, since from_array() is also used for legitimate direct scalar construction (e.g. xr.DataArray("hello") should still get numpy's inferred unicode dtype) and has no way to know a given scalar came from an object-dtype reduction versus user input.

Fix: handle it at the point where that information is still available -- in NamedArray.reduce(), right after computing the reduction result, if the original array's dtype was object and the result has no .dtype attribute (i.e. it's a raw scalar, not an array), wrap it with the same to_0d_object_array() helper before it reaches from_array().

Testing: verified against the exact reproduction in the issue (before: dtype <U9; after: dtype object, with the value intact), confirmed partial-axis reductions (already correct) are unaffected, and confirmed plain numeric reductions (e.g. summing a float64 array) are completely unaffected. Added a regression test (test_reduce_keeps_object_dtype_on_full_reduction) covering all three cases. I confirmed the test fails with the original code (dtype <U9) and passes with the fix.

Ran the full existing test_namedarray.py, test_variable.py, and test_dataarray.py suites (913 passed, 313 skipped, 2 xfailed, 1 xpassed -- 912 baseline + 1 new test, no regressions).

@github-actions github-actions Bot added the topic-NamedArray Lightweight version of Variable label Jul 21, 2026
… Variable

Rebased onto current upstream main to resolve a merge conflict flagged on the PR (a large rst->md documentation migration landed on main; this rebase doesn't touch any of the migrated doc files, only the same 3 files the PR always changed).
@agu2347
agu2347 force-pushed the fix-object-dtype-full-reduction branch from 1f5e58e to dd02f7a Compare August 11, 2026 06:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic-NamedArray Lightweight version of Variable

Projects

None yet

Development

Successfully merging this pull request may close these issues.

xr.DataArray.sum() converts string objects into unicode

1 participant