`unpackb()` may return corrupted `ExtraData.extra` for non-contiguous input

[marinelay]

I found an issue in unpackb() while fuzzing Python C extension modules with Atheris and ASan/UBSan enabled.

The sanitizer initially reported memcpy-param-overlap, but the problem is not limited to sanitizer builds. The same input also produces incorrect output with the PyPI wheel, without crashing or emitting a warning.

Environment

• msgpack 1.2.1
• CPython 3.12
• Linux x86_64

Reproducer

import msgpack

# A strided (non-contiguous) view whose logical content is b"\x00abc":
#   \x00 -> the integer 0
#   abc  -> trailing bytes, so unpackb raises ExtraData
buf = bytearray(b"\x00\xffa\xffb\xffc\xff")
view = memoryview(buf)[::2]
assert bytes(view) == b"\x00abc"

try:
    msgpack.unpackb(view)
except msgpack.ExtraData as e:
    print(e.unpacked)   # 0        -- correct
    print(e.extra)      # b'ab\x00' -- expected b'abc'

ASan report

AddressSanitizer: memcpy-param-overlap: memory ranges
  [0x7a2549f75340,0x7a2549f75343) and [0x7a2549f75341,0x7a2549f75344) overlap
    #1 PyBytes_FromStringAndSize
    #2 __pyx_pf_7msgpack_9_cmsgpack_2unpackb  msgpack/_unpacker.pyx:199

This looks like a bug to me, but I don't know the codebase well enough to rule out that it's expected for non-contiguous input.

[ThomasWaldmann]

Reproduced on macOS 15 / arm64, CPython 3.14.6, msgpack 1.2.1 (PyPI wheel) — same output as reported, and also on current main. The pure-Python fallback (MSGPACK_PUREPYTHON=1) is correct; only _cmsgpack is affected. Any non-contiguous view triggers it — [::2] and [::-1] alike.

The underlying problem is a use-after-free. For non-contiguous input, get_data_from_buffer() releases the original view and makes a temporary contiguous copy, so buf points into memory owned by view. unpackb() then releases view in its finally block — freeing that copy — and only afterwards reads buf+off to build the ExtraData payload.

That it is really freed memory, and not just an off-by-one, is easy to show with the macOS allocator's free-poisoning:

PYTHONMALLOC=malloc MallocScribble=1 python repro.py
extra=b'\xaa\xaa\x00'   # expected b'abc'

So arbitrary heap contents can end up in ExtraData.extra. The memcpy-param-overlap ASan reports is the same event seen from the other side: PyBytes_FromStringAndSize re-allocates the just-freed block and copies it onto itself.

Unpacker.feed() uses the same helper but copies via append_buffer() before releasing, so it is unaffected.

One more data point: msgpack 1.1.1 does not silently corrupt on this input — it raises SystemError: <class 'msgpack.exceptions.ExtraData'> returned a result with an exception set (from a BufferError), presumably a different symptom of the pre-#677 buffer handling. So the silent corruption looks specific to 1.2.x; the #677 fix did not cover this path.

Fix plus a regression test: #722

[ThomasWaldmann]

^ made with Claude Opus 5, carefully review please.