Repository: don't mask the original exception when unwinding with buffered chunks

[ThomasWaldmann]

Problem

Repository.close() asserts that the PackWriter has no unflushed chunks. When a command aborts with chunks still buffered, the with repository: unwind calls close() and that assertion raises AssertionError, masking the original exception. Buffered chunks also leave F_PENDING entries in the chunk index, which the close()-time index persist asserts on (chunk ... has no pack location yet).

Reproduce: open a Repository, put() one small chunk (below the pack size limit), raise inside the with-block — the AssertionError replaces the real error.

This affects the paths that put chunks without a Cache:

• ArchiveChecker.add_reference() — borg check --repair
• borg debug put-obj

Commands that use a Cache are not affected: Cache is the inner with, so Cache.close() unwinds first and flushes the pack writer via _maybe_write_chunks_index(force=True). ArchiveChecker.finish() flushes as well since #10072, but only on the success path, so an abort before that still reaches close() with a non-empty buffer.

Fix

On exception unwind, Repository.__exit__ calls PackWriter.discard() — the abort-side counterpart to flush():

• it joins a still in-flight pack store first, so a pack that was already stored gets recorded in the index. Joining first also means dropping buffered entries cannot break update_pack_info() for a chunk id that sits in the in-flight pack and in the buffer (dropping first would delete the shared index entry, update_pack_info() would then KeyError mid-pack and leave F_PENDING leftovers for the persist to trip over — masking the original error again through a different door).
• then it drops the buffered pieces and their still-pending index entries: those chunks were never stored and die with the aborted operation. Store errors from the join are logged, not raised.

close()'s assertion is unchanged: on a clean close it still catches a genuinely forgotten flush(). __exit__ is the only place that knows whether we are unwinding, so the discard decision lives there.

All abort-time index cleanup goes through one helper, PackWriter._drop_index_entries():

• it never builds the chunk index from the repo — that I/O can fail and mask the error being unwound. With no in-memory index there is nothing to delete: add() installs a chunk's entry before buffering its piece, so pending entries never outlive a dropped index.
• it only deletes entries that are still pending — a resolved entry means the chunk is in a stored pack; only the aborted duplicate piece dies.
• _apply_outcome() gets the same no-index guard, so joining a pack store while aborting cannot trigger an index rebuild either.

close() itself could still mask the original error a few lines further down: the close-time chunk index persist and the lock release both do store I/O, which fails again exactly when the abort was a store failure. Both are now logged instead of raised (the persisted index is only a cache and gets rebuilt; an unreleasable lock goes stale eventually). The lock release and store close now run in a finally block, so a close-time error — including the unflushed-chunks assertion — cannot leak the exclusive lock anymore.

Also: dropped buffer pieces are logged (debug level), the abort semantics are documented in docs/internals/packs.rst, and ChunkIndex.is_pending/F_PENDING were added to the .pyi stub.

Tests

• test_exception_unwind_drops_buffered_chunks — the base repro: asserts the original ValueError (not AssertionError) propagates, and that after reopening the buffered chunk is neither in the chunk index nor readable.
• test_exception_unwind_records_inflight_pack_drops_buffer — unwind with one pack in flight and more chunks buffered, including a chunk id that is in both: the stored pack's chunks stay recorded and readable, only the never-stored chunk dies. Fails without the join-before-drop ordering.
• test_exception_unwind_survives_failing_index_persist — the store dies mid-operation: the close-time index persist fails, is logged, the original exception propagates, and the lock is still released. Fails without the guarded persist.
• test_exception_unwind_does_not_rebuild_dropped_chunk_index — buffered chunks plus a dropped index: asserts no chunk-index rebuild happens during unwind and the original exception survives.
• test_close_with_unflushed_chunks_asserts — documents that a clean exit with buffered chunks still trips the assertion, and that even that failing close releases the lock and closes the store.

Rebased onto current master (b6b8ec062); the only conflict was context in hashindex.pyi against the prefix-iteritems change. Full test suite passes, ruff clean (one pre-existing ruff format divergence at repository.py:1173 is unrelated — newer local ruff).

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 82.60870% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.76%. Comparing base (bf6a449) to head (c15f87f).
⚠️ Report is 5 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/borg/repository.py	82.60%	7 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #10013      +/-   ##
==========================================
- Coverage   86.77%   86.76%   -0.01%     
==========================================
  Files          98       98              
  Lines       17277    17299      +22     
  Branches     2622     2624       +2     
==========================================
+ Hits        14992    15010      +18     
- Misses       1587     1592       +5     
+ Partials      698      697       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.