check --repair: resync past corrupt object headers when rebuilding the chunks index - #10094
check --repair: resync past corrupt object headers when rebuilding the chunks index#10094mr-raj12 wants to merge 1 commit into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #10094 +/- ##
==========================================
- Coverage 86.82% 85.61% -1.21%
==========================================
Files 99 99
Lines 17342 17398 +56
Branches 2631 2642 +11
==========================================
- Hits 15057 14896 -161
- Misses 1589 1792 +203
- Partials 696 710 +14 ☔ View full report in Codecov by Harness. |
|
Does #10092 (comment) apply here too?
The consequence is in Either is fine by me, just say which. |
a6533ae to
1877f65
Compare
…e chunks index, borgbackup#8476 When check --repair rebuilds the chunks index from the packs, a corrupt object header now makes iter_headers resync rather than raise: it takes a validate function and scans forward for the next object, in 1 MiB windows that overlap by one header so a header on a window boundary is still found. Repository-only checks pass no validate and keep raising IntegrityError on a corrupt header. OBJ_MAGIC also occurs inside payloads, so a candidate is accepted only when it authenticates. For AEAD keys, decrypting the metadata authenticates it against the header's magic, version and chunk_id, so the walk confirms a chunk id from a few hundred bytes. Keys that authenticate by chunk_id == id_hash(content) (id_check_is_authentication) read the whole object and parse() at the "repair" id place; validate.needs_data selects between the two. Authentication needs the key, so check --repair makes it before the rebuild with manifest_only=True. A repair that cannot read the manifest has no key and walks without resyncing.
1877f65 to
cc0d2c0
Compare
Follow-up to #10083 (header validation, #8476).
When
borg check --repairrebuilds the chunks index from the packs, a corrupt object header no longer aborts the whole pack.iter_headers(validate=...)scans forward for the next object, confirms it by authenticating it with the key, and resumes there, so the objects after the damaged region are still indexed. The skipped byte range is logged.Without a validate function (repository-only check, routine rebuild) the behavior is unchanged: a corrupt header raises IntegrityError.
Details:
_find_headersearches for OBJ_MAGIC a window at a time. OBJ_MAGIC also occurs inside payloads, so a candidate is accepted only when its header parses and validate confirms it.resync_validatorbuilds validate from the key: AEAD keys authenticate by decrypting the metadata (header + meta, a few hundred bytes); keys with id_check_is_authentication need the blob data and check chunk_id == id_hash(content).Tested in repository_test, cache_test and check_cmd_test.