create: add --read-special-timeout option - #10090
Merged
ThomasWaldmann merged 1 commit intoAug 13, 2026
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #10090 +/- ##
==========================================
+ Coverage 86.82% 86.92% +0.09%
==========================================
Files 99 99
Lines 17342 17404 +62
Branches 2631 2642 +11
==========================================
+ Hits 15057 15128 +71
+ Misses 1589 1582 -7
+ Partials 696 694 -2 ☔ View full report in Codecov by Harness. |
ThomasWaldmann
force-pushed
the
read-special-timeout-5422
branch
from
August 13, 2026 09:51
633da70 to
2074aa6
Compare
Reading from a fifo (or char device) with --read-special could block borg forever: opening a fifo read-only blocks until a writer connects, and reads block until data arrives - a broken producer thus hangs an unattended backup indefinitely. Now, when no data arrives for --read-special-timeout SECONDS (default: 1800, i.e. 30 minutes; this includes waiting for a fifo's writer to connect), borg gives up on that file: fifos and char devices are opened with O_NONBLOCK and all content reads go through the new SpecialFileReader, which waits for data via select() and raises OSError(ETIMEDOUT) when the timeout expires. That maps to the new BackupTimeoutError (rc 111), so the file is skipped with an error (and not retried) while the backup continues with the remaining files. --read-special-timeout=0 means: wait forever (previous behavior). Note: with a timeout active (the default), a writer connecting but closing without sending anything is reported as a timeout instead of being stored as a file with empty content. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ThomasWaldmann
force-pushed
the
read-special-timeout-5422
branch
from
August 13, 2026 09:56
2074aa6 to
5b9da3b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5422.
With
--read-special, backing up a fifo blocked forever when no writer ever connects: the read-onlyopen()blocks until a writer shows up, andread()blocks until data arrives. To the user it just looks like borg hangs, and an unattended backup never completes.What this does
Adds
borg create --read-special-timeout SECONDS, default 1800 (30 minutes), so users who don't know the option exists are protected too.--read-special-timeout=0keeps the previous wait-forever behavior.O_NONBLOCK(the open then succeeds even without a connected writer); block devices are unaffected and keep the current fast path.SpecialFileReader(helpers/fs.py), which waits for data viaselect()and raisesOSError(ETIMEDOUT)when no data arrives for SECONDS. The timeout bounds the gaps in the data flow, including waiting for the writer to connect - a slow producer that keeps sending never times out, a stalled or never-connecting one does. That matches the request in the issue for something configurable, since a legitimate producer may take a while before it starts sending.BackupTimeoutError(rc 111): the file is skipped with an error, not retried (retrying would just wait SECONDS again, 10 more times), and the backup continues with the remaining files - consistent with how other per-file IO errors behave.Behavior change worth noting (documented in
usage/notes.rst): with a timeout active (now the default), a writer that connects but closes without sending anything cannot be distinguished from "no writer at all", so it is reported as a timeout rather than stored as a file with empty content.No signals and no threads - plain POSIX non-blocking IO, so nothing here interferes with borg's SIGINT handling.
select()rather thanpoll(), because macOSpoll()has historically been unreliable for some character devices.Tests
--read-special-timeoutwithout--read-specialerrors out, and the big-fifo regression test on both read paths.SpecialFileReader(loop-fill, EOF, silent and stalled writer timeouts, fifo no-writer vs. late writer, ESPIPE on seek) andnonnegative_seconds.=0keeps waiting indefinitely; 1 MiB through a bursty fifo round-trips intact.🤖 Generated with Claude Code