bug(merge-tracker): three silent data-loss paths — dropped second update, wiped notes, no intra-run dedup

[Scott-Emberson]

Three paths in merge-tracker.mjs lose user data without an error, a warning, or a non-zero exit. All three are in the duplicate-handling block. data/applications.md is user-layer data that is never regenerated, so each of these is unrecoverable once the TSVs are archived.

1. A second TSV updating the same row in one run is silently dropped

merge-tracker.mjs:730 locates the row to rewrite by its original text:

const lineIdx = appLines.indexOf(duplicate.raw);
if (lineIdx >= 0) {
  ...
  appLines[lineIdx] = updatedLine;
  updated++;
}

duplicate.raw is captured during the initial parse (:567) and never refreshed after the in-place write. syncPdfFlags (:279) does refresh its cached row, so the two paths disagree.

So in a single merge run with two TSVs matching the same tracker row (3.5 → 4.0, then → 4.5): the first update rewrites the line, and the second finds indexOf returning -1, skips the whole block, increments no counter, prints no warning, and still logs its optimistic 🔄 Update line beforehand. The TSV is then archived to merged/ as if it had been applied. The 4.5 evaluation is gone.

This is reachable whenever a batch run re-evaluates the same posting twice, which is exactly what the batch workflow does after a scoring change.

2. The score-upgrade path discards the existing Notes

merge-tracker.mjs:740:

notes: `Re-eval ${addition.date} (${oldScore}→${newScore}). ${addition.notes}`,

duplicate.notes is dropped wholesale. Anything the user recorded with set-status --note (recruiter name, req number, interview context) is erased by a re-evaluation that happens to score higher.

This one compounds: the req/job-number guard added in #1524 reads extractReqNumber(app.notes) at :718 to decide whether two similar titles are genuinely distinct postings. Wiping the notes destroys the evidence that guard depends on, so a later merge is more likely to wrongly collapse two real postings.

3. No intra-run dedup, so one run can create the duplicate it exists to prevent

New rows go to newLines (:777) and are spliced in at the end (:797). They are never pushed to existingApps — line 567 is the only push, inside the initial parse. Duplicate detection at :660-721 therefore only ever sees rows that existed before the run started.

Two parallel workers evaluating the same posting each reserve their own number and write their own TSV. Both merge as new rows in a single run, producing exactly the company+role duplicate the file header claims to prevent. verify-pipeline.mjs then warns about it and dedup-tracker.mjs has to clean it up, at which point one row's status and notes are merged away.

Tests

I could not find coverage for any of the three in tests/merge-tracker.test.mjs or the merge sections of test-all.mjs. The invariants worth pinning, independent of how the fixes are written:

• Two TSVs updating the same row in one run: both updates land, or the second fails loudly. Never a silent no-op with the TSV archived.
• A score upgrade preserves the existing note content alongside the re-eval prefix, and a req number in the old note survives the merge.
• Two TSVs for the same company+role in one run produce one row, not two.
• A merge that no-ops for any reason must not archive the TSV that was not applied.

Happy to send a PR. Flagging first because fix 1 has a design choice in it (refresh the cached row versus track rows by index throughout), and fix 2 has a merge-semantics question about how old and new notes should be combined when both are non-empty.

[LayerC0de]

Field data for bug 2, if it's useful for prioritisation — I hit this on a long-running tracker and patched locally rather than wait.

Scale of the loss on a real tracker: 147 rows, ~112 KB of notes total, 14 rows carrying more than 2,000 characters, and one row holding 18,646 characters — recruiter contact details, a req number, screen and take-home dates, and interview context accumulated over two weeks. All of it in a single Notes cell that a higher-scoring re-evaluation would replace with the Re-eval marker alone.

Confirmed on current main. Same fixture, two worktrees:

before:  recruiter Jane Roe; req JR-10423; screen cleared 2026-07-01
after:   Re-eval 2026-07-15 (4→4.5). rescored after weighting change

The recruiter name, the req number and the screen date are gone, with 🔄 Update printed and exit 0.

On recoverability — this is the part that moved it up my list. data/applications.md is gitignored, so there is no VCS copy, and merge-tracker.mjs writes no .bak where dedup-tracker.mjs and normalize-statuses.mjs both do. There is no sidecar to restore from either. It is genuinely unrecoverable rather than inconvenient, which I think argues for fixing bug 2 ahead of 1 and 3 even though it is the smallest change of the three.

On your merge-semantics question (how to combine when both are non-empty): for what it's worth, the answer that seemed unarguable locally was never discard, and append rather than prepend — the re-eval marker is the newest fact while the existing note reads chronologically from the start. The — and empty-string placeholders need collapsing to the marker alone so no row gains a separator it didn't have before. That sidesteps the harder question of what a merge should mean, at the cost of longer cells.

Also worth noting for bug 2's compounding argument: the #1524 req-number guard reads extractReqNumber(app.notes), so a wipe destroys exactly the evidence that guard depends on. In the fixture above, req JR-10423 is what would have kept a later merge from collapsing two genuinely distinct postings.

Not looking to take this one — you flagged it first and the design calls in bugs 1 and 3 are yours to make. Happy to test a PR against a real tracker if that's useful.