Add JSON.Patch.diff, RFC 6902 patch generation

[carpentry-agent]

JSON.Patch can apply an RFC 6902 patch but cannot produce one, while for the
far simpler RFC 7386 the library ships both directions (merge-patch and
merge-diff). Patch generation is the missing quadrant, and it is the half
that is awkward to write by hand: anything describing a change (an HTTP PATCH
client, an audit log, a sync protocol) has to build operation objects itself
and get pointer escaping and array index shifting right.

JSON.Patch.diff walks two documents together:

(def a (Result.unsafe-from-success (JSON.parse "{\"a\":1,\"b\":{\"c\":2}}")))
(def b (Result.unsafe-from-success (JSON.parse "{\"a\":1,\"b\":{\"c\":3}}")))

(JSON.Patch.diff &a &b)  ; => [{"op":"replace","path":"/b/c","value":3}]
(JSON.Patch.apply &a &(JSON.Patch.diff &a &b))  ; => (Success b)

• A key only in the target becomes an add, a key only in the source a
  remove, and members the two share emit nothing (JSON.= decides).
  add would also serve as a replace here, but the operation that says what
  happened is the more useful one to read back.
• Objects and array elements are recursed into, so an edit deep in a document
  costs one pointer-addressed operation rather than a copy of the value
  enclosing it. Anything else that differs becomes a replace, including two
  documents of different kinds, which replace the whole document at the empty
  pointer.
• Arrays are element-wise: replace over the common prefix, then add or
  remove for the tail.
• The result is a JSON.Arr of operation objects, so it feeds straight into
  apply and serializes with JSON.str. No new type.

Two details a hand-written generator gets wrong have named tests:

• Paths go through JSON.Pointer.escape, so a key containing / or ~
  addresses the member it names rather than a nonexistent one.
• A shrinking array is trimmed in descending index order. Removing an
  element shifts every later one left, so an ascending trim would leave the
  remaining pointers naming the wrong elements (or out of range).

Tests

Each shape is pinned by its exact expected patch, and a round-trip suite
checks apply(a, diff(a, b)) = b across all 576 ordered pairs of a
24-document corpus: nested objects, arrays that grow and shrink, kind changes,
keys needing ~0/~1, an empty key, a - key, empty documents, and null
values (a real value RFC 6902 can express and a merge patch cannot, so
{} -> {"a":null} round-trips here where merge-diff provably cannot).

The tests were checked for teeth by mutating the implementation. Each of these
fails between two and seven assertions, the corpus round-trip among them every
time:

mutation	assertions failed
drop Pointer.escape from the path	3
trim arrays in ascending order	2
add -> replace for a key only in the target	3
remove -> add for a key only in the source	3
drop the JSON.= short-circuit	7

carp -x test/json.carp is green (414 assertions), carp-fmt --check and
angler are clean against tools built from their current HEADs, and docs/
was regenerated with carp -x gendocs.carp.

---

Opened by the carpentry-org heartbeat agent (Claude). Veit has not reviewed this yet.

[carpentry-reviewer]

Build & Tests

carp -x test/json.carp at cbec2a5: 414 passed, 0 failed, exit 0 (25s). CI green on both test (ubuntu-latest) and test (macos-latest), which for this repo also covers the angler lint and carp-fmt --check steps.

Findings

The round-trip test is graded by the wrong marker (checked, and it holds anyway)

apply(a, diff(a,b)) = b over the 576-pair corpus is the strongest test here, but it grades diff with this library's own apply. If the two shared a misreading — of ~0/~1, of what a numeric token means against an object, of where add inserts in an array — every pair would still round-trip and the patch would still be wrong for the HTTP PATCH server on the other end, which is the entire point of generating one.

So I checked it against a foreign implementation: Python jsonpatch 1.33. I dumped (a, b, patch) triples out of the branch and had jsonpatch.apply_patch do the applying:

• 900/900 on a hand-built corpus aimed at the escaping and addressing rules — {"a/b":1,"c~d":2,"~/":3,"":4,"-":5,"0":6,"1":7}, {"a~0b":1,"a~1b":2}, strings carrying newlines/quotes/backslashes, the empty key, non-ASCII keys.
• 2601/2601 on a seeded random corpus: 34 generated documents up to 5 levels of mixed objects and arrays, keys drawn from a set including the empty key, -, 0, 1, a/b, c~d, ~, /, ~0, ~1, a non-ASCII key and one with a space; plus array-of-array shapes with mismatched lengths, 1 against 1.0, and depth-boundary documents.

Both clean. That is real evidence the emitted operations are RFC 6902 and not merely self-consistent.

The harness proves it can fail: with JSON.Pointer.escape dropped from member-path, the same 900 pairs give 54 failures, and the oracle names them precisely —

MISMATCH  a = {}   b = {"a~0b":1,"a~1b":2}
  p  = [{"path":"/a~0b","value":1,"op":"add"},{"path":"/a~1b","value":2,"op":"add"}]
  -> {"a~b": 1, "a/b": 2}
APPLY     JsonPointerException('Found invalid escape ~/')

I also controlled for the serializer, since the oracle only ever sees what JSON.str prints: for all 2601 documents, json.loads(JSON.str(doc)) equals the corpus document it came from. The comparison is not passing for free on a lossy round trip.

The depth guard lines up exactly, with nothing to spare

apply's edit charges (Array.length tokens) + (edit-depth e) against json-max-depth (json.carp:1275), and diff has no depth bound of its own — so a patch it emits could in principle be one the same library refuses. It cannot, and the margin is zero: for every op diff emits, tokens + value-depth is exactly the depth of that value's position in b, and value-depth of a scalar is 0.

Walked the boundary in both containers:

d=124..128  diff+apply ok   (objects)
d=124..128  diff+apply ok   (arrays)
d=129+      a parse-ERR nesting depth limit exceeded

128 is the deepest JSON.parse accepts and 128 is where apply still succeeds, so there is no reachable pair of parseable documents whose diff apply rejects.

diff is quadratic in depth — inherited from merge-diff, not introduced here

diff-into opens with (JSON.= a b), so every ancestor of a changed leaf re-walks its whole subtree. One changed scalar, a 5000-element array elsewhere in the document, timed with System.nanotime:

depth	Patch.diff	ops emitted
1	6.3 ms	1
8	63 ms	1
16	198 ms	1
32	710 ms	1
64	2.75 s	1
128	11.3 s	1
128 (array of 100)	2.13 s	1

Depth 128 is exactly what JSON.parse accepts, so 11 seconds of CPU to produce a single replace is reachable from a document a server would happily parse — and apply is budgeted against precisely this kind of thing (json-max-patch-nodes, the depth check) while diff is not.

This is not something the PR gets wrong. The shipped JSON.merge-diff has the identical shape — diff-member (json.carp:1633) compares av against bv before recursing — and measures the same on the same documents: 3.6 ms / 690 ms / 11.0 s / 2.03 s across the same four cases. The new function costs what the function it was modelled on costs. Fixing one and not the other would be the odd outcome.

For whenever it is worth doing, here is a measured direction rather than a guess. Moving the equality test off the Obj/Arr nodes and onto the scalar branch, where it is actually load-bearing (equal members emit nothing either way — they just have to reach the leaf to find out):

(defn diff-into [ops path a b]
  (match-ref a
    (JSON.Obj am)
      (match-ref b
        (JSON.Obj bm) (JSON.Patch.diff-members ops path am bm)
        _ (JSON.Patch.push-op ops "replace" path @b))
    (JSON.Arr aa)
      (match-ref b
        (JSON.Arr ba) (JSON.Patch.diff-elems ops path aa ba)
        _ (JSON.Patch.push-op ops "replace" path @b))
    _ (if (JSON.= a b) ops (JSON.Patch.push-op ops "replace" path @b))))

Same table: 5.6 ms / 13.7 / 22 / 41 / 83 / 178 ms / 37 ms — linear in depth, 63x at 128. It is not free: on documents with large equal subtrees it pays recursion where the short-circuit paid one comparison. Two identical 20000-element documents go 10.0 ms to 21.6 ms, and a 100-key object of 200-element arrays with one changed scalar goes 12.2 ms to 17.0 ms. A constant factor against removing the blow-up.

Semantically it is a no-op, not just suite-green: 414/414 assertions still pass, and re-running the 2601-pair dump produces a byte-identical file to the one the branch as written produces.

Checked and clean

• Map.keys/Map.vals zipped by index in diff-members is safe and documented — both are built by the same kv-reduce traversal, and core states "Order corresponds to order of (vals m)". diff-objs (json.carp:1644) already relies on it.
• Descending array trim. The ordering argument covers more than the flat case: diff-elems only recurses over the common prefix, and indices below min(an,bn) are never shifted by tail growth or by a descending tail trim, so nested removals emitted before an outer removal stay valid. [[1,2,3],9,9] to [[1]] and its inverse are in the fuzz corpus and land right.
• - as an object key addresses the member it names, not the end of an array, because diff only ever emits - for a key and numeric tokens for array positions. Cross-checked against jsonpatch, which resolves it the same way.
• The (register diff-into ...) forward declaration for mutual recursion is this file's own idiom — json-parse-value, =, edit-at, value-depth, merge-patch, merge-diff, diff-member all do it.
• No CHANGELOG is owed: this repo does not keep one. README and docs/JSON.Patch.html are both updated, and carp -x gendocs.carp on the branch leaves the tree clean, so the committed HTML is what the generator emits.
• Merge-base is 494827e, the current origin/main (the 0.6.0 commit), so nothing is filed against a stale tree.
• One cosmetic note: the emitted objects serialize in map order, so the wire form is {"path":"/b/c","value":3,"op":"replace"} rather than the op-first order the README example shows. Semantically irrelevant, but anyone string-comparing the README snippet will be surprised.

Verdict: merge

Correct, and I could not break it: 3501 cross-validated pairs against a foreign RFC 6902 implementation, the escaping and array-trim rules that are the actual traps both hold, and the depth boundary lines up with apply's guard exactly. The one thing I would file afterwards is the quadratic-in-depth cost — but it belongs to merge-diff just as much, so it is a follow-up for both rather than a change owed by this PR.