Phase 36: continue native Joni parity after PR 1087

dev/design/phase36-regex-parity.md

@@ -64,6 +64,11 @@ Remaining scanner-removal queue:
   match-all text substitutions used to represent deferred properties.
 - [ ] Preserve ordinary/extended-class context and source positions in Joni so
   the adapter no longer scans bracket depth or generates replacement classes.
+- [ ] Replace `hasControlVerbState(String)` with a compiled Joni fact, and move
+  the `\K`-inside-lookaround diagnostic into Joni before deleting the remaining
+  production `analyzePerlSyntax` scan.
+- [ ] Delete test-only routing scanners (`requiresJoniBackend` and its empty-
+  class/name helpers) once their assertions are replaced by direct Joni facts.
 - [ ] Retain only runtime-neutral trusted-callout materialization and documented
   Perl source-policy checks outside Joni; delete the scanner when these gates
   pass.
@@ -82,11 +87,11 @@ repeatable baseline.
 
 ### Phase 1 — Ordinary-pattern Joni parity
 
-- [ ] Close all remaining PR 958 ordinary-pattern regressions.
-- [ ] Prove capture, duplicate-name, branch-reset, region/bounds, zero-width
+- [x] Close all remaining PR 958 ordinary-pattern regressions.
+- [x] Prove capture, duplicate-name, branch-reset, region/bounds, zero-width
   progression, `\G`, `/g`, `/c`, `/o`, substitution, reuse, and nested
   match-state behavior.
-- [ ] Prove byte/Unicode pattern and subject variant selection.
+- [x] Prove byte/Unicode pattern and subject variant selection.
 - [x] Delete the disconnected production backend selector while preserving its
   historical compatibility tests under test scope.
 
@@ -128,7 +133,7 @@ Exit: Unicode and native syntax corpora execute without compatibility masking.
 - [x] Native numeric, named-character, quantifier, group-name, extended-class,
   range, and selected POSIX diagnostics.
 - [x] `pat_re_eval.t` semantic contract.
-- [ ] Finish all remaining same-source `reg_mesg.t` diagnostic families.
+- [x] Finish all remaining same-source `reg_mesg.t` diagnostic families.
 - [x] Finish analyser warning-policy rows.
 - [ ] Finish remaining debug-trace rows.
 - [ ] Refresh complete `regexp.t`, `reg_mesg.t`, and runtime-source gates.
@@ -151,14 +156,16 @@ outside it.
 
 ### Phase 6 — Release and documentation
 
-- [ ] Pass immutable complete 622-file JVM acceptance against PR 958.
+- [ ] Pass immutable complete latest-Perl JVM acceptance against PR 958 by
+  matched path, including every newly synced test (currently 623 files versus
+  the 622-file baseline).
 - [ ] Pass the complete regex-bearing ledger on the interpreter and reconcile
   every JVM/interpreter semantic difference.
 - [ ] Pass direct/thread parity, bounded `pat_psycho*` and `speed*`,
   affected CPAN suites, packaging, notice/license, and warmed performance gates.
 - [x] Pass the five-run warmed ordinary-regex comparison: candidate median
   12.23s versus exact-parent 12.68s under alternating contended runs.
-- [ ] Pass warning-free `make`, Ubuntu, Windows, and complete CI.
+- [x] Pass warning-free `make`, Ubuntu, Windows, and complete CI.
 - [ ] Reconcile `docs/reference/feature-matrix.md`,
   `dev/implementation/regex.md`, and `docs/design/joni-callout-fork.md`
   with shipped behavior.
@@ -169,40 +176,50 @@ Exit: release evidence and public/internal documentation match the code.
 
 ## Current Release Gate
 
-The current integration source composition ends at `702bb2853`. Its
-constituent source tranches are each
-warning-free full-build green. Raw property, warning, escaped-class callback,
-byte-NEL line break, and word-boundary whitespace coverage pass on JVM and
-interpreter. The byte-NEL tranche makes unchanged `regexp.t` 2210/2210 on both
-backends. The native word-boundary tranches make `uniprops10.t`
-20,070/20,070 on both backends, recovering 1,190 assertions from their exact
-parents with no introduced assertion IDs; direct Joni and focused boundary
-oracles are green. One exact combined build of this composition is the current
-checkpoint barrier. The integrated overload-copy increment correction
-passes unchanged `lib/overload_fallback.t` 4/4 on JVM and interpreter,
-adjacent overload gates, and a warning-free full build. The preceding exact
-combined code barrier `18b9a0133` passes packaging, licensed Joni, all unit
-shards, and the shadow JAR. PR 1087 remains the next
-incremental release PR; update it only from an exact warning-free integration
-head whose complete comparator has no pass-count decrease from PR 958 and no
-new invalid, missing, timeout, truncated, incomplete, or zero-TAP row.
-
-The authoritative 622-file, jobs-5, timeout-300 gate at `d3a4bb074` completed
-with 677871/695187 assertions passing and 95 improved rows. Fresh private-tree
-isolation found no regex pass decrease: both `pat.t` variants improve by 147
-passing assertions. Plan/platform drift explains four raw decreases; missing
-private-import inputs explain two new porting errors. The
-`lib/overload_fallback.t` 3/4 decrease is fixed at `cbee747f1`.
-`io/socket.t` is environmental rather than a product decrease: the exact
-one-file runner outside the restricted sandbox passes 25/25 with zero
-errors/timeouts, while the sandbox denies localhost bind and clamps
-`SO_SNDBUF` to 816; focused JVM/interpreter option probes outside it both
-return 32768. There is no remaining reproduced non-regex pass-count blocker.
-The complete runner, comparator, isolation, dossier, 622-file list, reducers,
-and verified checksum manifest are
-durably retained under
-`../PerlOnJava/logs/test_20260821_080900_d3a4bb074_a16_*`; do not repeat the
-full corpus merely to rediscover this evidence.
+PR 1091 is the sole release and user-acceptance target at exact head
+`ff68371af`. It contains all 332 range-diff-equivalent PR 1087 patches rebased
+onto current `master`, preserves the user's latest-Perl sync commit, and includes
+the six release fixes formerly staged in PR 1090. Imported fixtures remain
+authoritative and must not be reverted or patched to recover old counts. The
+exact unified head has a clean warning-free full `make`, and Ubuntu and Windows
+CI are green. The user's complete production-load acceptance is in progress.
+Workers must not mutate, rebase, or push this immutable test branch. Acceptance
+rejects every new invalid, missing, timeout, truncated, incomplete, or zero-TAP
+row and every unresolved PR-958 pass-count decrease.
+
+In the live ten-process run, `pat.t` completed at 1249/1302 in 568.99s,
+`pat_thr.t` reached the 600s outer watchdog, and `anyof.t` reached the 300s
+watchdog. Rerun only `pat_thr.t` and `anyof.t` at exact PR 1091 head with 900s
+per-file timeouts; do not restart the full corpus. The post-1091 runner now
+gives both `pat.t` variants a 900s production-load-safe minimum and the
+`anyof.t` variants a 600s minimum.
+
+In writable current-source state `op/do.t` is 71/71 on both backends;
+`class/accessor.t` executes all 30 rows and improves to 17 passes versus the
+PR-958 12-pass floor; isolated JAPH is deterministically 109/130 with ID 51
+matching current Perl's upstream regression. Aggregate binding warnings and
+fatal construction diagnostics cover the two assertions missing from the
+initial `pat*` localization. The clean serial exact-head gate executes
+all 1302 rows in each file at 1249/1302 for both `pat.t` and `pat_thr.t`, with
+zero errors, timeouts, or incomplete rows.
+
+Exact-head packaging acceptance is complete: full `make`, standalone Joni
+packaging verification, notices/licenses, merged SBOM, and artifact hashes are
+green. The executable tree and ordered 28,518-entry JAR inventory are identical
+to the predecessor CPAN-tested artifact, so its four-module offline matrix
+transfers without a redundant rerun.
+
+Independent implementation continues on `integrate/phase36-post1087-wip` at
+`9bbd3f18e` or later. Unicode property closure, lexical package propagation,
+named-character source mode, and overloaded-subject handling are integrated.
+Typed native Joni debug facts now expose semantic instructions and immutable
+character-class membership. Property rows 1662–1664 and 1685, the nine named-
+character fold/trie rows, and overloaded-subject row 379 are closed in focused
+worker maps. The expected remaining `pat_advanced.t` frontier is debug rows
+1673–1683; an exact combined-head map must confirm that frontier after the
+display-provenance consumer lands. This WIP stays separate until PR 1091
+merges, then rebases onto the new `master` before combined acceptance and its
+PR update.
 
 ### Execution Tracker
 
@@ -224,46 +241,41 @@ full corpus merely to rediscover this evidence.
 
 Active ownership:
 
-- P3: close the ordinary native matcher checklist and implement the largest
-  remaining disjoint matcher-state root, if any.
-- P4: implement the general frontend array-match/substitution compile warning
-  root that accounts for the missing planned `pat.t`/`pat_thr.t` assertion.
-- P5: generate and wire current-Perl InSC/InPC data covering 2,120 residual
-  assertions; the warn-retirement dossier is complete.
-- P6: classify and close the largest remaining native `reg_mesg.t`
-  Parser/Lexer diagnostic family.
+- P3: finish native Perl display provenance for rows 1673–1683 and prove the
+  complete 1,187-row `anyof.t` and 1,687-row `pat_advanced.t` maps.
+- P5: move extended-class string/deferred-property policy and source positions
+  into Joni, delete `validateExtendedPropertyPolicy`, and close the remaining
+  property-specific scanner inventory.
+- P6: own the complete non-property, non-debug parser/diagnostic and first
+  matcher-semantic scanner-removal tranche, starting with replacing the
+  control-verb source scan by a compiled Joni fact.
+- P4: fenced from the parser/scanner tranche pending restoration of its mailbox
+  monitor; it has no authoritative implementation ownership.
 - Coordinator: integration, conflict resolution, immutable acceptance, PR/CI,
   plan state, combined build, and release evidence.
 
 ## Ordered Next Steps
 
-1. Run one exact combined warning-free `make` at the current clean integration
-   head, including P3 a18 and P6 a35.
-2. Refresh focused Unicode,
-   `regexp.t`, `regex_sets.t`, `charset.t`, `pat.t`,
-   `pat_advanced.t`, `pat_re_eval.t`, and `reg_mesg.t` gates on that exact
-   immutable head.
-3. Preserve the documented fixture-invalid and sandbox-environment dispositions.
-   Refresh the comparator evidence for the exact integration
-   head without repeating the already-retained discovery run. Push a head with no PR-958
-   pass-count regression to PR 1087, run CI, and make the incremental PR
-   reviewable.
-4. Continue P4 documentation, P5 generated Indic properties, P6 pat/diagnostic
-   work, and remaining native Unicode/runtime roots in a new WIP
-   PR. Repeat focused tests per semantic tranche and one combined build per
-   integration batch.
-5. Delete remaining production migration scaffolding and prove all constants,
+1. Keep PR 1091 immutable while the user runs the complete latest-Perl corpus,
+   strict PR-958 comparison, additional release checks, and exact-head CI.
+2. Merge PR 1091 directly to `master` only when every checkpoint gate is green
+   or has a documented baseline-equivalent environmental disposition; then
+   close PRs 1087 and 1090 as superseded.
+3. Meanwhile integrate P3 debug/matcher, P4 parser/scanner, P5 property, and P6
+   acceptance deliveries only into the separate post-1091 WIP. Batch focused
+   tests and one combined full build, then rebase this WIP onto merged `master`.
+4. Delete remaining production migration scaffolding and prove all constants,
    closures, conditions, verbs, recursion, dynamic source, byte strings, and
    Unicode strings execute through Joni.
-6. Run complete JVM/interpreter acceptance, direct/thread parity, affected CPAN
+5. Run complete JVM/interpreter acceptance, direct/thread parity, affected CPAN
    suites, five warmed performance samples, packaging, notices/licenses,
    warning-free build, and platform CI.
-7. Reconcile final documentation and remove redundant design material.
-8. After the final implementation PR is merged to `master`, remove automatic
+6. Reconcile final documentation and remove redundant design material.
+7. After the final implementation PR is merged to `master`, remove automatic
    regex `JPERL_UNIMPLEMENTED=warn` injection from
    `dev/tools/perl_test_runner.pl`; rerun the complete corpus; then delete the
    RuntimeRegex warning-plus-never-match downgrade and obsolete tests/docs.
-9. On final `master`, review shipped behavior against
+8. On final `master`, review shipped behavior against
    `pod/perlreref.pod`, `pod/perlrecharclass.pod`,
    `pod/perlrequick.pod`, `pod/perlrepository.pod`, `pod/perlre.pod`,
    `pod/perlretut.pod`, and `pod/perlrebackslash.pod`. Record every
@@ -294,11 +306,16 @@ Active ownership:
   absent or pull latest upstream when present, then run
   `dev/import-perl5/sync.pl`. Sync must include required generated inputs such
   as `unicore/Name.pl`, preserve needed non-regex patches, and be idempotent.
+- Unicode generators must consume the latest checked-out `perl5/` tables and
+  derive their version and source hashes as provenance. Historical source
+  hashes or Perl/Unicode versions must not act as pins that block a valid latest
+  upstream refresh; checked-in output hashes remain reproducibility gates.
 
 ## Final Acceptance
 
 - [ ] Every semantic row passing in PR 958 still passes.
-- [ ] Complete 622-file JVM output is compared file-by-file with PR 958.
+- [ ] Complete latest-Perl JVM output is compared file-by-file with PR 958
+  (the PR 1091 sync currently discovers 623 files).
 - [ ] Complete regex-bearing JVM/interpreter and direct/thread results agree.
 - [ ] Joni covers constants, closures, conditions, control verbs, recursion,
   dynamic source, byte strings, and Unicode strings.

dev/implementation/regex.md

@@ -156,6 +156,15 @@ eligibility through Joni's
 `NamedCharacterCache`; `PerlPropertyValueMatcher` evaluates wildcard value
 expressions without depending on PerlOnJava runtime classes.
 
+Property-value wildcard delimiters follow current Perl's punctuation grammar:
+ASCII punctuation other than `-`, `+`, `_`, and `{` may delimit the value
+subpattern; `(`, `[`, and `<` use their paired closing character, and a
+backslash-escaped opener requires the same escape before its closer. The same
+parser owns Numeric_Value, Block, Name, Age, and the other enumerated property
+families. The resolver also exposes Perl's internal
+`utf8::_perl_surrogate` property as D800-DFFF in both Joni's encoding-domain
+and wide-scalar ranges.
+
 The generator registry is
 `dev/tools/perl_unicode_data_generators.json`. It records the consumed Perl
 checkout, Unicode version, input hashes, generated outputs, and output hashes.