Skip to content

Phase 36: unify regex parity integration - #1042

Merged
fglock merged 60 commits into
masterfrom
integration/phase36-regex-parity
Aug 18, 2026
Merged

Phase 36: unify regex parity integration#1042
fglock merged 60 commits into
masterfrom
integration/phase36-regex-parity

Conversation

@fglock

@fglock fglock commented Aug 18, 2026

Copy link
Copy Markdown
Owner

Summary

  • integrate all 35 ready Phase 36 regex PRs on one branch
  • preserve one squashed review-unit commit per source PR
  • retain the repair commits as separate review and rollback units
  • retain a merge commit as the final whole-integration rollback boundary; do
    not squash or rebase-merge this PR
  • restore PR Fix WWW::Mechanize compiler tooling #958 per-file parity, with exact legacy transcript artifacts
    normalized explicitly

Source PR order

  1. Reporting and control foundation: Report full forced regex backend differential #1006, WIP: Complete Phase 36 regex control semantics #1005, Preserve callbacks in apostrophe-delimited regexes #1007, fix: preserve overloaded regex callback provenance #1010
  2. Performance and safety lane: perf: reuse Joni subject offset tables #1008, fix: preserve properties inside extended regex sets #1009, fix(regex): enable safe Joni backtracking checks #1012
  3. Property/source-policy lane: WIP: Match user Unicode property diagnostics #1011, WIP: Preserve deferred Unicode property provenance #1013, fix: execute inline p modifiers through Joni #1014, fix: preserve regex control escape operands #1015
  4. Join point: fix: integrate Joni Unicode and G-position parity #1016
  5. Strict integration stack: test: expose lossless Unicode property corpus #1018feat(regex): add pinned Script parity #1040

Every source PR number appears in its integration commit subject. Original
head SHAs are recorded in the commit messages.

Merge policy

Use Create a merge commit. Keeping the source-PR and repair commits
preserves review, blame, and bisect history, while the merge commit permits a
whole-integration rollback with git revert -m 1 <merge-sha>.

Repair batch

The final integration keeps eleven unsquashed repair commits covering:

  • concat-assignment overload, tied-fetch, interpolation, and taint semantics
  • named CV replacement and compile-time-displaced CV binding
  • byte-backed eval decoding under lexical unicode_eval
  • undefined hash-interpolation warnings
  • qualified SUPER diagnostics and ordinary glob identity
  • active foreach-cell rebinding and canonical interpreter undef
  • opt-in normalization of the two exact PR Fix WWW::Mechanize compiler tooling #958 transcript artifacts

Validation

  • exactly 35 source-PR commits, one per ready PR
  • AI attribution present on every source-PR commit
  • duplicate and temporary integration commits excluded
  • git diff --check origin/master...HEAD
  • corpus comparator tests: original 13/13 plus artifact-normalization 7/7
  • authoritative feat(regex): add pinned Script parity #1040 Script stack and protected generated-property gates
  • warning-free combined make, including Joni, all five unit shards,
    packaging verification, and shadow jar
  • combined 14-file reducer/control gate: 118/118 on JVM and interpreter
  • final named-CV/GV correction controls: 21/21 on JVM and interpreter
  • op/symbolcache.t restored from 7/8 to 8/8 on both backends
  • fresh 622-file acceptance run plus the targeted corrected blocker rerun
  • normalized PR Fix WWW::Mechanize compiler tooling #958 comparison: zero regression files, 70 improved,
    550 unchanged, two known environmental flakes excluded, +201,634 passing
    assertions
  • focused branch-reset fallback: both pat_advanced variants complete
    1,687 tests at 1,522 passing, +146 each versus PR Fix WWW::Mechanize compiler tooling #958
  • exact PR Fix WWW::Mechanize compiler tooling #958 reconstruction classifies op/do.t duplicated TAP and
    japh/abigail.t's logged extra pass as transcript artifacts
  • stale forced Java/Joni matrix explicitly transferred to post-Phase 36: unify regex parity integration #1042
    native-migration PR fix(regex): advance native Joni pattern semantics #1052, where expected counts will be established from
    the then-current corpus
  • Ubuntu CI
  • Windows CI

Evidence

  • combined build: /tmp/phase36-pr1042-integrated-make.log
  • correction build: /tmp/phase36-pr1042-symbolcache-fix-make.log
  • acceptance log: ../PerlOnJava/logs/test_20260818_180000_1042.log
  • corrected targeted composite:
    /tmp/phase36-pr1042-acceptance-with-symbolcache-rerun.log
  • final normalized comparison:
    /tmp/phase36-pr1042-final-corrected-normalized-compare.log

Review notes

  • Current head is 19569d86d565f98372dbfb979174799290be8e0c.
  • Ordinary lookbehind and ordinary constant branch-reset/subroutine-call
    routing through Java are temporary merge scaffolding. Explicit Joni,
    callbacks, dynamic patterns, conditions, and control verbs remain Joni.
  • Full Joni remains the target architecture; post-Phase 36: unify regex parity integration #1042 native-Joni work is
    intentionally excluded from this release branch.
  • The PR remains draft until current Ubuntu and Windows CI pass.
  • Existing source PRs should remain open until this unified PR is green.

Generated with Codex

fglock added a commit that referenced this pull request Aug 18, 2026
Record PR #1042 and advance the integration tracker to CI and final
forced-backend validation.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
fglock and others added 29 commits August 18, 2026 10:43
Squashed integration of PR #1006 at e1d96ef.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1005 at 2bb3ebc.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1007 at 94cf0ff.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1010 at 3c0189a. The duplicated PR #1007
change is already present as its own integration commit.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1008 at a42ed91.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1009 at 60bd283.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1012 at b80a537.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1011 at 7410cab.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1013 at 9eae6b8.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1014 at 785aa06.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1015 at d5a7b00.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1016 at b43687d. PRs #1008, #1009, and
#1012 are preserved as preceding integration commits; temporary join merges
are intentionally excluded.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1018 at 1d6972a.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1019 at 39be7ed.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1020 at c997374.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1021 at d66a5e9.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1022 at 9cca1c0.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1023 at 5bba948.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1024 at f22de2d.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1025 at 12360ac.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
…tes (#1026)

Squashed integration of PR #1026 at fb297a5.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1027 at bf77f09.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1028 at 5283843.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1029 at badbdfe.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1030 at 5f2ce0f.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1031 at 22b9d32.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1032 at c1a7a99.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1033 at dd5bcd6.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Squashed integration of PR #1034 at 4755b55.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Track the minimal Perl Unicode 17.0 sources through the importer manifest so
generator reproducibility tests pass in clean CI checkouts while retaining
byte-for-byte source, hash, version, and notice verification.

Generated with [Codex](https://openai.com/codex/)

Co-Authored-By: Codex <codex@openai.com>
@fglock
fglock force-pushed the integration/phase36-regex-parity branch from 44adf32 to fc316a0 Compare August 18, 2026 08:44
fglock and others added 7 commits August 18, 2026 11:17
Disable Git text normalization for raw-byte-hashed Unicode generator inputs so
Windows checkouts retain the pinned source bytes validated by the generators.

Generated with [Codex](https://openai.com/codex/)

Co-Authored-By: Codex <codex@openai.com>
Translate the resolver's existing Perl-only Unicode property aliases before
Joni compilation so ordinary default-Joni patterns retain their implemented
Titlecase, POSIX, and Perl identifier semantics.

Generated with [Codex](https://openai.com/codex/)

Co-Authored-By: Codex <codex@openai.com>
Keep the byte-reproduced Block and Numeric_Value Java data sources on LF in
Windows checkouts so their raw-LF generators compare byte for byte.

Generated with [Codex](https://openai.com/codex/)

Co-Authored-By: Codex <codex@openai.com>
Classify the established IsDigit, IsLower, and IsUpper aliases for frontend
translation and splice their generated ranges into an enclosing character
class without nested brackets. This prevents Joni from misparsing adjacent
Perl property aliases in ordinary classes.

Generated with [Codex](https://openai.com/codex/)

Co-Authored-By: Codex <codex@openai.com>
Translate standalone Perl code points above U+10FFFF into exact
internal-marker alternatives before Joni compilation. This preserves
Perl's ANYOFH membership behavior without changing range or negated-class
semantics.

Generated with [OpenAI Codex](https://openai.com/codex/)

Co-Authored-By: OpenAI Codex <codex@openai.com>
Translate Perl's (*:NAME) shorthand to Joni's supported (*MARK:NAME)
form and retain control-verb state publication for REGMARK.

Generated with [OpenAI Codex](https://openai.com/codex/)

Co-Authored-By: OpenAI Codex <codex@openai.com>
Route Perl property aliases through the extended-class resolver, normalize
its generated property names for Joni, and encode control-character escapes
before delimiter-sensitive class parsing.

Generated with [OpenAI Codex](https://openai.com/codex/)

Co-Authored-By: OpenAI Codex <codex@openai.com>
fglock and others added 2 commits August 18, 2026 14:09
Route ordinary automatic-mode patterns through the established Java backend
while preserving mandatory Joni routing for callbacks, recursion, conditions,
lookbehind, and control verbs. Keep the Perl unit corpus explicitly forced to
Joni so the temporary acceptance policy cannot hide unfinished full-Joni
parity.

Verified by the focused nine-file comparison (no regressions versus forced
Java) and a warning-free full make.

Generated with [OpenAI Codex](https://openai.com/codex/)

Co-Authored-By: OpenAI Codex <codex@openai.com>
Temporarily keep ordinary positive and negative lookbehind on the Java backend
while Joni's nested-lookahead admission is incomplete. Patterns containing
callbacks, recursion, conditions, or control verbs continue to require Joni,
and explicit forced-Joni mode remains available for parity work.

This fallback is scheduled for removal once the native Joni lookbehind patch
passes the same corpus.

Generated with [OpenAI Codex](https://openai.com/codex/)

Co-Authored-By: OpenAI Codex <codex@openai.com>
Temporarily keep ordinary branch-reset subroutine-call patterns on the Java
compatibility backend while the native Joni named-call admission patch is in
progress. Mixed patterns containing callbacks, conditions, control verbs, or
other Joni-only constructs continue to require Joni, as does explicit Joni
mode.

This is tracked removal debt for the Phase 36 regex parity integration.

Generated with [Codex](https://openai.com/codex/)

Co-Authored-By: Codex <codex@openai.com>
fglock and others added 11 commits August 18, 2026 18:10
Probe explicit concat-assignment overloads without enforcing generic fallback
after a miss, then retain the ordinary concat path for stringify-only objects.
Preserve copy-constructor isolation for explicit mutators.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <codex@openai.com>
Let the compound-assignment drivers perform the sole lvalue store so tied and
environment proxies retain Perl FETCH/STORE counts and taint metadata. Resolve
tied regex interpolation parts once before singleton or blessed inspection.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <codex@openai.com>
Resolve named sort comparators once at sort entry, distinguish direct named
calls from saved code references, and preserve compiled calls across stash
deletion while following visible typeglob replacements.

Remove duplicate B.pm constant declarations that emitted spurious redefine
warnings. Add standard-Perl-validated coverage for sort redefinition, direct
call versus saved-CV identity, and B constant loading.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <codex@openai.com>
Honor unicode_eval for ordinary byte-backed eval source, retain Perl's
extended UTF-8 delimiter form, and share strict decoding between runtime
and StringParser.

Generated with [Codex](https://openai.com/codex/)

Co-Authored-By: Codex <codex@openai.com>
Preserve quoted-string hash source context through both code generators so
Perl's uninitialized warning includes the hash name and runtime key. Keep the
path lexical-warning-aware and separate from regex interpolation.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Keep lexical package qualification for method lookup while hiding that
internal qualification in missing-method diagnostics.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Reserve the scalar-shaped reference override for explicit readonly
constant.pm proxies so ordinary pseudo-CV exports retain glob identity.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Treat active lexical rebinding as a registry side effect so aggregate alias
cells are not cast back to scalar loop variables by either execution backend.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <codex@openai.com>
Avoid wrapping the shared undef scalar when foreach aliases interpreter
arguments, preserving Perl's observable reference identity.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <codex@openai.com>
Add an opt-in comparison mode for the two independently reconstructed PR 958
log signatures. Always report raw and normalized counts, and prove unrelated
regressions remain visible.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
Keep direct named calls dynamic for runtime glob replacement, while pinning
the parse-time CV when a BEGIN block displaced it during compilation.

Generated with [Codex](https://openai.com/codex/)

Co-Authored-By: Codex <codex@openai.com>
@fglock
fglock marked this pull request as ready for review August 18, 2026 17:30
@fglock
fglock merged commit 1af7828 into master Aug 18, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant