Skip to content

fix(regex): advance native Joni pattern semantics - #1052

Merged
fglock merged 11 commits into
masterfrom
fix/phase36-branch-reset-named-call
Aug 19, 2026
Merged

fix(regex): advance native Joni pattern semantics#1052
fglock merged 11 commits into
masterfrom
fix/phase36-branch-reset-named-call

Conversation

@fglock

@fglock fglock commented Aug 18, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Summary

  • retain each physical named-group definition while branch-reset alternatives share Perl's logical capture number
  • publish duplicate-name arrays from committed Joni matcher state and restore physical spans on backtracking
  • resolve Perl \N{name} escapes natively inside forked Joni
  • admit nested lookahead within Perl fixed and bounded lookbehind
  • route ordinary Perl \K assertions through native Joni instead of the Java marker rewrite
  • emit Perl's exact native-Joni diagnostic for \K inside all four lookaround forms
  • add a Joni character-property range-resolver API and parse pinned General_Category assignments inside Joni
  • retain automatic Java routing for branch-reset subroutine calls until the forced-Joni corpus gate authorizes removing that temporary guard

Stack

PR #1042 is merged. This draft is rebased directly onto master at merge commit
1af78283ff84afff600b846676cb439c2052b3ba. PRs #1054–#1056 are stacked on
this branch and remain separate review slices.

Current validation

  • post-rebase warning-free full make at 86e474835: 17 tasks, all direct Joni tests, unit shards, packaging, and shadow JAR green in 7m02s
  • forced-Joni branch-reset reducers: 13/13 on JVM and 13/13 on interpreter
  • master automatic-routing contract retained: branch-reset subroutine calls stay on Java until their combined imported-corpus gate passes
  • combined focused native-Joni matrix before rebase: 217/217 on JVM and 217/217 on interpreter
  • standard Perl physical-publication oracle: 3/3
  • standard Perl native named-character oracle: 11/11
  • forced-Joni named-character focused suite: 27/27 on both backends
  • direct Joni named-character resolver suite: 6/6
  • standard Perl Unicode-property oracle: 8/8
  • forced-Joni General_Category/property gate: 156/156 on both backends
  • native KEEP-lookaround oracle: 43/43 on standard Perl and both product backends; direct Joni 4/4
  • git diff --check
  • Ubuntu and Windows CI at current head
  • refresh the complete four-leg Java/Joni × JVM/interpreter 80-file perl5_t/t/re matrix from one current binary; require all 80 files and classify every timeout, zero-TAP result, and Java-passing/Joni-failing assertion before removing temporary routing

Deferred forced-backend gate

JPERL_REGEX_BACKEND=java perl dev/tools/perl_test_runner.pl --timeout 300 --jobs 5 --output /tmp/phase36-regex-java-jvm.json perl5_t/t/re
JPERL_REGEX_BACKEND=joni perl dev/tools/perl_test_runner.pl --timeout 300 --jobs 5 --output /tmp/phase36-regex-joni-jvm.json perl5_t/t/re
JPERL_REGEX_BACKEND=java JPERL_INTERPRETER=1 perl dev/tools/perl_test_runner.pl --timeout 300 --jobs 6 --output /tmp/phase36-regex-java-interpreter.json perl5_t/t/re
JPERL_REGEX_BACKEND=joni JPERL_INTERPRETER=1 perl dev/tools/perl_test_runner.pl --timeout 300 --jobs 6 --output /tmp/phase36-regex-joni-interpreter.json perl5_t/t/re

The old frozen counts are not current expectations because generated-property
coverage and the native Joni surface have changed. Remaining follow-ups are
separate slices: remove the redundant KEEP adapter guard, preserve per-member
fold policy in composed classes, migrate property wildcards, and remove each
temporary Java route only after its native differential gate is green.

@fglock fglock changed the title fix(regex): preserve physical branch-reset named captures fix(regex): advance native Joni pattern semantics Aug 18, 2026
This was referenced Aug 18, 2026
Merged
Draft
Draft
fglock and others added 10 commits August 18, 2026 19:55
@fglock @codex
fix(regex): support branch-reset named subroutine calls
37667aa 
Preserve the leftmost physical capture target when branch-reset alternatives
reuse a logical group number, admit duplicate-name calls only for the
PerlOnJava syntax, and retain caller captures after multiplex subroutine calls.

Add direct Joni and Perl regression coverage for named, numeric, duplicate-name,
capture-publication, and ordinary-capture behavior.

Generated with [Codex](https://openai.com/codex/)

Co-Authored-By: Codex <codex@openai.com>
@fglock @codex
fix(regex): preserve physical branch-reset named captures
4efdfef 
Track each named-group definition independently inside the forked Joni
bytecode matcher while retaining Perl's shared logical branch-reset numbering.
Publish duplicate-name arrays from committed physical spans, restore them on
backtracking, and keep subpattern calls from publishing capture slots.

Generated with Codex (https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
@fglock @codex
feat(regex): parse named characters natively in Joni
c22cf1f 
Add an optional syntax-level named-character resolver and let the Perl lexer
emit code-point tokens for braced named escapes inside and outside classes.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
@fglock @codex
refactor(regex): route named characters through Joni
3f576e4 
Inject the Perl named-character resolver through the Joni syntax and retire
the frontend textual rewrite while preserving Unicode and byte-mode behavior.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
@fglock @codex
fix(regex): admit nested lookahead in Perl lookbehind
74e5804 
Allow positive and negative lookahead anchors within Perl-syntax
lookbehind while preserving the existing Ruby-syntax restrictions.
Add direct Joni and standard-Perl regression coverage for semantics,
captures, bounded lengths, malformed patterns, and syntax isolation.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <codex@openai.com>
@fglock @codex
refactor(regex): route Perl KEEP assertions through Joni
fa52767 
Admit real Perl KEEP assertions to the native Joni opcode, reject KEEP inside
lookaround, and retire the Java marker capture and replacement bookkeeping.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
@fglock @codex
feat(joni): accept character-property range resolvers
05b5b2f 
Let syntax owners resolve property names to validated native code-point ranges
while preserving JCodings fallback behavior and Joni's parser ownership.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <codex@openai.com>
@fglock @codex
refactor(regex): parse pinned general categories in Joni
7f4bf39 
Pass Perl 5.44 General_Category assignments to Joni unchanged and resolve them
through pinned native ranges instead of expanding property text in the adapter.

Generated with [OpenAI Codex](https://openai.com/codex)

Co-Authored-By: OpenAI Codex <codex@openai.com>
@fglock @codex
docs(regex): describe native Joni migration boundaries
560261b 
Record native nested lookahead in lookbehind, KEEP routing, and the
General_Category range-resolver boundary in the feature matrix.

Generated with Codex (https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
@fglock @codex
fix(regex): reject KEEP inside native Joni lookaround
aae6d0a 
Track Perl lookaround analysis state so the native KEEP anchor reports
Perl's exact compile-time diagnostic in all four assertion forms while
preserving non-Perl Joni behavior and subexpression-call boundaries.

Generated with [Codex](https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
@fglock
fglock changed the base branch from integration/phase36-regex-parity to master August 18, 2026 17:56
@fglock
fglock force-pushed the fix/phase36-branch-reset-named-call branch from 62d34ae to aae6d0a Compare August 18, 2026 17:56
@fglock @codex
fix(regex): retain branch-reset routing gate
86e4748 
Keep automatic branch-reset subroutine-call routing on Java until the native
implementation passes its combined imported-corpus gate. Forced Joni mode
continues to exercise the new engine support directly.

Generated with Codex (https://openai.com/codex)

Co-Authored-By: Codex <codex@openai.com>
@fglock
fglock marked this pull request as ready for review August 18, 2026 19:10
@fglock
fglock merged commit 0335e25 into master Aug 19, 2026
2 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@fglock