Advance native Joni numeric, class-range, and branch-reset parity

dev/design/phase36-regex-parity.md

@@ -145,6 +145,17 @@ affected corpus before taking another slice.
   when the complete match fails, remain visible when another alternative
   succeeds, and commit when destructive control verbs cut the path, including
   across a dynamic continuation.
+- Native Joni parses bounded decimal, braced, relative, and overflow numeric
+  backreferences without Java-side rewriting. The focused four-backend fixture
+  passes 63/63, and the exact `regexp.t` numeric tranche has zero introductions.
+- Native Joni distinguishes non-ASCII unfinished ranges and accepts Perl false
+  class ranges around `\d`, `\s`, and POSIX classes. The focused system-Perl,
+  direct-Joni, four-backend, and imported-row gates agree.
+- Physical branch-reset names now bind named calls and named conditions to the
+  correct definition without changing numeric condition operands. The combined
+  numeric-backreference, class-range, and branch-reset boundary passes full
+  `make`; exact `regexp.t` is 2,074/2,210, fixing 57 identities from the prior
+  integrated boundary with no newly failing identity.
 
 ## Execution Phases
 
@@ -256,38 +267,39 @@ behavior.
 
 ## Ordered Next Steps
 
-1. Integrate the native `/xx`, adjacent full-fold partition, and `L_` alias
-   batch on the exact merged predecessor. Run one warning-free full build plus
-   zero-introduction `regexp.t` and fold/property gates, then publish a review
-   PR and require exact-head Ubuntu/Windows CI.
-2. Complete recursive and runtime `(??{...})` as native nested Joni execution:
-   preserve captures, `$^R`, `pos`, modes, byte/Unicode provenance, callback
-   unwind, backtracking re-evaluation, and recursion safety. Route every embedded
-   closure to Joni and delete constant inlining, progressive errors, and the
-   dynamic Java adapter as their gates pass.
-3. Complete byte/Unicode pattern provenance through runtime interpolation and
-   template composition, then finish `/d`/`u`/`a`/`aa` forward/reverse literal,
-   class, property, and backreference folding from generated data. Require
-   direct Joni plus ordinary/forced JVM/interpreter zero-introduction gates.
-4. Finish the remaining lexical `use re 'strict'`, unescaped-brace, and non-hex
-   diagnostic families. Refresh complete
-   `reg_mesg.t`, `pat.t`, and `pat_advanced.t` maps after each combined batch.
-5. Repeat the four-leg 80-file Java/Joni × JVM/interpreter matrix on the exact
-   successor artifact and compare every file with the PR 958 log. Resolve every
-   regression, zero-TAP record, timeout, truncation, or incomplete file before
-   user acceptance.
-6. Remove each proven-obsolete regex transformation from `dev/import-perl5`
-   sync sources, regenerate a private unpatched corpus twice, prove byte-for-byte
-   idempotence, and run the affected upstream tests without editing them.
-7. Use the refreshed impact report to move all remaining ordinary constants to
-   native Joni, deleting their Java routes and matcher-semantic preprocessing in
-   the same validated slices. Keep `pat_re_eval.t` at 555/555 throughout.
-8. Delete Java matching, selector, fallback state, and unreachable preprocessors;
-   then run direct/thread regex, CPAN, performance, packaging, notice/license,
-   warning-free build, Ubuntu, Windows, and full CI gates.
+1. Complete and integrate the active non-overlapping successor lanes: dynamic
+   source boundaries, ordinary repeated-capture clearing, native named-character
+   and escape diagnostics, and whole-pattern recursion. Preserve each lane as
+   reviewable semantic commits, then run one warning-free full `make`, four-leg
+   focused fixtures, and an exact zero-introduction `regexp.t` comparison on the
+   combined SHA.
+2. Close the remaining capture/region identities, including inactive named and
+   numeric branch-reset slots, final successful quantified iterations, failed
+   alternatives, recursive-frame publication, and nested match-state restore.
+3. Move remaining Perl named-character, escape, strict-mode, brace, control-
+   character, and warning semantics into Joni lexer/compiler internals. Remove
+   each corresponding `JoniRegexPattern`/preprocessor rewrite in the same gated
+   slice; keep only source-policy and final diagnostic rendering outside Joni.
+4. Finish whole-pattern recursion `(?R)`, recursive numbered/named calls,
+   recursion conditions, capture publication, and recursion safety. Keep
+   runtime `(??{...})`, callback unwind, and `pat_re_eval.t` 555/555 green.
+5. Finish `/d`/`u`/`a`/`aa` forward/reverse literal and backreference folding
+   from generated data, then rerun complete Unicode, `pat.t`, `pat_advanced.t`,
+   `reg_mesg.t`, and bounded speed/psycho gates on one immutable artifact.
+6. Use the refreshed impact ledger to close every remaining semantic regex
+   identity and move all ordinary constants to Joni. Reject zero-TAP, timeout,
+   truncated, incomplete, JVM/interpreter, or direct/thread mismatches.
+7. Remove proven-obsolete `dev/import-perl5` regex patches, rerun targeted sync
+   twice, prove byte-for-byte idempotence, and validate the restored unchanged
+   upstream tests.
+8. Delete Java matching, the backend selector, fallback state, matcher-semantic
+   preprocessors, and unreachable adapter code. Prove performance, CPAN,
+   packaging, notice/license, and warning-free build gates before removal is
+   accepted.
 9. Update the feature matrix and final as-implemented/fork documents, remove or
-   summarize redundant design documents, rebase the final stack on `master`, and
-   run the complete PR 958 parity audit before declaring Phase 36 complete.
+   summarize redundant design documents, rebase each final PR on `master`, pass
+   Ubuntu/Windows CI, and compare the complete runner output file-by-file with
+   the immutable PR 958 baseline.
 
 ## Parallel Work
 
@@ -341,7 +353,7 @@ failure blocks backend removal, not semantic fixes.
 
 - [x] Phase 0 — reproducible differential baseline
 - [ ] Phase 1 — ordinary-pattern Joni parity
-- [x] Phase 2 — conditions and backtracking-visible state
+- [ ] Phase 2 — conditions and backtracking-visible state
 - [ ] Phase 3 — Unicode and native pattern syntax
 - [ ] Phase 4 — runtime source and diagnostics
 - [ ] Phase 5 — remove migration scaffolding
@@ -375,6 +387,15 @@ gates may reopen it if a semantic regression appears.
 - [x] Native branch reset and removal of its capture-map adapter
 - [x] Native plain `\N` non-newline atom and interval forms
 - [x] Native recursive/runtime `(??{...})` and removal of dynamic adapters
+- [x] Native bounded numeric backreference parsing and removal of Joni-side
+      brace-backreference rewriting
+- [x] Native Perl false-class ranges and unfinished non-ASCII range diagnostics
+- [ ] Final-iteration, optional, alternation, and failed-path capture clearing
+- [x] Physical branch-reset named calls and conditions
+- [ ] Inactive branch-reset slot publication
+- [ ] Native named-character whitespace/missing-brace/comment diagnostics and
+      removal of the duplicate Java translation path
+- [ ] Whole-pattern `(?R)` recursion and recursive capture publication
 - [ ] Retire proven-obsolete `dev/import-perl5` regex patches
 - [ ] Refresh the complete Unicode, `pat.t`, `pat_advanced.t`, `reg_mesg.t`, and
       80-file forced-Joni gates on one integrated artifact

src/main/java/org/perlonjava/runtime/regex/JoniRegexPattern.java

@@ -807,22 +807,6 @@ private static String translatePattern(String pattern, RegexFlags flags,
                         continue;
                     }
                 }
-                // In Perl, \g{name} is a backreference.  Ruby/Oniguruma uses
-                // \g<name> for a subexpression call and \k<name> for the
-                // backreference, so passing the brace form through makes Joni
-                // diagnose it as an invalid subexpression call.  Numeric and
-                // relative brace forms follow the same translation.
-                if (!inClass && pattern.startsWith("\\g{", i)) {
-                    int end = pattern.indexOf('}', i + 3);
-                    if (end > i + 3) {
-                        String backreference = pattern.substring(i + 3, end);
-                        if (isValidPerlBraceBackreference(backreference)) {
-                            out.append("\\k<").append(backreference).append('>');
-                            i = end;
-                            continue;
-                        }
-                    }
-                }
                 out.append(ch);
                 escaped = true;
                 continue;
@@ -949,22 +933,6 @@ private static String translatePattern(String pattern, RegexFlags flags,
         return out.toString();
     }
 
-    private static boolean isValidPerlBraceBackreference(String content) {
-        if (content.isEmpty()) return false;
-        int start = content.charAt(0) == '-' ? 1 : 0;
-        if (start == content.length()) return false;
-        if (start == 1 || Character.isDigit(content.charAt(0))) {
-            for (int i = start; i < content.length(); i++) {
-                if (!Character.isDigit(content.charAt(i))) return false;
-            }
-            return true;
-        }
-        for (int i = 0; i < content.length(); i++) {
-            if (Character.isWhitespace(content.charAt(i))) return false;
-        }
-        return true;
-    }
-
     private static void appendResolvedNamedCharacter(StringBuilder out, int codePoint,
                                                       RegexFlags flags) {
         boolean extendedSyntax = flags.isExtended()

src/main/java/org/perlonjava/runtime/regex/RuntimeRegex.java

@@ -824,7 +824,8 @@ private static synchronized RuntimeRegex compileSynchronized(
                 }
                 if ("invalid backref number/name".equals(e.getMessage())
                         || "invalid backref number".equals(e.getMessage())) {
-                    throw new PerlCompilerException("Reference to nonexistent group");
+                    throw new PerlCompilerException(
+                            "Reference to nonexistent group in regex");
                 }
                 String invalidProperty = invalidUnicodePropertyName(e.getMessage());
                 if (invalidProperty != null) {

src/test/resources/unit/regex/branch_reset_capture_semantics.t

@@ -0,0 +1,41 @@
+use strict;
+use warnings;
+use Test::More;
+
+plan skip_all => 'GH 20653 semantics require Perl 5.44+' if $] < 5.044;
+
+ok('bbab' =~ /(?|(?<a>a)|(?<b>b))\1(?&a)(?&b)/,
+    'branch-reset numbered and named subroutine targets share captures');
+ok('byb' =~ /(?|(?<a>a)|(?<b>b))(?(<a>)x|y)\1/,
+    'named condition sees unset sibling capture');
+ok(!('bxb' =~ /(?|(?<a>a)|(?<b>b))(?(<a>)x|y)\1/),
+    'named condition rejects wrong unset-capture branch');
+ok('axa' =~ /(?|(?<a>a)|(?<b>b))(?(<a>)x|y)\1/,
+    'named condition sees set capture');
+
+'a' =~ /(?|(?<a>a)|(?<b>b))/;
+is("$1-$+{a}-" . (defined $+{b} ? $+{b} : ''), 'a-a-',
+    'first branch publishes only its named capture');
+'b' =~ /(?|(?<a>a)|(?<b>b))/;
+is("$1-" . (defined $+{a} ? $+{a} : '') . "-$+{b}", 'b--b',
+    'second branch publishes only its named capture');
+
+for my $case (
+    ['preabcpost', 'a-b-c'],
+    ['predepost', 'd-e-'],
+    ['prefpost', 'f--'],
+) {
+    my ($subject, $expected) = @$case;
+    $subject =~ /(?<pre>pre)(?|(?<a>a)(?<b>b)(?<c>c)|(?<d>d)(?<e>e)|(?<f>f))(?<post>post)/;
+    is("$2-" . (defined $3 ? $3 : '') . '-' . (defined $4 ? $4 : ''),
+        $expected, 'branch-reset physical slots preserve post-group numbering');
+}
+
+for my $letter (qw(a b c)) {
+    my $subject = $letter x 2;
+    ok($subject =~ /((?|(?<a>a)(?-1)|(?<b>b)(?-1)|(?<c>c)(?-1)))/,
+        'relative subroutine target resolves inside each branch-reset alternative');
+    is($1, $subject, 'relative subroutine target consumes the matching pair');
+}
+
+done_testing;

src/test/resources/unit/regex/joni_extended_class_range_diagnostic.t

@@ -0,0 +1,9 @@
+use strict;
+use warnings;
+use Test::More;
+
+my $invalid_range = eval q{ qr/[\xdf-/i; 1 } ? '' : $@;
+like($invalid_range, qr/^Invalid \[\] range/,
+     'unterminated character-class range uses the Perl diagnostic');
+
+done_testing;

src/test/resources/unit/regex/joni_false_class_ranges.t

@@ -0,0 +1,20 @@
+use strict;
+use warnings;
+use Test::More;
+
+my @cases = (
+    [ qr/([a-\d]+)/,                'za-9z', 'a-9', 'backslash d after a-dash' ],
+    [ qr/([\d-z]+)/,                'a0-za', '0-z', 'backslash d before dash-z' ],
+    [ qr/([\d-\s]+)/,              'a0- z', '0- ', 'backslash d through dash-space' ],
+    [ qr/([a-[:digit:]]+)/,          'za-9z', 'a-9', 'POSIX digit after a-dash' ],
+    [ qr/([[:digit:]-z]+)/,          '=0-z=', '0-z', 'POSIX digit before dash-z' ],
+    [ qr/([[:digit:]-[:alpha:]]+)/, '=0-z=', '0-z', 'POSIX digit and alpha around dash' ],
+);
+
+for my $case (@cases) {
+    my ($regex, $input, $want, $name) = @$case;
+    my ($got) = $input =~ $regex;
+    is($got, $want, $name);
+}
+
+done_testing;

src/test/resources/unit/regex/native_numeric_backreference_boundaries.t

@@ -0,0 +1,32 @@
+use strict;
+use warnings;
+use Test::More;
+
+for my $pattern (q{\87}, q{a\87}, q{a\97}) {
+    my $compiled = eval "qr/$pattern/";
+    like($@, qr/Reference to nonexistent group in regex/,
+         "$pattern is a nonexistent decimal backreference");
+}
+
+for my $digits (qw(2147483648 2147483649 2147483650
+                   4294967296 4294967297 4294967298)) {
+    for my $prefix ('', 'a') {
+        for my $form ("\\g$digits}", "\\g{$digits}", "\\g{ $digits }") {
+            my $compiled = eval "qr/${prefix}(.)$form/";
+            like($@, qr/Reference to nonexistent group in regex/,
+                 "$form rejects overflow safely after ${prefix}capture");
+        }
+    }
+
+    my ($octal, $tail) = $digits =~ /^([0-7]{1,3})(.*)$/;
+    for my $prefix ('', 'a') {
+        my $pattern = "${prefix}(.)\\$digits";
+        my $compiled = eval "qr/$pattern/";
+        ok(defined($compiled), "$pattern compiles as octal plus literal tail");
+        my $subject = $prefix . 'b' . chr(oct($octal)) . $tail;
+        ok($subject =~ $compiled && $1 eq 'b',
+           "$pattern matches its octal boundary without integer overflow");
+    }
+}
+
+done_testing;

third_party/joni/src/org/joni/Analyser.java

@@ -449,14 +449,14 @@ private int getMinMatchLength(Node node) {
             BackRefNode br = (BackRefNode)node;
             if (br.isRecursion()) break;
 
-            if (br.back[0] > env.numMem) {
+            if (invalidBackrefNode(br.back[0])) {
                 if (!syntax.op3OptionECMAScript()) newValueException(INVALID_BACKREF);
             } else {
                 min = getMinMatchLength(env.memNodes[br.back[0]]);
             }
 
             for (int i=1; i<br.backNum; i++) {
-                if (br.back[i] > env.numMem) {
+                if (invalidBackrefNode(br.back[i])) {
                     if (!syntax.op3OptionECMAScript()) newValueException(INVALID_BACKREF);
                 } else {
                     int tmin = getMinMatchLength(env.memNodes[br.back[i]]);
@@ -596,7 +596,7 @@ private int getMaxMatchLength(Node node) {
             }
 
             for (int i=0; i<br.backNum; i++) {
-                if (br.back[i] > env.numMem) {
+                if (invalidBackrefNode(br.back[i])) {
                     if(!syntax.op3OptionECMAScript()) newValueException(INVALID_BACKREF);
                 } else {
                     int tmax = getMaxMatchLength(env.memNodes[br.back[i]]);
@@ -1414,6 +1414,10 @@ protected final void setupSubExpCall(Node node) {
                             newValueException(MULTIPLEX_DEFINITION_NAME_CALL, cn.nameP, cn.nameEnd);
                         } else {
                             cn.groupNum = ne.backRef1; // ne.backNum == 1 ? ne.backRef1 : ne.backRefs[0]; // ??? need to check ?
+                            if (ne.backNum == 1) {
+                                cn.lexicalTarget = env.physicalNamedMemNode(
+                                        ne.getPhysicalBackRefs()[0]);
+                            }
                             if (ne.backNum > 1) cn.setRecursion();
                             setCallAttr(cn);
                         }
@@ -2500,7 +2504,7 @@ protected final Node setupTree(Node node, int state) {
         case NodeType.BREF:
             BackRefNode br = (BackRefNode)node;
             for (int i=0; i<br.backNum; i++) {
-                if (br.back[i] > env.numMem) {
+                if (invalidBackrefNode(br.back[i])) {
                     if (!syntax.op3OptionECMAScript()) newValueException(INVALID_BACKREF);
                 } else {
                     env.backrefedMem = bsOnAt(env.backrefedMem, br.back[i]);
@@ -3066,4 +3070,9 @@ protected final void setOptimizedInfoFromTree(Node node) {
             Config.log.println(regex.optimizeInfoToString());
         }
     }
+
+    private boolean invalidBackrefNode(int number) {
+        return number <= 0 || number > env.numMem || env.memNodes == null
+                || number >= env.memNodes.length || env.memNodes[number] == null;
+    }
 }

third_party/joni/src/org/joni/ArrayCompiler.java

@@ -1123,6 +1123,7 @@ protected void compileEncloseNode(EncloseNode node) {
                             : OPCode.CONDITION);
                     addMemNum(node.calloutConditionId >= 0 ? node.calloutConditionId
                             : node.recursionConditionGroup >= 0 ? node.recursionConditionGroup
+                            : node.physicalNamedCondition > 0 ? -node.physicalNamedCondition
                             : node.regNum);
                     addRelAddr(len + OPSize.JUMP);
                 }

third_party/joni/src/org/joni/ByteCodeMachine.java

@@ -946,6 +946,16 @@ private void opExactNICSb() {
     private void opCondition() {
         int mem = code[ip++];
         int addr = code[ip++];
+        if (mem < 0) {
+            int physical = -mem;
+            if (physicalNamedCaptureBeg == null
+                    || physical >= physicalNamedCaptureBeg.length
+                    || physicalNamedCaptureBeg[physical] == INVALID_INDEX
+                    || physicalNamedCaptureEnd[physical] == INVALID_INDEX) {
+                ip += addr;
+            }
+            return;
+        }
         if (mem > regex.numMem || repeatStk[memEndStk + mem] == INVALID_INDEX || repeatStk[memStartStk + mem] == INVALID_INDEX) {
             ip += addr;
         }