Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/reference/feature-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -379,7 +379,7 @@ my @copy = @{$z}; # ERROR
- ✅ **Backreferences to Named Groups**: Using `\k<name>` or `\g{name}` for backreferences to named groups is supported.
- ✅ **Relative Backreferences**: Using `\g{-n}` for relative backreferences.
- ✅ **Basic Unicode Properties**: Common `\p{...}` and `\P{...}` forms such as `\p{L}` execute through Joni. General_Category assignments now enter the forked Joni parser unchanged and resolve to pinned Perl ranges there.
- 🟡 **Perl Unicode Property Syntax**: Perl-specific properties execute through Joni. `Age`, cumulative `In`/`Present_In`, General_Category, Canonical_Combining_Class, Bidi_Class, Decomposition_Type, East_Asian_Width, Numeric_Value, Joining_Group, Block, Script, and Script_Extensions assignments are generated from pinned Perl 5.44 Unicode 17.0 data with loose and wildcard aliases, ordered missing defaults, official compact aliases and shortcuts, Script-versus-Script_Extensions policy, reserved or composite values, exact rationals, and Perl's generated decimal keyword aliases; `ASCII_Hex_Digit`/`AHex` accepts Perl's eight boolean value aliases. General_Category and standalone Block, Script, and Script_Extensions assignments use Joni's range-resolver API with explicit per-family case-fold policy. No-fold properties inside composed character classes and wildcard values remain adapter-translated until Joni represents those syntax semantics natively. Other generated property/value aliases still have pinned acceptance/rejection gaps.
- 🟡 **Perl Unicode Property Syntax**: Perl-specific properties execute through Joni. `Age`, cumulative `In`/`Present_In`, General_Category, Canonical_Combining_Class, Bidi_Class, Decomposition_Type, East_Asian_Width, Numeric_Value, Joining_Group, Block, Script, and Script_Extensions assignments are generated from pinned Perl 5.44 Unicode 17.0 data with loose and wildcard aliases, ordered missing defaults, official compact aliases and shortcuts, Script-versus-Script_Extensions policy, reserved or composite values, exact rationals, and Perl's generated decimal keyword aliases; `ASCII_Hex_Digit`/`AHex` accepts Perl's eight boolean value aliases. General_Category and standalone exact Block, Script, Script_Extensions, Canonical_Combining_Class, Bidi_Class, Decomposition_Type, East_Asian_Width, Numeric_Value, and Joining_Group assignments use Joni's range-resolver API with explicit per-family case-fold policy. No-fold properties inside composed character classes, wildcard values, and Age-family assignments remain adapter-translated until Joni represents those syntax semantics natively. Other generated property/value aliases still have pinned acceptance/rejection gaps.
- ✅ **Possessive Quantifiers**: Quantifiers like `*+`, `++`, `?+`, and `{n,m}+`, which disable backtracking, are supported.
- ✅ **Atomic Grouping**: Use of `(?>...)` for atomic groups is supported.
- ✅ **`\K` assertion**: Keep left — in `s///`, text before `\K` is preserved; match variables reflect only the portion after `\K`. Ordinary KEEP assertions route through native Joni and no longer use the Java marker rewrite; the adapter still rejects KEEP inside lookaround until the Joni analyser emits Perl's diagnostic directly.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -999,6 +999,15 @@ static CharacterPropertyResolver.Result resolveJoniProperty(
|| PerlUnicodeScriptData.isScriptExtensionsPropertyAlias(name)) {
if (perlNumericWildcardBody(value) != null) return null;
caseFold = false;
} else if (isCanonicalCombiningClassProperty(name)
|| PerlUnicodeBidiClassData.isPropertyAlias(name)
|| PerlUnicodeDecompositionTypeData.isPropertyAlias(name)
|| PerlUnicodeEastAsianWidthData.isPropertyAlias(name)) {
caseFold = false;
} else if (PerlUnicodeNumericValueData.isPropertyAlias(name)
|| PerlUnicodeJoiningGroupData.isPropertyAlias(name)) {
if (perlNumericWildcardBody(value) != null) return null;
caseFold = false;
} else {
return null;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,24 @@ void preservesStandaloneBlockAndScriptFoldPolicyInsideJoni() {
assertFalse(script.matcher("K", java.util.List.of()).find());
}

@Test
void passesRemainingExactEnumeratedPropertiesToJoni() {
String[][] cases = {
{"\\p{Canonical_Combining_Class=Above}", "\u0301"},
{"\\p{Bidi_Class=Right_To_Left}", "\u05D0"},
{"\\p{Decomposition_Type=Canonical}", "\u00C0"},
{"\\p{East_Asian_Width=Fullwidth}", "\u3000"},
{"\\p{Numeric_Value=1/2}", "\u00BD"},
{"\\p{Joining_Group=Alef}", "\u0627"},
};

for (String[] testCase : cases) {
JoniRegexPattern pattern = new JoniRegexPattern(testCase[0], FLAGS);
assertEquals(testCase[0], pattern.patternDescription());
assertTrue(pattern.matcher(testCase[1], java.util.List.of()).find());
}
}

@Test
void flattensTranslatedPropertiesInsideOrdinaryCharacterClasses() {
JoniRegexPattern pattern = new JoniRegexPattern(
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
use strict;
use warnings;
use utf8;
use Test::More tests => 12;

ok("\x{0301}" =~ /\p{Canonical_Combining_Class=Above}/,
'canonical combining class assignment');
ok("\x{0300}" !~ /\p{Canonical_Combining_Class=Below}/,
'canonical combining class exclusion');
ok("\x{05D0}" =~ /\p{Bidi_Class=Right_To_Left}/,
'bidi class assignment');
ok('A' !~ /\p{Bidi_Class=Right_To_Left}/,
'bidi class exclusion');
ok("\x{00C0}" =~ /\p{Decomposition_Type=Canonical}/,
'decomposition type assignment');
ok('k' !~ /\p{Decomposition_Type=Canonical}/i,
'decomposition type does not gain fold members');
ok("\x{3000}" =~ /\p{East_Asian_Width=Fullwidth}/,
'east Asian width assignment');
ok('k' !~ /\p{East_Asian_Width=Ambiguous}/i,
'east Asian width does not gain fold members');
ok("\x{00BD}" =~ m{\p{Numeric_Value=1/2}},
'numeric value rational assignment');
ok('A' !~ m{\p{Numeric_Value=1/2}}i,
'numeric value does not gain fold members');
ok("\x{0627}" =~ /\p{Joining_Group=Alef}/,
'joining group assignment');
ok('A' !~ /\p{Joining_Group=Alef}/i,
'joining group does not gain fold members');