Skip to content

Added five addtional bidi test cases and fixed BidiReference.java #1544

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
roozbehp wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Added five addtional bidi test cases and fixed BidiReference.java #1544

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
483a0f7
Added five addtional bidi test cases and fixed BidiReference.java
roozbehp Sep 18, 2026
6f211c4
Update version and timestamp
roozbehp Oct 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 20 additions & 2 deletions unicodetools/data/ucd/dev/BidiCharacterTest.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# BidiCharacterTest-18.0.0.txt
# Date: 2026-02-03
# BidiCharacterTest-19.0.0.txt
# Date: 2026-10-10, 22:00:00 GMT
# © 2026 Unicode®, Inc.
# Unicode and the Unicode Logo are registered trademarks of Unicode, Inc. in the U.S. and other countries.
# For terms of use and license, see https://www.unicode.org/terms_of_use.html
Expand Down Expand Up @@ -121,6 +121,24 @@
0061 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0062 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029;1;1;2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2;0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127
0061 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0062 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029 0029;1;1;2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1;129 128 127 126 125 124 123 122 121 120 119 118 117 116 115 114 113 112 111 110 109 108 107 106 105 104 103 102 101 100 99 98 97 96 95 94 93 92 91 90 89 88 87 86 85 84 83 82 81 80 79 78 77 76 75 74 73 72 71 70 69 68 67 66 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

################################################################################
# Additional tests for isolating run sequences, depth limits, and overrides. [added to test cases for Unicode 19.0]

# Preserve PDI identity when joining isolating run sequences after an override (BD9, BD13, X10).
202E 2067 05D0 2069 202C 202B 0061 2067 05D1 2069 0062;0;0;x 1 3 1 x x 2 2 3 2 2;6 7 8 9 10 3 2 1

# An overflow isolate must not split a level run (BD7, X5b, X10).
202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 05D0 2066 05D1;0;0;x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x 125 125 125;64 63 62

# Do not reverse a level-126 run when no odd level is present (L2).
202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 202A 0661 0662;0;0;x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x 126 126;62 63

# Apply the active directional override to an unmatched PDI (X6a).
202A 0061 202C 202E 2069 202A 0062;0;0;x 2 x x 1 x 2;6 4 1

# Bracket-stack overflow discards pairs found earlier in the isolating run sequence (BD16).
0061 0028 0062 0029 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028 0028;1;1;2 2 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1;67 66 65 64 63 62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 0 1 2

################################################################################
# Miscellaneous test cases

Expand Down
54 changes: 51 additions & 3 deletions unicodetools/data/ucd/dev/BidiTest.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# BidiTest-18.0.0.txt
# Date: 2026-02-03, 23:09:43 GMT
# BidiTest-19.0.0.txt
# Date: 2026-10-10, 22:00:00 GMT
# © 2026 Unicode®, Inc.
# Unicode and the Unicode Logo are registered trademarks of Unicode, Inc. in the U.S. and other countries.
# For terms of use and license, see https://www.unicode.org/terms_of_use.html
Expand Down Expand Up @@ -496724,6 +496724,18 @@ FSI ON AN AN PDI; 4

#Count: 1

@Levels: x 2 x x 1 x 2
@Reorder: 6 4 1
LRE L PDF RLO PDI LRE L; 3

#Count: 1

@Levels: x 2 x x 3 x 4
@Reorder: 1 6 4
LRE L PDF RLO PDI LRE L; 4

#Count: 1

@Levels: 0 1 2 1 2 0
@Reorder: 0 4 3 2 1 5
FSI ON EN R L PDI; 3
Expand Down Expand Up @@ -496826,6 +496838,18 @@ ON FSI R AL PDI RLI R AL PDI; 4

#Count: 1

@Levels: x 1 3 1 x x 2 2 3 2 2
@Reorder: 6 7 8 9 10 3 2 1
RLO RLI R PDI PDF RLE L RLI R PDI L; 3

#Count: 1

@Levels: x 3 5 3 x x 4 4 5 4 4
@Reorder: 6 7 8 9 10 3 2 1
RLO RLI R PDI PDF RLE L RLI R PDI L; 4

#Count: 1

@Levels: 0 0 0 0 0 0 0 0 0 0
@Reorder: 0 1 2 3 4 5 6 7 8 9
ET LRI PDI EN RLI PDI ET FSI PDI EN; 3
Expand Down Expand Up @@ -497482,6 +497506,12 @@ L L WS L L L L WS ON R R WS R R WS ON AN AN AN CS AN AN AN ON CS WS R R ON; 4

#Count: 2

@Levels: x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x 126 126
@Reorder: 62 63
LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE AN AN; 7

#Count: 1

@Levels: 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 2 2 2 1 1 2 2 2 1 1 1 1 1 0
@Reorder: 0 1 2 3 4 5 6 7 8 28 27 26 25 24 21 22 23 20 19 16 17 18 15 14 13 12 11 10 9 29
L L WS L L L L WS ON R R WS R R WS ON EN EN EN CS WS EN EN AN ON CS WS R R ON; 3
Expand Down Expand Up @@ -497538,6 +497568,12 @@ R AL WS R AL R AL WS ON L L WS L L WS L WS L L L ON L ON WS AL R AL WS R AL R; 5

#Count: 1

@Levels: x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x 125 125 125
@Reorder: 64 63 62
LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE R LRI R; 7

#Count: 1

@Levels: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2
@Reorder: 25 26 27 28 29 30 31 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
AL R R R R AL R WS AL R R WS R AL AL R WS R R R AL R CS WS AL EN EN EN CS EN EN AN; 7
Expand Down Expand Up @@ -497586,6 +497622,18 @@ LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE LRE

#Count: 1

#Total Count: 490846
@Levels: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
@Reorder: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67
L ON L ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON; 3

#Count: 1

@Levels: 2 2 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
@Reorder: 67 66 65 64 63 62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 0 1 2
L ON L ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON ON; 4

#Count: 1

#Total Count: 490854

# EOF
25 changes: 25 additions & 0 deletions unicodetools/src/main/java/org/unicode/bidi/BidiConformanceTestBuilder.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,31 @@ public class BidiConformanceTestBuilder {
private static final byte PDI = BidiReference.PDI;

private static byte[][] extraTests = {
// Preserve PDI identity when joining isolating run sequences after an override.
{RLO, RLI, R, PDI, PDF, RLE, L, RLI, R, PDI, L},
// An overflow isolate must not split a level run.
{
LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE,
LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE,
LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE,
LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, R, LRI, R
},
// Do not reverse a level-126 run when no odd level is present.
{
LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE,
LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE,
LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE,
LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, LRE, AN, AN
},
// An unmatched PDI must receive the active directional override.
{LRE, L, PDF, RLO, PDI, LRE, L},
// Class-only counterpart; bracket overflow is tested in BidiCharacterTest.txt.
{
L, ON, L, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON,
ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON,
ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON, ON,
ON, ON
},
// Test data provided by Behdad Esfahbod, Dov Grobgeld, Aharon Lanin, and Roozbeh Pournader
{AL, AL, R, WS, R, R, WS, L, L, L, WS, L, L, L, WS, R, R, WS, AL, R, R, R, R, R, R},
{
Expand Down
3 changes: 3 additions & 0 deletions unicodetools/src/main/java/org/unicode/bidi/BidiReference.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -485,6 +485,9 @@ private void determineExplicitEmbeddingLevels() {
--validIsolateCount;
}
resultLevels[i] = stack.lastEmbeddingLevel();
if (stack.lastDirectionalOverrideStatus() != ON) {
resultTypes[i] = stack.lastDirectionalOverrideStatus();
}
break;

// Rule X7
Expand Down
Loading