Skip to content

fix: Report non-ASCII characters in unicode escapes as parse errors - #801

Merged
eed3si9n merged 1 commit into
typelevel:mainfrom
RaphaelFakhri:fix-non-ascii-unicode-escape
Nov 4, 2026
Merged

eed3si9n merged 1 commit into
typelevel:mainfrom
RaphaelFakhri:fix-non-ascii-unicode-escape

Conversation

@RaphaelFakhri

Copy link
Copy Markdown
Contributor

Parser.descape indexes the 128-entry HexChars table with the raw escape character. A non-ASCII character in a \u escape, such as "\u00é0", throws ArrayIndexOutOfBoundsException. Parser.parse treats that as end of input and throws IncompleteParseException("exhausted input"). As a result:

  • Sync parsers fail with "exhausted input" and no position, instead of ParseException: expected valid unicode escape.
  • AsyncParser.absorb and finalAbsorb throw instead of returning Left.

This change treats any character outside the table as an invalid hex digit, so these inputs fail the same way "\uqqqq" already does.

Tests

SyntaxCheck adds two properties for non-ASCII escape characters (Latin-1, two-byte and three-byte UTF-8). One checks the String and ByteBuffer parsers and the other checks AsyncParser. Both fail on main and pass with the fix. parserJVM/test and astJVM/test pass on Scala 2.12, 2.13 and 3.

Fixes #800

@eed3si9n eed3si9n left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@eed3si9n
eed3si9n merged commit 4f643b7 into typelevel:main Nov 4, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Non-ASCII character in a \u escape is reported as "exhausted input"

2 participants