Skip to content

RFI: Rebalancing unicode escaping in str::escape_debug for internationalization #159694

Description

@the8472

Triggered by discussion in #158286 @rust-lang/libs-api concluded that our debug escapes are escaping too many characters and are not internationalization-friendly.
E.g. ຄົນສົ່ງໜັງສື currently gets escaped to ຄ\u{ebb}ນສ\u{ebb}\u{ec8}ງໜ\u{eb1}ງສ\u{eb7}

We don't have precise rules ourselves what should and shouldn't be escaped and we'd like to gather input and help from unicode/internationalization experts to find a better balance between various debugging goals and learn what the tradeoffs are.
Once we better understand the tradeoffs and have decided on an approach implementation help, testdata and similar would also be appreciated.

Some desiderata that were brought up during discussion:

  • living languages¹ should not be excessively mangled by escaping and remain readable.
    A few ligatures getting torn per sentence probably is ok, every other letter getting replaced by an escape wouldn't be
  • surprising/unexpected/unprintable characters - e.g. a ZWJ or BOM in the middle of ascii strings/latin script - should get escaped
  • debug printing shouldn't mess up terminals → control sequences and directionality overrides should remain escaped
  • binary size might also be a concern if additional unicode tables are required

¹ "Any text one could pluck from a random international wikipedia page that isn't about ancient typography"

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-UnicodeArea: UnicodeA-strArea: str and StringC-enhancementCategory: An issue proposing an enhancement or a PR with one.E-help-wantedCall for participation: Help is requested to fix this issue.T-libsRelevant to the library team, which will review and decide on the PR/issue.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions