Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Engine C/C++ external APIs should convert to UTF-8 #21

Open
isc-woodfin opened this issue May 12, 2020 · 1 comment
Open

Engine C/C++ external APIs should convert to UTF-8 #21

isc-woodfin opened this issue May 12, 2020 · 1 comment
Assignees

Comments

@isc-woodfin
Copy link

ctypedef unsigned short Entity_Ref

This line suggests the engine API is exposing our internal UTF-16 string representation. Though cognizant of the performance hit, I think we need an API where all string content is ordinary char*'s of UTF-8 (maybe not the only one, but certainly the one that code like @adang1345's would use)

@JosDenysGitHub
Copy link
Collaborator

We refer to literals via offsets in the text (and not making a copy as we used to do), that text is supposed to be of a fixed character size (UCS-2), if we convert to (multibye) utf8, we should also convert to byte offset.
Python also used UCS-2 as it's internal text representation, for all string - offset combinations to work in more exotic languages. But for emitting tracing strings, that surely can be utf8 encoded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants