Skip to content

Fix: JSON import UTF-8 BOM handling issue #155

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

Shyam-Raghuwanshi
Copy link
Contributor

Fix JSON import UTF-8 BOM handling issue

  • Add strip_utf8_bom() function to remove UTF-8 BOM (EF BB BF) from file content
  • Apply BOM stripping to JsonLines and JsonArray import formats
  • Add comprehensive tests for UTF-8 BOM handling across all import formats
  • Ensure backward compatibility with files without BOM
  • Fix 'Not valid JSON: expected value at line 1 column 1' error for BOM files

fix #138


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@nipunn1313
Copy link
Collaborator

Hi.
We ended up fixing clientside with

b671cd8

We thought about doing something serverside, but ended up deciding that BOM was not supported by the server - and we preferred solutions that errors if a BOM is sent up.

Currently, it still says
'Not valid JSON: expected value at line 1 column 1'

If you could modify the PR to give a more specific error in this case, we'd take that instead, but we'd like to keep BOM as unsupported.

@nipunn1313
Copy link
Collaborator

Sorry - I didn't follow up on #138 promptly

Copy link
Collaborator

@nipunn1313 nipunn1313 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall looks good - thanks
One comment.

Copy link
Collaborator

@nipunn1313 nipunn1313 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool looks good. I'll merge it in later.

@nipunn1313
Copy link
Collaborator

merged in 42a8752

@Shyam-Raghuwanshi - I made some small changes to your PR to simplify. Take a look - trying to really make the diff more minimal from the previous behavior.

@nipunn1313 nipunn1313 closed this Jul 28, 2025
@Shyam-Raghuwanshi Shyam-Raghuwanshi deleted the fix/json-import-utf8-bom-issue branch July 29, 2025 02:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] Import | JSON import fails - Not valid JSON: expected value at line 1 column 1 - due to UTF8-BOM character
2 participants