Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test inferred schema for json array #454

Merged
merged 1 commit into from
Jul 17, 2023

Commits on Jul 17, 2023

  1. Test inferred schema for json array

    Arrow infer schema internally uses IndexMap to keep track of inferred schema
    while updating it through given json objects. Thus the order in which
    insertion happens to this map determines the output order for schema fields
    as well.
    
    Since the json object in serde_json uses BtreeMap, the iteration is always
    ascending by key. This leads to infer_schema producing schema fields in
    ascending order when we infer using only one json object.
    In case of json array any new fields encountered are added to the last.
    Thus output order of infer schema did not provide good enough guarantee.
    This is solved in parseablehq#450. This commit just adds a test for it.
    trueleo committed Jul 17, 2023
    Configuration menu
    Copy the full SHA
    e927059 View commit details
    Browse the repository at this point in the history