Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adjusts the way that dictionary values are handled. Before this PR, they were broken in the following ways:
FlightDataEncoder
by default willHydrate
dictionary values, hoisting their values rather than encoding dictionaries. This wasn't handled correctly in thenetwork_
execution plans, that were expecting the schema to not change, and so adding some schema adapters resolved thisFlightDataEncoder
, this means that the dictionary ids can be all messed up.Rather than pfaff about with all that side of things, I've adjusted it to be a "peek" style stream, whereby we wait until the last flight data, and add the metrics to that