Add decimal support to VARIANT casting - #23858
Draft
vuule wants to merge 6 commits into
Draft
Conversation
cast_variant and extract_variant_field now accept DECIMAL32/64/128 targets. The VARIANT encoding scales every value individually while a cuDF column carries a single scale, so each value is rescaled to the requested scale, truncating toward zero, and a value that no longer fits the target representation is nulled with the OVERFLOW status.
…hmark decimals Adds a DECIMAL16 test at the int128 limits, which the previous cases left the high half of the payload zeroed for, and a sliced 512-row case so the decimal kernel's grid-stride loop and slice offset are covered. Factors the incoming-status and null-bit preamble the cast paths share into should_decode_row, so the protocol lives in one place instead of three, and extends the variant nvbench with decimal32 and decimal128 cases.
vuule
added
feature request
Adds a DECIMAL64 arm to the overflow test, the only place the int64_t range check is reachable, and a decimal64 case to the cast benchmark's type axis.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
closes #23817
cast_variantandextract_variant_fieldnow acceptDECIMAL32/DECIMAL64/DECIMAL128target types, decoding the DECIMAL4/8/16 VARIANT primitives.The encoding stores a scale per value while a cuDF column carries a single scale, so each value is rescaled to
desired_type.scale(). A value that does not fit the target after rescaling is nulled and reported asvariant_operation_status::OVERFLOW. An out-of-range scale byte or a truncated payload reportsMALFORMED_VARIANT, and a non-decimal encoding reportsTYPE_MISMATCH.Also added test and benchmark coverage.
Checklist