Keep same-scale decimal widening casts lazy

A same-scale cast that grows the canonical width copied every value
into the wider type, even when the caller never reads them all. Return
a decimal Widen over the source instead, whose dtype is a true label
for its values, so the copy waits for execution. The child also keeps
the statistics of the source, such as exact extrema, because it holds
the same values. A cast of a decimal Widen passes to its child.

This is the first production code that builds a decimal Widen. Widen
execution already copies decimal values instead of yielding a cast,
which now also keeps it from yielding a cast that returns a Widen
again. A test checks that the default file writer round-trips a
decimal Widen.

Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>

Author: connortsui20

vortex-array/benches/widen_decimal.rs

@@ -205,6 +205,9 @@ impl Case {
         10i64.pow(u32::from(self.child_precision))
     }
 
+    /// Returns a same-scale target with a wider canonical width than the parent, so the cast of a
+    /// `Widen` stays lazy (see `grows_canonical_width`). A target at the same canonical width would
+    /// measure a different cast path.
     fn cast_target(self) -> DType {
         let precision = if self.precision == 38 { 76 } else { 38 };
         DType::Decimal(

vortex-array/src/arrays/decimal/compute/cast.rs

@@ -21,6 +21,7 @@ use crate::arrays::ConstantArray;
 use crate::arrays::Decimal;
 use crate::arrays::DecimalArray;
 use crate::arrays::PrimitiveArray;
+use crate::arrays::WidenArray;
 use crate::arrays::decimal::DecimalArrayExt;
 use crate::dtype::BigCast;
 use crate::dtype::DType;
@@ -44,8 +45,6 @@ use crate::validity::Validity;
 
 impl CastReduce for Decimal {
     fn cast(array: ArrayView<'_, Decimal>, dtype: &DType) -> VortexResult<Option<ArrayRef>> {
-        // Only nullability changes within the same decimal dtype are reducible without execution.
-        // Precision/scale changes need the kernel.
         let DType::Decimal(to_decimal_dtype, to_nullability) = dtype else {
             return Ok(None);
         };
@@ -56,7 +55,10 @@ impl CastReduce for Decimal {
             );
         };
 
-        if from_decimal_dtype != to_decimal_dtype {
+        // Without execution, only a nullability change and a same-scale cast that grows the
+        // canonical width reduce. Every other change of the decimal dtype needs the kernel.
+        let grows_width = grows_canonical_width(*from_decimal_dtype, *to_decimal_dtype);
+        if from_decimal_dtype != to_decimal_dtype && !grows_width {
             return Ok(None);
         }
 
@@ -67,6 +69,10 @@ impl CastReduce for Decimal {
             return Ok(None);
         };
 
+        if grows_width {
+            return widen_decimal_values(array, dtype, new_validity).map(Some);
+        }
+
         // SAFETY: validity has the same length, only its nullability tag changes.
         unsafe {
             Ok(Some(
@@ -121,6 +127,10 @@ impl CastKernel for Decimal {
             .clone()
             .cast_nullability(*to_nullability, array.len(), ctx)?;
 
+        if grows_canonical_width(*from_decimal_dtype, *to_decimal_dtype) {
+            return widen_decimal_values(array, dtype, new_validity).map(Some);
+        }
+
         // Reuse the values buffer untouched when no rescale is required, the target precision
         // only widens (so every value still fits), and the current physical type is already wide
         // enough to hold the target precision. This keeps the common precision-widening cast
@@ -169,6 +179,38 @@ impl CastKernel for Decimal {
     }
 }
 
+/// Returns whether a cast keeps the scale and grows the canonical width, which implies a higher
+/// precision.
+fn grows_canonical_width(from: DecimalDType, to: DecimalDType) -> bool {
+    from.scale() == to.scale() && from.canonical_values_type() < to.canonical_values_type()
+}
+
+/// Returns `array` as a decimal `Widen` of `dtype`, over a child with `validity`, which the caller
+/// cast to the nullability of `dtype`.
+///
+/// The child keeps the dtype of `array` as its label, which holds every valid value because reads
+/// trust the precision. It also keeps the statistics of `array`, such as exact extrema, because it
+/// holds the same values. Only the nullability can change, and only when no value is null.
+fn widen_decimal_values(
+    array: ArrayView<'_, Decimal>,
+    dtype: &DType,
+    validity: Validity,
+) -> VortexResult<ArrayRef> {
+    // SAFETY: values and decimal dtype are unchanged, and the caller cast validity without
+    // changing its length.
+    let values = unsafe {
+        DecimalArray::new_unchecked_handle(
+            array.buffer_handle().clone(),
+            array.values_type(),
+            array.decimal_dtype(),
+            validity,
+        )
+    };
+    values.statistics().inherit_from(array.statistics());
+
+    Ok(WidenArray::try_new(values.into_array(), dtype.clone())?.into_array())
+}
+
 fn cast_to_integer<T: IntegerPType + BigCast>(
     array: ArrayView<'_, Decimal>,
     nullability: Nullability,

vortex-array/src/arrays/widen/compute/cast.rs

@@ -16,6 +16,17 @@ use crate::scalar_fn::fns::cast::CastReduce;
 impl CastReduce for Widen {
     fn cast(array: ArrayView<'_, Self>, dtype: &DType) -> VortexResult<Option<ArrayRef>> {
         let values = array.values();
+        if matches!(values.dtype(), DType::Decimal(..)) {
+            // The child holds exactly the values of the `Widen`, so its cast to the same target
+            // gives the same result. The decimal cast checks or rescales every value itself, and
+            // a same-scale cast that grows the canonical width returns a `Widen` again. Other
+            // targets decline, so execution widens the values before the cast.
+            return match dtype {
+                DType::Decimal(..) | DType::Primitive(..) => values.cast(dtype.clone()).map(Some),
+                _ => Ok(None),
+            };
+        }
+
         let (DType::Primitive(target_ptype, nullability), DType::Primitive(values_ptype, _)) =
             (dtype, values.dtype())
         else {

vortex-array/src/arrays/widen/mod.rs

@@ -14,8 +14,9 @@
 //! values cannot check: the child precision **must** hold every valid value. Compare, between and
 //! fill null decide from the child dtype alone, so a false label gives wrong results.
 //! [`WidenArray::encode_decimal`] and [`WidenArray::from_stored_decimal`] pick the smallest
-//! precision that holds the valid values and the scale. Decimal execution copies the child values
-//! into the logical canonical width.
+//! precision that holds the valid values and the scale, and a widening decimal cast labels the
+//! child with its source dtype. Decimal execution copies the child values into the logical
+//! canonical width, without casting, because a widening decimal cast can itself return a `Widen`.
 //!
 //! Narrowing is a physical choice, so it **must not** change a result. Execution casts the values
 //! to the logical dtype for integers, and an operation runs on the values only where it gives the