[connortsui20]

Summary

• Tracking Issue: Tracking Issue: Fix Canonical Decimal #10701
• CD4 (Support decimal values in Widen #10946) builds on this PR.

Interleave accepts decimal values but panics when it executes them, because only the bool and primitive kernels exist. Decimal Widen (CD4) falls back to canonical decimals for interleave, so it needs a decimal kernel first.

Changes

Most of the diff is a refactor of the primitive kernel: its gather moves into a fixed_width module next to the kernels, so decimals reuse it instead of getting a second copy. Canonical decimals are fixed-width buffers just like primitives, so one set of selector bounds checks and one tuned gather loop can serve both. The gather stays in interleave and not in arrays/fixed_width because it owns the selector contract of Interleave and has no other caller.

Decimal sources stored at a non-canonical width are copied to the canonical width before the gather. That copy goes away once CD8a stores every decimal at its canonical width.

API Changes

Interleave executes decimal values and returns them at the canonical width of their dtype, where it used to panic.

[codspeed]

Merging this PR will regress 5 benchmarks

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 3 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚡ 3 improved benchmarks
❌ 5 regressed benchmarks
✅ 2266 untouched benchmarks
⏩ 370 skipped benchmarks¹
🗄️ 30 archived benchmarks run²

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	CPU Simulation	random_i8[0.01]	99 µs	123.3 µs	-19.72%
❌	CPU Simulation	encode_primitives[u8, (2000, 4)]	136.2 µs	164.7 µs	-17.29%
❌	CPU Simulation	filter_powerlaw_by_correlated_runs[100000]	51.2 µs	61.9 µs	-17.2%
❌	CPU Simulation	take_filter_list_small_uncached_random_mask_random_indices[768, 10]	105.4 µs	119.4 µs	-11.72%
❌	CPU Simulation	max_f64	282 µs	315.2 µs	-10.52%
⚡	Walltime	mul_u64_nonnull_neon	38.9 µs	20.3 µs	+92.09%
⚡	Walltime	lt_i64_nullable_avx512	8.1 µs	7.3 µs	+10.91%
⚡	Walltime	deferred_bool_avx2[16384, (Columns, NullOnlyFailure)]	302 µs	273.6 µs	+10.38%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing ct/decimal-interleave (4e054c4) with develop (40e0b1c)}

Footnotes

1. 370 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. 30 benchmarks were run, but are now archived. If they were deleted in another branch, consider rebasing to remove them from the report. Instead if they were added back, click here to restore them. ↩

[connortsui20]

Im pretty sure we want to panic here instead of erroring, that would be consistent with how dict take works.

I'll have a followup PR on this later (and I think that runend and chunked take should also panic on OOB instead of erroring)

[connortsui20]

ok nvm this is actually the correct behavior, we want dict take to error instead

[connortsui20]

this is mostly a refactor of the current primitive.rs in this module