[danking]

Reject raw serialization of sliced PCO arrays because the format does not encode slice offsets. This prevents writing data that cannot reconstruct the original slice; callers can materialize the slice before serialization.

[codspeed]

Merging this PR will degrade performance by 15.5%

4 warnings, 1 note

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ Unable to measure the execution of 2 benchmarks

These benchmarks measured no execution time, so their results are not comparable and count as unchanged. This usually happens when the compiler removes the code under test.

Prevent compiler optimizations

See affected benchmarks

⚠️ 21 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚠️ Base and head ran on different runner types

Some benchmarks with significant performance changes were compared across different runner types, so the changes might not come from the code. Run base and head on the same runner type, or use CodSpeed Macro Runners for more stable results.

Learn more about Macro Runners

Open the report in CodSpeed to investigate

ℹ️ 40 benchmarks have no flamegraph

The flamegraph wasn't generated: the symbols of the profiling trace could not be resolved, because no information about the language frames was collected.

See affected benchmarks

❌ 1 regressed benchmark
✅ 2438 untouched benchmarks
⏩ 12 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	CPU Simulation	test_repeated_scan_scalar_at[int_float]	370.4 µs	438.4 µs	-15.5%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing dk/pco-reject-sliced-serialization (44261d2) with develop (f476db4)}

Footnotes

1. 12 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[danking]

@claude review

[github-actions]

Claude finished @danking's task in 1m 9s —— View job

---

Review: Reject raw serialization of sliced PCO arrays

• Read the diff (encodings/pco/src/array.rs, encodings/pco/src/tests.rs)
• Checked serialization callers and the VTable::serialize contract
• Looked for other encodings with the same problem
• Checked whether the file write path could hit the new error

Verdict: The fix is correct and small. I found nothing blocking. The one follow-up that matters is that Zstd has the same bug and is still unfixed (point 1 below).

Correctness

The guard slice_start == 0 && slice_stop == unsliced_n_rows (array.rs:201-204) is the right check. Before this PR, a sliced PCO array serialized PcoMetadata, which has no slice bounds. It also wrote the unsliced validity child while the node's len was the sliced length. So on deserialization children.get(0, &Validity::DTYPE, len) would see a length mismatch, or the data would silently decode the wrong rows. Failing loudly is much better.

I checked whether this could break normal file writes. RepartitionStrategy slices chunks (vortex-layout/src/layouts/repartition.rs:150,237), but it fully decodes the result (execute::<Canonical>) before passing it on, and compression happens later. So the writer should never serialize a sliced PCO array. The paths that can now fail are IPC/serialize of a sliced, in-memory PCO array, for example after reading a file and slicing. Those paths were producing corrupt output before.

Findings

1. Zstd has the same problem (worth a follow-up). ZstdData also stores slice_start/slice_stop/unsliced_n_rows (encodings/zstd/src/array.rs:511-512). Its serialize (array.rs:220-225) still encodes metadata with no check, so sliced Zstd arrays hit the same corruption. The same guard plus a matching test would fix it. Fix this →

2. Nit: this goes against the documented serialize contract. vortex-array/src/array/vtable/mod.rs:123-126 says to return Ok(None) when an array cannot be serialized and not to return an error, leaving the caller to decide how to report it. Today the result is the same either way: ArraySerializationTree::try_new turns None into an error. Your error message is more useful than the generic "does not support serialization". Either update the doc comment to allow "this particular instance can't be serialized", or switch to Ok(None). Up to you.

3. Nit: the test could check the error message. sliced_pco_cannot_be_serialized only asserts is_err(). Matching on "cannot serialize sliced PCO" would stop the test passing because of some unrelated failure, such as a ctx/encoding-ID problem, without reaching this guard. AGENTS.md asks that invalid-input fixtures reach the guard they claim to test.

4. Possible later improvement. Adding slice_start/slice_stop to PcoMetadata (proto fields default to 0, which keeps the format backward compatible) would let sliced arrays round-trip without decompressing. This PR's approach is the right short-term fix.

CodSpeed regression

The flagged test_repeated_scan_scalar_at[int_float] (−15.5%) is a Python scan benchmark. It doesn't exercise PCO serialization, and the report says base and head ran on different runner types. It looks like noise and isn't caused by this change.

I didn't run any builds or tests; this was a read-only review. The PR is already merged, so the findings above are follow-up suggestions.