[robert3005]

ArrayOptimizer::optimize uses only the static rules of the encodings, so it misses the kernels that a session registers. This PR deprecates it, and array constructors no longer optimize the arrays they build.

Changes

• optimize_ctx and optimize_recursive take &impl AsOptimizerKernels, which is a VortexSession or an ExecutionCtx created from one. Existing optimize_ctx(&session) callers compile without changes.
• Constructors such as slice, filter, take, cast, fill_null, get_item and apply return lazy arrays. Execution optimizes each node it reaches (Optimize arrays top down as execution reaches them, and retry a parent when its children change #11090). Callers that read the encoding of an array directly now optimize it with their session first: scan tasks, plan v2 operators, the dict layout writer, the FFI scan and CUDA dispatch.
• The default append_to_builder optimizes the array first. It then appends through the encoding the array reduces to, so a lazy slice of a constant appends as a constant.
• Tests now call optimize_ctx(&SESSION) instead of optimize().

API Changes

• ArrayOptimizer::optimize is deprecated. Use optimize_ctx with the session or an ExecutionCtx.
• ArrayOptimizer::optimize_ctx and optimize_recursive are generic over AsOptimizerKernels.
• ArrayRef::slice, filter, take and the ArrayBuiltins methods no longer return an optimized array.

[codspeed]

Merging this PR will degrade performance by 10.05%

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 1 benchmark is dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚡ 11 improved benchmarks
❌ 33 regressed benchmarks
✅ 1726 untouched benchmarks
⏩ 47 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	CPU Simulation	filter_random_by_random[10000]	11.2 µs	38.7 µs	-71.14%
❌	Walltime	take_neon[small_m/shuffled/primitive/nonnull/chunks=256/indices=64]	41.7 µs	60.8 µs	-31.43%
❌	Walltime	take_neon[small_m/shuffled/primitive/nonnull/chunks=2048/indices=64]	51.9 µs	75.2 µs	-30.93%
❌	CPU Simulation	slice_dict_tight_loop[10000]	521 µs	743.9 µs	-29.96%
❌	Walltime	take_neon[small_m/shuffled/primitive/nonnull/chunks=256/indices=16]	14.7 µs	20.2 µs	-27.23%
❌	Walltime	take_neon[value_shape/shuffled/primitive/nonnull/chunks=16/indices=100]	15.8 µs	21.3 µs	-25.71%
❌	Walltime	take_neon[small_m/shuffled/primitive/nonnull/chunks=2048/indices=16]	17.8 µs	23.8 µs	-25.03%
❌	CPU Simulation	search_index_below_min_chunked	210 µs	274.8 µs	-23.57%
❌	Walltime	take_neon[core/shuffled/primitive/nonnull/chunks=16/indices=250]	18.6 µs	24.1 µs	-23.17%
❌	CPU Simulation	slice_primitive_tight_loop[10000]	345.3 µs	445.8 µs	-22.55%
❌	CPU Simulation	search_index_mixed_out_of_range_chunked	234.5 µs	300.2 µs	-21.87%
❌	CPU Simulation	search_index_full_range_random_chunked	240 µs	305.8 µs	-21.54%
❌	Walltime	take_neon[routing/grouped_chunks/primitive/nonnull/chunks=32/indices=1000]	41.7 µs	53.1 µs	-21.36%
❌	Walltime	take_neon[nullable/sorted/primitive/nullable_all_valid/chunks=16/indices=1000]	24.2 µs	29.5 µs	-18.01%
❌	CPU Simulation	search_index_in_range_chunked	336.1 µs	407.2 µs	-17.45%
❌	CPU Simulation	search_index_above_max_chunked	334.8 µs	405 µs	-17.33%
❌	CPU Simulation	missing_stats[16]	75.8 µs	91.4 µs	-17.06%
❌	Walltime	take_neon[core/shuffled/primitive/nonnull/chunks=16/indices=1000]	30 µs	36 µs	-16.82%
❌	Walltime	take_neon[duplicates/duplicate90/primitive/nonnull/chunks=16/indices=1000]	29.1 µs	34.7 µs	-16.31%
❌	CPU Simulation	bench_compare_varbin[(50000, 2)]	144.3 µs	170 µs	-15.12%
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing rk/deprecate-sessionless-optimize (8e61ec4) with develop (068d819)}

Footnotes

1. 47 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[robert3005]

#11092 recovers majority of the regression, could merge them into one but this is a smaller change