Skip to content

Deprecate the session-less optimize and keep constructors lazy - #11091

Draft
robert3005 wants to merge 1 commit into
rk/optimize-execute-loopfrom
rk/deprecate-sessionless-optimize
Draft

robert3005 wants to merge 1 commit into
rk/optimize-execute-loopfrom
rk/deprecate-sessionless-optimize

Conversation

@robert3005

@robert3005 robert3005 commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

ArrayOptimizer::optimize uses only the static rules of the encodings, so it misses the kernels that a session registers. This PR deprecates it, and array constructors no longer optimize the arrays they build.

Changes

  • optimize_ctx and optimize_recursive take &impl AsOptimizerKernels, which is a VortexSession or an ExecutionCtx created from one. Existing optimize_ctx(&session) callers compile without changes.
  • Constructors such as slice, filter, take, cast, fill_null, get_item and apply return lazy arrays. Execution optimizes each node it reaches (Optimize arrays bottom up and as execution reaches them #11090). Callers that read the encoding of an array directly now optimize it with their session first: scan tasks, plan v2 operators, the dict layout writer, the FFI scan and CUDA dispatch.
  • The default append_to_builder optimizes the array first. It then appends through the encoding the array reduces to, so a lazy slice of a constant appends as a constant.
  • Tests now call optimize_ctx(&SESSION) instead of optimize().

API Changes

  • ArrayOptimizer::optimize is deprecated. Use optimize_ctx with the session or an ExecutionCtx.
  • ArrayOptimizer::optimize_ctx and optimize_recursive are generic over AsOptimizerKernels.
  • ArrayRef::slice, filter, take and the ArrayBuiltins methods no longer return an optimized array.

@robert3005
robert3005 added this pull request to stack #11093 November 2, 2026 12:06
@robert3005 robert3005 added the changelog/deprecation A change that introduces a series of API deprecations label Nov 2, 2026
@robert3005
robert3005 force-pushed the rk/deprecate-sessionless-optimize branch 2 times, most recently from b724ab6 to 083a5b5 Compare November 2, 2026 12:28
@robert3005
robert3005 force-pushed the rk/deprecate-sessionless-optimize branch 2 times, most recently from 9d26236 to 45724a5 Compare November 2, 2026 15:11
@robert3005
robert3005 force-pushed the rk/deprecate-sessionless-optimize branch from 45724a5 to f9f348f Compare November 2, 2026 16:53
ArrayOptimizer::optimize uses only the static rules of the encodings, so
it misses the kernels a session registers. It is now deprecated in favour
of optimize_ctx, which takes the session or an ExecutionCtx created from
it through the new AsOptimizerKernels trait. optimize_recursive takes the
same argument.

Constructors such as slice, filter, take, cast, fill_null, get_item and
apply no longer optimize the arrays they build. Execution optimizes each
node it reaches, and the callers that read an array's encoding directly
optimize it with their session first: the scan tasks, the plan v2
operators, the dict layout writer, the FFI scan, and CUDA dispatch.

The default append_to_builder optimizes the array first and appends
through the encoding it reduces to, so a lazy slice of a constant appends
as a constant.

Signed-off-by: Robert Kruszewski <github@robertk.io>
@robert3005
robert3005 force-pushed the rk/deprecate-sessionless-optimize branch from f9f348f to 37c91a6 Compare November 2, 2026 19:18

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/deprecation A change that introduces a series of API deprecations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant