Move expressions out of vortex-array into vortex-expr

Unbound `Expression`, binding, traversal, lambdas, proto serialization and
the expression constructors now live in `vortex-expr`; `vortex-array`
keeps arrays, the scalar function machinery, the builtins it is built on,
and `BoundExpression` with its reduce node and optimizer. The scalar
function vtable reduces over bound expressions, so keeping them next to it
lets every reduction rule stay monomorphized for expression trees as well
as array trees. `vortex-expr` re-exports `BoundExpression`. The layering is
`vortex-array -> vortex-expr -> vortex-functions`.

- `ScalarFnVTable::simplify` is generic over `ReduceNode`, like `reduce`,
  but stays a separate hook that only the expression optimizer applies: a
  simplification may rewrite a node into one whose nullability differs,
  which the array optimizer cannot accept (it must keep the exact dtype and
  defer constant masks to the kernel rules). `ReduceNode::is_same` lets
  rules such as the COALESCE rewrite compare operands.
- `new_expr`, `try_new_expr` and `try_new_bound_expr` move to
  `vortex_expr::ScalarFnExprExt`, `ArrayRef::apply`/`apply_bound` to
  `vortex_expr::ArrayExprExt`, and the expression half of the builtins to
  `vortex_expr::ExprBuiltins`. `ExprDisplay` moves to
  `vortex_array::scalar_fn::display`, and the tree renderer to
  `vortex_array::expr::display`.
- `ArrayRef::take_slot_unchecked`/`put_slot_unchecked` become public, like
  `with_slots`, so row-count substitution in `vortex-expr` still rewrites
  lazy arrays in place.

`vortex_array::expr` otherwise keeps only `stats`, which moves with the
aggregate split. The core builtins' expression-level tests move to
`vortex-expr`; the few array-level tests that used expressions now go through
`ArrayBuiltins`. The `expr_*` benches move to `vortex-expr`, which joins
the CodSpeed shard that builds `vortex-functions`.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ps4mp7KF5c4p66aoSoXx5j
Signed-off-by: Claude <noreply@anthropic.com>

Author: claude

.github/workflows/codspeed.yml

@@ -56,7 +56,7 @@ jobs:
           - { shard: 3, name: "Array dictionaries", packages: "vortex-array", array-group: "dictionary", features: "--features _test-harness" }
           - { shard: 4, name: "Array builders & views", packages: "vortex-array", array-group: "builders", features: "--features _test-harness" }
           - { shard: 5, name: "Core & encodings", packages: "vortex-buffer vortex-error vortex-mask vortex-compute vortex-file vortex-alp vortex-bytebool vortex-datetime-parts" }
-          - { shard: 6, name: "Main library, functions & storage", packages: "vortex vortex-functions vortex-arrow vortex-btrblocks vortex-compressor vortex-row" }
+          - { shard: 6, name: "Main library, functions & storage", packages: "vortex vortex-expr vortex-functions vortex-arrow vortex-btrblocks vortex-compressor vortex-row" }
           - { shard: 7, name: "FastLanes & decimals", packages: "vortex-decimal-byte-parts vortex-fastlanes", features: "--features _test-harness" }
           - { shard: 8, name: "Encodings & layout", packages: "vortex-onpair vortex-pco vortex-runend vortex-sequence vortex-sparse vortex-zigzag vortex-zstd vortex-layout" }
           - { shard: 9, name: "Strings, tensor & spatial", packages: "vortex-fsst vortex-tensor vortex-spatial", features: "--features _test-harness" }
@@ -92,7 +92,7 @@ jobs:
           groups = {
               "dictionary": ("dict_", "chunked_dict_", "patches_"),
               "selection": ("take_", "filter_", "piecewise_sequence_"),
-              "compute": ("aggregate_", "cast_", "expr_", "binary_ops", "compare", "kleene_bool"),
+              "compute": ("aggregate_", "cast_", "binary_ops", "compare", "kleene_bool"),
           }
           manifest = tomllib.loads(pathlib.Path("vortex-array/Cargo.toml").read_text())
           benches = []

AGENTS.md

@@ -22,6 +22,8 @@ documentation in `docs/`, and benchmark tooling in `vortex-bench/` and `benchmar
 - `vortex-array` contains the core `Array` trait and the base encodings, including most
   Apache Arrow-style encodings.
 - `encodings/*` contains more specialized compressed encodings.
+- `vortex-expr` contains expression trees, their optimizer, and the analyses and rewrites built on
+  them. `vortex-functions` contains the scalar functions that `vortex-array` itself is not built on.
 - `vortex-file` implements file IO. It uses `LayoutReader` from `vortex-layout`.
 - `vortex-io` holds the core async and blocking IO traits, plus the generic `object_store`
   adapters that implement them.

Cargo.lock

@@ -9,8 +9,10 @@ to the returned data.
 Expressions are defined as abstract scalar functions. These vtables define the signature of the function, properties
 such as whether the function is strict, and the actual logic for executing the function over input arrays.
 
-The built-in scalar functions can be found in the `vortex-array::expr` module, with additional use-case specific
-functions provided by integration and plugin crates.
+The scalar function framework and the handful of functions the array crate itself is built on live in
+`vortex_array::scalar_fn`; expression trees, their optimizer and the analyses built on them live in the
+`vortex-expr` crate; `vortex-functions` holds the rest of the built-in scalar functions, with additional
+use-case specific functions provided by integration and plugin crates.
 
 In the future, we plan to add full support for spatial functions, date-time functions, and more.
 

docs/concepts/expressions.md

@@ -22,6 +22,7 @@ rand = { workspace = true, optional = true }
 vortex-array = { workspace = true }
 vortex-buffer = { workspace = true }
 vortex-error = { workspace = true }
+vortex-expr = { workspace = true }
 vortex-functions = { workspace = true }
 vortex-mask = { workspace = true }
 vortex-session = { workspace = true }
@@ -36,6 +37,7 @@ rand = { workspace = true }
 rstest = { workspace = true }
 test-with = { workspace = true }
 vortex-array = { workspace = true, features = ["_test-harness"] }
+vortex-expr = { workspace = true }
 
 [[bench]]
 name = "fsst_compress"

encodings/fsst/Cargo.toml

@@ -16,13 +16,14 @@ use vortex_array::arrays::VarBinArray;
 use vortex_array::arrays::VarBinViewArray;
 use vortex_array::arrays::varbin::VarBinArrayExt;
 use vortex_array::builtins::ArrayBuiltins;
-use vortex_array::expr::lit;
-use vortex_array::expr::root;
 use vortex_array::memory::MemorySession;
 use vortex_array::optimizer::kernels::KernelSession;
 use vortex_array::scalar::Scalar;
 use vortex_array::scalar_fn::builtin::operators::Operator;
 use vortex_array::session::ArraySession;
+use vortex_expr::ArrayExprExt;
+use vortex_expr::lit;
+use vortex_expr::root;
 use vortex_fsst::fsst_compress;
 use vortex_fsst::fsst_train_compressor;
 use vortex_fsst::test_utils::HIGH_MATCH_DOMAIN;

encodings/fsst/benches/fsst_url_compare.rs

@@ -44,8 +44,9 @@ mod tests {
     use vortex_array::assert_arrays_eq;
     use vortex_array::dtype::DType;
     use vortex_array::dtype::Nullability;
-    use vortex_array::expr::root;
     use vortex_error::VortexResult;
+    use vortex_expr::ArrayExprExt;
+    use vortex_expr::root;
     use vortex_functions::expr::byte_length;
     use vortex_mask::Mask;
     use vortex_session::VortexSession;

encodings/fsst/src/kernel.rs

@@ -37,6 +37,7 @@ vortex-edition = { workspace = true }
 rstest = { workspace = true }
 tokio = { workspace = true, features = ["full"] }
 vortex-array = { workspace = true, features = ["_test-harness"] }
+vortex-expr = { workspace = true }
 vortex-file = { workspace = true, features = ["tokio"] }
 vortex-io = { workspace = true, features = ["tokio"] }
 vortex-layout = { workspace = true }

encodings/parquet-variant/Cargo.toml

@@ -26,10 +26,11 @@ use vortex_array::assert_nth_scalar_is_null;
 use vortex_array::dtype::DType;
 use vortex_array::dtype::Nullability;
 use vortex_array::dtype::PType;
-use vortex_array::expr::root;
 use vortex_error::VortexResult;
 use vortex_error::vortex_bail;
 use vortex_error::vortex_err;
+use vortex_expr::ArrayExprExt;
+use vortex_expr::root;
 use vortex_functions::expr::variant_get;
 use vortex_functions::variant_get::VariantPath;
 use vortex_json::Json;

encodings/parquet-variant/src/json_to_variant_tests.rs

@@ -325,13 +325,14 @@ mod tests {
     use vortex_array::dtype::DType as VortexDType;
     use vortex_array::dtype::Nullability;
     use vortex_array::dtype::PType;
-    use vortex_array::expr::root;
     use vortex_array::validity::Validity;
     use vortex_arrow::ArrowSessionExt;
     use vortex_error::VortexResult;
     use vortex_error::vortex_bail;
     use vortex_error::vortex_ensure;
     use vortex_error::vortex_err;
+    use vortex_expr::ArrayExprExt;
+    use vortex_expr::root;
     use vortex_functions::expr::variant_get;
     use vortex_functions::variant_get::VariantPath;
     use vortex_functions::variant_get::VariantPathElement;