feat(datalog): aggregates, float constants, between sugar, and typed head constants

[theaspirational]

What this PR gives you

Write smarter rules without dropping to C. Before this PR, Datalog rules in rayforce2 could only do joins and negation. Now you can:

• Aggregate across rows — (count ?n items), (sum ?total invoice value), (avg ?mean sensor reading by ?region col). Grouped or scalar, with the semantics you'd expect: empty groups produce no rows for min/max/avg, and zero for count/sum.
• Use float constants and mixed arithmetic — (= ?bmi (/ ?weight (* ?height ?height))) just works, even when some columns are integers and others are floats. Results auto-promote to f64 when needed.
• Write range filters concisely — (between ?x 18 65) instead of two separate comparison literals.
• Put constants in rule heads — (rule (band "small") (weight ?w) (< ?w 60)) derives labeled categories directly. String, integer, and float constants all work, including across dependent rules (IDB-to-IDB).
• Reference any env-bound table in rule bodies — no need to pre-register each EDB by hand. If a table is bound in the ray env, rules can query it automatically.

These features together let consumers like ray-exomem replace procedural Rust derivation code with declarative Datalog rules — simpler to write, easier to reason about, and evaluated by the engine's optimizer.

Persistent-consumer support: ray_runtime_create_with_sym

Why it was added

The existing ray_runtime_create interns ~175 builtin symbol names (+, -, count, rule, etc.) at IDs 0..N during ray_lang_init. For long-lived consumers that persist data across process restarts, this causes a chicken-and-egg problem:

1. User asserts a fact → interns a string → gets sym ID, say 200.
2. User's column data stores that fact with sym ID 200.
3. Process restarts. Builtins register at 0..174 first, claiming those slots.
4. ray_sym_load(user_file) now tries to restore user strings at their saved positions, but positions 0..174 are already occupied by builtins — divergence detection returns RAY_ERR_CORRUPT.

Upstream-only callers never hit this because they don't persist sym state across runs. Persistent consumers (databases, daemons, anything that reloads splayed tables on startup) need the sym table loaded before builtins claim slots — so that persisted user IDs keep their positions and new builtins append afterwards.

API

• ray_runtime_create_with_sym(path) — loads path if it exists, ignores errors (convenient for callers that don't need to distinguish failure modes).
• ray_runtime_create_with_sym_err(path, &out_err) — returns the load result via out-param so callers can distinguish RAY_OK, RAY_ERR_CORRUPT, and I/O failures, and decide recovery policy.

Original ray_runtime_create(argc, argv) is unchanged — no behavioral difference for existing callers.

Hardening (latest commit)

• Init ordering — __VM and mem_budget are set before ray_sym_load runs, so file I/O errors surface through ray_error() (previously silently dropped) and allocations are bounded by the budget.
• Size pre-flight — stat() the sym file before ray_col_load; reject if st_size > mem_budget / 2 to prevent a malicious or corrupted multi-GB file from OOMing the process before budget guards engage.
• Error transparency — the _err variant surfaces load failures cleanly instead of collapsing them into a single pass/fail.

Safety hardening (review feedback)

• Expression evaluator rejects non-numeric (RAY_SYM) columns instead of assuming 8-byte elements — RAY_SYM has adaptive width and the old memcpy would overread on narrow dictionaries.
• Head-const type conflicts fail fast when two rules for the same predicate disagree on a slot's type, instead of silently letting the last rule win.
• Grouped aggregate bounds — group key column indices are now validated against the source relation's arity.
• Error-path leak in IDB type alignment fixed (table pointer was lost when ray_table_add_col failed).

Test coverage

• 693 / 693 make test pass (ASan + UBSan)
• 9 head-constant tests: single-rule, i64, f64, cross-IDB, with-agg, negation, stratification, surface syntax
• Aggregate: scalar (count/sum/min/max/avg), grouped (count/sum/min/max/avg), empty-source (all 5)
• Aggregate parsing: scalar + grouped surface syntax, 4 reject paths
• Float arithmetic + mixed i64/f64 promotion
• Between sugar parse
• Env-bound EDB auto-registration: end-to-end through ray_query_fn

🤖 Generated with Claude Code

---

Post-review hardening (commits since 4465c88c)

A wave of follow-up fixes by @singaraiona addressed two rounds of Copilot review plus a broader silent-failure audit. Summarized by theme:

Silent-failure eradication in dl_eval and friends

The biggest correctness gain. Before these commits the datalog runtime could return "success with no rows" on genuine compile/evaluation failures — query consumers saw 0 from dl_eval either way.

• dl_eval now returns -1 on failure via a program-scoped eval_err flag flipped at every failure site in dl_compile_rule and the runtime pipeline (dl_project OOM/type/add_col, grouped-agg key/value OOR, scalar-agg OOR, non-numeric value, ray_execute / table_rename_cols / table_union / table_distinct errors). Legitimate empty rules still return 0. (d440773240b3)
• Phase B of the semi-naive loop had several continue-without-flag paths that bypassed eval_err — Phase A/B ray_graph_new OOM, union accumulator, table_distinct / table_antijoin per-IDB merge, delta-into-relation union. All now flag. (25a494f47a13)
• table_union no longer silently skips columns whose concat failed, no longer narrows to min(ncols_a, ncols_b), and the schema-mismatch check runs before the empty-rows short-circuit. All three failures now return typed ray_error. (8dbd16087cd1, 52d374c2dd12, b9ffe63267d3)
• table_distinct / table_antijoin canonicalize/graph_new failures return ray_error instead of falling back to the raw input (which may not be distinct, or isn't the antijoin of anything). (8dbd16087cd1)
• dl_project propagates errors instead of continue-ing past the failed slot, and now falls back to the IDB's existing column types when accum collapses to 0 rows with no schema (e.g. fully-filtered antijoin) to preserve table_union schema agreement. (039496e8a5f8)

Aggregate and auto-register correctness

• Scalar SUM/MIN/MAX/AVG over RAY_F64: previously treated any non-RAY_I64 as "return 0"; now i64→i64, f64→f64, non-numeric rejected. (510bf600e02c)
• Empty-source F64 SUM identity: the type check was inside the nrows > 0 branch, so empty f64 sources emitted RAY_I64 0 instead of RAY_F64 0.0. Lifted above the split. (15f01bf3f55d)
• Scalar-agg value_col bounds check moved above the empty-source early return so OOR rejects regardless of row count. (7af5b693521e)
• Env-bound EDB auto-register: dl_add_edb returns 0, 1, 2, …, not just 0; the != 0 check rejected every EDB beyond the primary one. Fixed to < 0, so multi-EDB rule bodies actually work. (ffbd67924d6c)
• Agg parser arity: when prog == NULL (surface syntax, globals) the parser hardcoded pred_arity=1, so env-bound tables with any other arity were silently skipped during auto-register. Now uses 0 as an "unknown at parse time" sentinel. (039496e8a5f8)
• Grouped-agg bounds: group_key_col and value_col are bounds-checked against source arity before indexing col_names[]; out-of-range key previously OOB-read, out-of-range value was silently clamped to col 0. (510bf600e02c)
• Env auto-register missing-column rejection: a NULL from ray_table_get_col_idx used to continue, producing a cleaned table with fewer than pred_arity columns that dl_add_edb still registered. Now returns a schema error. (8f79c9430dfc)

Narrow-width SYM projection and STR pool propagation

• dl_project SYM width: ray_vec_new(RAY_SYM, …) always returns W64, so memcpy with the source's narrower stride left upper bytes uninit and produced bogus sym IDs. Now routes SYM columns through ray_sym_vec_new(src->attrs & RAY_SYM_W_MASK, …). (510bf600e02c)
• dl_project STR pool: the memcpy path for RAY_STR copied the 16-byte ray_str_t handles but left dst->str_pool NULL, dereferencing pool_off into a NULL pool on reads of strings >12 bytes. Now calls col_propagate_str_pool(dst, src). (ce9aef53eb76)

Error-block memory lifetime

ray_release, ray_retain, and ray_free all short-circuit on RAY_IS_ERR, so every if (RAY_IS_ERR(x)) ray_release(x); site was quietly leaking the error block.

• New ray_error_free(ray_t*) published in include/rayforce.h — retypes the error to a leaf atom so the standard free path reclaims it. Wired at ~15 sites across dl_idb_align_head_const_types, dl_project, dl_compile_rule, dl_eval Phase A/B, ray_query_fn env auto-register, dl_col_as_f64 / dl_eval_expr. Regression test /datalog/error_free_reclaims loops 256 create/free cycles and asserts ray_mem_stats() free_count advances and bytes_allocated stays flat. (ed7520e2a1eb, ce9aef53eb76)
• table_union pass-through now retains b even when it's a RAY_ERROR, so callers that release the returned pointer don't alias freed memory. (8f79c9430dfc)

Persistent-consumer runtime surface polish

• ray_runtime_create_with_sym / _with_sym_err prototypes published in src/core/runtime.h — previously defined in runtime.c but undeclared, forcing consumers to hand-duplicate the signature. (8f79c9430dfc)
• stat() errno handling in ray_runtime_create_with_sym_err: ENOENT (fresh install) stays RAY_OK; every other errno (EACCES, ENOTDIR, EIO, …) now sets *out_sym_err = RAY_ERR_IO instead of silently producing an empty sym table. (ce9aef53eb76)
• test/test_runtime.c covers the persistent-consumer surface: absent-is-ok, ENOTDIR-surfaces-error, plain-variant behavior. (ce9aef53eb76)

Test count

693/693 → 704/704 (11 new tests), green on debug (ASan+UBSan) and release.

[Copilot]

Pull request overview

This PR extends the rayforce2 Datalog engine with aggregate body literals, float constants in expressions, between surface-syntax sugar, and typed head constants (including constant-slot broadcast projection across IDBs), plus associated integration and unit tests.

Changes:

• Add DL_AGG body literals (scalar + grouped) with stratification support and evaluation in dl_compile_rule.
• Add float constants to the assignment expression AST (DL_EXPR_CONST_F64) with mixed i64/f64 promotion.
• Extend surface syntax parsing to support aggregates, between, typed head constants, and string constants in body positions; add extensive tests.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
test/test_datalog.c	Adds many new Datalog feature tests, including surface-syntax runtime-driven cases.
src/ops/datalog.h	Extends public Datalog API/types for aggregates, float constants, and typed head constants.
src/ops/datalog.c	Implements aggregate compilation/eval, float expression support, typed constant projection, parser updates, and env-bound EDB auto-registration.
docs/plans/2026-04-18-datalog-aggregates-and-onboarding.md	Adds the implementation plan document referenced by the PR.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

dl_idb_align_head_const_types rebuilds fresh and then returns early on RAY_IS_ERR(fresh) without releasing the error object / partially-built table. This can leak and also leaves the IDB schema in an indeterminate state for the remainder of compilation. Ensure that any error return releases fresh (and ideally leaves rel->table unchanged) before returning.

Suggested change

	if (!fresh || RAY_IS_ERR(fresh)) return;
	for (int c = 0; c < rel->arity; c++) {
	ray_t* empty_col = ray_vec_new(desired[c], 0);
	if (!empty_col || RAY_IS_ERR(empty_col)) { ray_release(fresh); return; }
	fresh = ray_table_add_col(fresh, rel->col_names[c], empty_col);
	ray_release(empty_col);
	if (RAY_IS_ERR(fresh)) return;
	if (!fresh) return;
	if (RAY_IS_ERR(fresh)) { ray_release(fresh); return; }
	for (int c = 0; c < rel->arity; c++) {
	ray_t* empty_col = ray_vec_new(desired[c], 0);
	if (!empty_col || RAY_IS_ERR(empty_col)) { ray_release(fresh); return; }
	fresh = ray_table_add_col(fresh, rel->col_names[c], empty_col);
	ray_release(empty_col);
	if (RAY_IS_ERR(fresh)) { ray_release(fresh); return; }

[Copilot]

dl_idb_align_head_const_types can rebuild the same IDB table multiple times while the relation is still empty, meaning the final column types depend on rule-add order. If two rules for the same head predicate specify conflicting head_const_types for a slot, evaluation will later fail when projecting/unioning mismatched column types. Consider tracking a per-relation “committed” column type per slot (first non-zero wins) and failing fast on conflicts instead of silently re-aligning.

[Copilot]

The runtime API is re-declared manually here (forward-declared struct + externs). This is brittle if the signature or globals change, and the setup doesn’t check ray_runtime_create() for failure before dereferencing __RUNTIME in teardown. Prefer including the owning header (e.g. core/runtime.h) and asserting the returned runtime pointer is non-NULL, then pass that pointer to ray_runtime_destroy().

[Copilot]

In dl_eval_expr's DL_EXPR_VAR case, any non-RAY_F64 source column is treated as RAY_I64 and copied with a fixed 8-byte element size. If a variable ever binds to a non-numeric column (e.g., RAY_SYM or other types), this will reinterpret data and can overread/mis-copy when the underlying element size differs. Consider explicitly rejecting non-(RAY_I64|RAY_F64) sources (return an error) or copying with ray_sym_elem_size(src->type, src->attrs) and preserving the source type where appropriate.

[Copilot]

Grouped aggregate compilation indexes src_rel->col_names[body->agg_group_key_cols[i]] without validating that each group_key_col is within [0, src_rel->arity). Since key column indices come from rule builders / surface syntax, an out-of-range value can cause out-of-bounds reads and crashes. Add bounds checks (and return a structured error) before using agg_group_key_cols, and similarly validate agg_value_col for the grouped case.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

dl_project allocates dst via ray_vec_new(src->type, nrows). For RAY_SYM columns this always creates a W64 vector, but the memcpy uses src->attrs element size (possibly 1/2/4). This under-copies and leaves garbage (and breaks sym reads) when src is a narrow SYM vec. Allocate dst with the same SYM width (e.g. ray_sym_vec_new(src->attrs & RAY_SYM_W_MASK, nrows) or the existing col_vec_new helper) or explicitly widen while writing via ray_read_sym/ray_write_sym.

[Copilot]

Grouped-aggregate compilation leaks the ray_graph_t gg on the early error path when a group key column index is out of range (it returns after allocating gg). Ensure gg is freed before returning on these validation failures, and consider also checking ray_scan results for NULL/error before proceeding.

[Copilot]

runtime_create_impl treats any stat() failure as “no sym file” (out_sym_err stays RAY_OK) and skips ray_sym_load entirely. That hides real I/O problems (e.g., EACCES) and defeats the whole "load sym before builtins" guarantee for persistent consumers. Consider distinguishing ENOENT from other errors and either (a) attempt ray_sym_load directly and rely on its error codes, or (b) set out_sym_err appropriately for non-ENOENT stat failures.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Scalar SUM/MIN/MAX/AVG aggregation only handles RAY_I64 value columns; if the source column is RAY_F64 (which is now more common with float constants / mixed arithmetic), the code silently returns 0 instead of a correct result or an explicit error. Consider supporting both RAY_I64 and RAY_F64 here (mirroring the grouped path via ray_group), or returning a type error when the value column isn't numeric.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

dl_project() currently treats allocation failures by continue-ing, which can (a) leak the RAY_ERROR object returned from ray_vec_new()/dl_broadcast_const_col() and (b) produce an output table missing columns (silently corrupting the derived relation schema). Instead, release any error objects and propagate a failure (return an error table) so callers can abort rule evaluation cleanly.

[Copilot]

In the grouped-aggregate compile path, if a group key column index is out of range you early-return without freeing the sub-graph gg (created just above). This leaks the graph and any retained resources. Ensure gg is freed on all error paths after ray_graph_new(), including the out-of-range key-col case.

Suggested change

	if (kc < 0 || kc >= src_rel->arity) { ray_release(accum); return NULL; }
	if (kc < 0 || kc >= src_rel->arity) {
	ray_graph_free(gg);
	ray_release(accum);
	return NULL;
	}

[Copilot]

Aggregate parsing sets pred_arity to 1 when prog is NULL. That value is later used by ray_query_fn's env-bound EDB auto-registration (it checks ncols != pred_arity for DL_AGG too), so aggregates over env-bound relations (or over any non-arity-1 predicate defined via surface syntax before a program exists) can fail to auto-register and then fail at compile time. Consider storing an 'unknown' arity (e.g. 0/-1) for DL_AGG when prog is NULL and relaxing the env auto-register arity check in that case, or resolving arity at compile time instead of parse time.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

dl_idb_align_head_const_types(): if ray_table_add_col() returns a RAY_ERROR, the code releases prev but leaks the returned error object (fresh). Release the error before returning to avoid accumulating errors on repeated calls (same issue for a RAY_ERROR return from ray_table_new).

Suggested change

	if (!fresh || RAY_IS_ERR(fresh)) return;
	for (int c = 0; c < rel->arity; c++) {
	ray_t* empty_col = ray_vec_new(desired[c], 0);
	if (!empty_col || RAY_IS_ERR(empty_col)) { ray_release(fresh); return; }
	ray_t* prev = fresh;
	fresh = ray_table_add_col(fresh, rel->col_names[c], empty_col);
	ray_release(empty_col);
	if (RAY_IS_ERR(fresh)) { ray_release(prev); return; }
	if (!fresh) return;
	if (RAY_IS_ERR(fresh)) { ray_release(fresh); return; }
	for (int c = 0; c < rel->arity; c++) {
	ray_t* empty_col = ray_vec_new(desired[c], 0);
	if (!empty_col || RAY_IS_ERR(empty_col)) { ray_release(fresh); return; }
	ray_t* prev = fresh;
	fresh = ray_table_add_col(fresh, rel->col_names[c], empty_col);
	ray_release(empty_col);
	if (RAY_IS_ERR(fresh)) { ray_release(prev); ray_release(fresh); return; }

[Copilot]

The new public entrypoints ray_runtime_create_with_sym() / ray_runtime_create_with_sym_err() are defined here, but they aren’t declared in src/core/runtime.h or the public include/rayforce.h (tests work around this via manual externs). Please add the prototypes to the appropriate headers so consumers can use the APIs without duplicating declarations, and so signature changes can’t silently drift.

Suggested change

	/* NOTE: These public entrypoints must also be declared in src/core/runtime.h
	* and include/rayforce.h to keep the API surface synchronized.
	*/

[Copilot]

ray_query_fn env auto-registration: if ray_table_get_col_idx(env_val, c) returns NULL, the loop currently continues, which can build a clean table with fewer than pred_arity columns but still registers it via dl_add_edb(prog, pred_name, clean, pred_arity). That can leave the program in a schema-inconsistent state. Treat a missing column as a hard error (or at least abort registration for that predicate) so only fully-formed tables are registered.

Suggested change

	if (!col) continue;
	if (!col) {
	ray_release(clean);
	dl_program_free(prog);
	ray_release(db);
	return ray_error("invalid", "query: env-backed EDB table missing expected column");
	}

[Copilot]

In table_union(), when a is NULL or an error, the function returns b but only retains it if b is not an error. Callers typically release their original b after union; if b is a RAY_ERROR this can become a use-after-free (merged points to freed memory). Fix by retaining b whenever returning it (even if it is an error), or by documenting/adjusting ownership so callers don’t release the input they pass through unchanged.

Suggested change

	if (b && !RAY_IS_ERR(b)) ray_retain(b);
	if (b) ray_retain(b);

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

dl_col_as_f64() (and other helpers in this area) returns NULL when ray_vec_new() returns a RAY_ERROR. Because ray_release() is a no-op for RAY_ERROR, this leaks the error object. Prefer returning the RAY_ERROR to the caller (so dl_compile_rule/dl_eval can surface/free it) or calling ray_error_free(out) before returning NULL.

Suggested change

	if (!out || RAY_IS_ERR(out)) return NULL;
	if (!out) return NULL;
	if (RAY_IS_ERR(out)) {
	ray_error_free(out);
	return NULL;
	}

[Copilot]

dl_project() memcpy-copies RAY_STR columns into a freshly allocated vector but doesn't propagate the source column's str_pool. For pooled strings (>12 bytes), dst elements' pool_off will point into a NULL/empty pool, causing invalid reads later. After allocating dst for RAY_STR, propagate the pool (e.g., col_propagate_str_pool(dst, src)) or use a string-aware copy path like other ops do.

[Copilot]

runtime_create_impl(): when stat(sym_path, …) fails, the code currently leaves out_sym_err = RAY_OK and skips ray_sym_load entirely. That hides real I/O errors (EACCES, ENOTDIR, etc.) and contradicts the header comment that the _err variant surfaces I/O failures. Consider: (1) treat errno==ENOENT as OK/first-run, but for other errno set *out_sym_err = RAY_ERR_IO; and/or (2) call ray_sym_load regardless and only use stat for the size pre-flight when it succeeds.

[Copilot]

New public APIs ray_runtime_create_with_sym / ray_runtime_create_with_sym_err change init ordering and error semantics, but there are still no tests exercising the key cases (load-before-builtins stability, corrupt file -> RAY_ERR_CORRUPT, oversized file -> RAY_ERR_OOM, and non-ENOENT I/O failure -> RAY_ERR_IO). Adding targeted runtime/sym integration tests would help prevent regressions in persistent-consumer scenarios.

[singaraiona]

@copilot resolve the merge conflicts in this pull request

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

Comments suppressed due to low confidence (1)

src/core/runtime.c:193

• runtime_create_impl initializes global subsystems (ray_heap_init/ray_sym_init) and then has multiple allocation failure exits that return NULL without tearing those subsystems down or resetting globals. This can leave the process in a partially-initialized state and make a subsequent runtime_create call behave unpredictably. Add a single cleanup path that calls ray_sym_destroy/ray_heap_destroy and clears __VM/__RUNTIME on failure.

    /* Init subsystems */
    ray_heap_init();
    ray_sym_init();

    /* Allocate runtime and set __VM + mem_budget BEFORE any file I/O so
     * that ray_error() has a live VM to record diagnostics against and
     * allocations are bounded by the budget. */
    ray_runtime_t* rt = (ray_runtime_t*)ray_sys_alloc(sizeof(ray_runtime_t));
    if (!rt) return NULL;
    memset(rt, 0, sizeof(*rt));

    /* Create main VM (id=0) */
    rt->n_vms = 1;
    rt->vms = (ray_vm_t**)ray_sys_alloc(sizeof(ray_vm_t*));
    if (!rt->vms) { ray_sys_free(rt); return NULL; }
    rt->vms[0] = (ray_vm_t*)ray_sys_alloc(sizeof(ray_vm_t));
    if (!rt->vms[0]) { ray_sys_free(rt->vms); ray_sys_free(rt); return NULL; }

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

test_create_with_sym_io_error_surfaces currently asserts only err != RAY_OK. Since runtime_create_impl explicitly maps non-ENOENT stat failures to RAY_ERR_IO, this test should assert the specific expected code (RAY_ERR_IO) so it will catch accidental changes in the error mapping.

Suggested change

	munit_assert_int((int)err, !=, (int)RAY_OK);
	munit_assert_int((int)err, ==, (int)RAY_ERR_IO);

[Copilot]

The runtime-with-sym tests cover ENOENT and a non-ENOENT stat failure, but they still don't exercise several key behaviors called out in the PR description / prior review feedback: (1) corrupt sym file surfaces RAY_ERR_CORRUPT via the _err variant, (2) oversized sym file triggers the pre-flight RAY_ERR_OOM path, and (3) load-before-builtins ordering (user IDs preserved, builtins appended). Adding targeted tests for these cases would prevent regressions in the persistence contract.

[Copilot]

In dl_project's empty-accum fallback, when the head relation column type is RAY_SYM it creates an empty column via ray_vec_new(RAY_SYM, 0), which will default to W64 and can mismatch a head_rel SYM column that uses a narrower width. Since this path exists specifically to keep schemas compatible for downstream unions, it should preserve SYM attrs/width (e.g., by mirroring hcol->attrs via ray_sym_vec_new) rather than forcing W64.

[Copilot]

dl_set_body_pos now interns quoted string literals into SYM constants, which will cause the constant-filter path in dl_compile_rule (dl_filter_eq) to run against RAY_SYM columns. dl_filter_eq currently treats the filter column as int64_t* and compares raw 8-byte values, which is incorrect for adaptive-width SYM vectors (W8/W16/W32) and can miscompare/overread. The filter implementation should become type-aware (use ray_read_sym for RAY_SYM and element-size-aware copying for non-I64 types).

Suggested change

	if (node->type == -RAY_STR) {
	/* Quoted string literal in body: intern as sym so it compares
	* equal to other sym-interned constants. Mirrors the head
	* parser convention. */
	int64_t sym = ray_sym_intern(ray_str_ptr(node), ray_str_len(node));
	dl_body_set_const(rule, bidx, pos, sym);
	return NULL;
	}
	/* Do not eagerly intern quoted string literals here.
	*
	* Interning body strings into SYM constants routes them into the
	* constant-filter path, which must be type-aware for adaptive-width
	* SYM vectors. Until that filter path is updated, fall through to the
	* generic evaluator instead of manufacturing a SYM constant here.
	*/

[Copilot]

In the scalar aggregate path, allocation failure for new_col is silently ignored (break), and if ray_vec_new returns a RAY_ERROR it is not freed. This can both leak the error object and allow evaluation to proceed with an unbound aggregate target var. Treat this as a hard failure: free any RAY_ERROR returned from ray_vec_new, set prog->eval_err, and return NULL so dl_eval returns -1.

Suggested change

	if (!new_col || RAY_IS_ERR(new_col))
	break;
	if (!new_col) {
	ray_release(accum);
	prog->eval_err = true;
	return NULL;
	}
	if (RAY_IS_ERR(new_col)) {
	ray_release(new_col);
	ray_release(accum);
	prog->eval_err = true;
	return NULL;
	}

[singaraiona]

@copilot resolve the merge conflicts in this pull request