feat: SQL type & expression coverage (temporal, numeric, array, predicates, functions)#29
Merged
Conversation
Add a DATE logical type — days since the Unix epoch, backed by i32 and rendered as YYYY-MM-DD. chrono::NaiveDate handles calendar parsing, leap-year validation, and formatting; the on-disk form stays a plain i32. Supports `DATE 'YYYY-MM-DD'` typed literals, DATE columns, and date-vs-date comparison (no integer coercion). Fix the signed-integer KeyCodec to be order-preserving: flip the sign bit and emit big-endian, so lex byte-sort matches numeric order across the negative/positive boundary (little-endian sorted negatives after positives). DATE keys inherit the corrected i32 encoding. Also fold unary minus on integer literals at bind time, so negative keys are reachable from SQL (previously `-100` was an unsupported unary op). Coverage: temporal/codec/expression unit tests plus E2E specs (types.slt, key_ordering.slt) exercising literals, validation, NULL storage, and PK ordering including pre-epoch dates.
Add a tz-aware TIMESTAMP logical type — an absolute instant stored as i64 microseconds since the Unix epoch, UTC. Following Postgres timestamptz semantics: an offset in a literal is honored and normalized to UTC, a literal with no offset is taken as UTC (we have no session time zone), and output renders as UTC (+00:00). chrono parses the space- and T-separated forms with optional offset/fractional seconds. Bare TIMESTAMP (WITHOUT TIME ZONE) is rejected as unsupported — it's a distinct type we don't model yet. Timestamps ride the i64 codec, so they inherit the order-preserving key encoding. Coverage: temporal unit tests (offset normalization, Z/short offset, no-offset-is-UTC, render roundtrip) plus E2E specs in types.slt and a key_ordering.slt section for pre-epoch instants.
Add IEEE-754 float types following Postgres sizing: REAL / FLOAT4 are 4-byte (f32), DOUBLE PRECISION / FLOAT8 are 8-byte (f64), bare FLOAT is double precision, and FLOAT(p) is single for p ≤ 24. Both ride new fixed-width codecs with an order-preserving key encoding (IEEE total-order bit-flip + big-endian) so floats sort -inf < … < +inf under lex compare. Comparisons coerce numerics to f64 via total_cmp (consistent with the key order); a float against an integer compares numerically. Unsuffixed decimal/exponent literals (1.5, 1e3) bind as Float64, unary minus folds negative float literals, and int→float / cross-width float coercion is allowed on insert (float→int still needs an explicit cast). Output renders at native width with Postgres spelling (Infinity/-Infinity/NaN). Coverage: expression coercion tests, codec float key-order test (signed zero + infinities), and E2E specs — REAL-vs-DOUBLE precision is observable, plus a DOUBLE key-ordering section spanning ±Infinity.
The parser already accepts function calls; this wires them through the binder and evaluator. Adds an Expr::Call node with a ScalarFunc (Abs/Greatest/Least) and row-wise evaluation: - ABS(x): absolute value over ints/floats; NULL propagates; i*::MIN overflows to a type error (Postgres behavior). - GREATEST/LEAST(a, b, …): largest/smallest non-null argument, returned as-is; all-null yields NULL. MAX(a, b, …)/MIN(a, b, …) with two or more args are aliases for these. Single-argument MAX/MIN stay unsupported — they're aggregates that need GROUP BY. Unknown functions, named/wildcard args, FILTER/OVER/DISTINCT, and qualified names are rejected. Coverage: expression unit tests (abs incl. overflow, greatest/least with null-skipping and numeric coercion) plus an E2E functions.slt spec covering literals, columns, coercion, and the unsupported aggregate path.
Add an exact decimal type (NUMERIC / DECIMAL) backed by rust_decimal. Values come from typed `NUMERIC '…'` literals; a bare 1.5 still binds as a float. In-row storage uses rust_decimal's compact 16-byte form. The key encoding is the substance: rust_decimal's own bytes aren't sortable, so storage::codec implements a CockroachDB-style order-preserving encoding — sign bucket, then exponent, then big-endian base-100 "centimal" digits, with the payload one's-complemented for negatives. Equal values (1.5 vs 1.50) encode identically, so a NUMERIC primary key sorts and dedups correctly. Comparisons are exact for numeric-vs-numeric and numeric-vs-integer, and fall back to f64 when a float is involved (matching Postgres promotion). ABS works on numerics; int→numeric and numeric→float coercions are allowed on insert. Precision/scale aren't enforced yet (values keep their own scale; ~28 significant digits). Coverage: codec tests (value roundtrip, order preservation across sign and scale, equal-value-equal-key), expression exactness tests, and an E2E numeric.slt plus a NUMERIC key-ordering section.
codec.rs had grown to ~750 lines holding every per-type impl plus the
decimal encoder and a large test module. Split it into a codec/ module:
- mod.rs — traits, errors, Value dispatch, the shared `take` helper, tests
- scalar.rs — fixed-width: bool, i16/i32/i64, f32/f64 (incl. the
order-preserving integer/float key encodings)
- text.rs — variable-length: Text, Bytes, Json
- decimal.rs — Numeric: 16-byte value form + the order-preserving key encoder
No behavior change; child modules reach the private `take` via `super`.
All tests and specs green.
Extend the scalar-function set: - CEIL / FLOOR / ROUND — round numbers (int passes through, float and numeric round in their own type). CEIL/FLOOR arrive as sqlparser's own AST nodes (they support `CEIL(x TO field)`), bound via bind_ceil_floor; the TO/scale forms are rejected for now. - COALESCE(a, b, …) — first non-null argument. - LENGTH (character count), REPEAT(s, n), UPPER, LOWER — text helpers. Coverage: expression unit tests and functions.slt assertions for each.
binder.rs had grown to ~914 lines. Split it into a binder/ module by
construct:
- mod.rs — Binder, BindNode trait, statement dispatch, shared
three_part_name / is_tz_aware helpers, the Bind pass
- ddl.rs — CREATE TABLE / CREATE SCHEMA + column & type mapping
- dml.rs — INSERT + value coercion
- query.rs — SELECT (query body, FROM, WHERE, projection)
- expr.rs — scalar expressions, functions, typed-string literals
No behavior change. Submodules reach the Binder's private scope methods
and the shared helpers through the parent; the three statement entry
points are pub(super). All tests and specs green.
The Limit operator existed but wasn't bound from SQL, and there was no OFFSET. Add an Offset operator (skip first n rows) across the logical / physical / lowering / Volcano layers, and bind `LIMIT k OFFSET n` in the query binder: OFFSET wraps closest to the input, LIMIT outside it, so it reads as "skip n, then take k". Counts are constant non-negative integers (negative LIMIT/OFFSET is a bind error). ORDER BY and FETCH were previously parsed and silently ignored — now rejected as unsupported so they don't yield wrong results. The server's output-column count looks through Limit/Offset to find the projection. DELETE stays unwired on purpose: a correct DELETE is an MVCC tombstone (a new version marked deleted), which needs MVCC we don't have yet. Coverage: Volcano offset/paging unit tests and a limit.slt spec (LIMIT, OFFSET, combined paging, zero/past-end, negative, ORDER BY reject).
- IS NULL / IS NOT NULL: a new total Expr::IsNull (always true/false, never NULL — `NULL IS NULL` is true). - LIKE / NOT LIKE: a new Expr::Like with a `%`/`_` wildcard matcher (two-pointer with backtracking); NULL operands propagate, case- sensitive. LIKE ANY / ESCAPE are rejected for now. - BETWEEN / NOT BETWEEN and IN / NOT IN lower at bind time to existing primitives — `x >= low AND x <= high`, and an OR-chain of equalities — so SQL 3-valued logic (e.g. `3 IN (1, NULL)` -> NULL) falls out for free. Empty `IN ()` is false. Coverage: expression unit tests and a predicates.slt spec covering each operator plus the NULL cases.
- Unary minus is now a runtime Expr::Neg evaluated per row, so `-column` (and any expression) works, not just constant literals. Replaces the bind-time constant-folding shim; i*::MIN overflows to a type error. - NULLIF(a, b) and CONCAT(a, b, …) as scalar functions (plain calls). CONCAT stringifies non-text args and skips NULLs. - SUBSTRING (1-indexed, optional FROM/FOR) and TRIM (BOTH/LEADING/ TRAILING, optional chars) — both are dedicated sqlparser AST nodes, so they bind through their own arms into Substr / Trim* calls with the SQL defaults filled in. Coverage: expression unit tests for each, plus functions.slt additions including `-v` over a column.
Until page-span overflow exists, a TEXT/BYTES value larger than a storage page can't be written and fails opaquely at the page layer. Add a size check in TableWriter::put against tunable MAX_TEXT_SIZE (8000 B, comfortably under the 8 KiB page) and MAX_ARRAY_SIZE (1024, for the future ARRAY type), surfacing a clear "needs page-span support" error instead. Raise or remove once overflow pages land.
`a || b` maps to a new BinaryOperator::Concat that stringifies both operands (reusing concat_str) and joins them; NULL propagates. Same stringification as CONCAT, but the binary operator form.
New Expr::Case: the first branch whose condition is true wins, else the ELSE result (or NULL). The binder lowers simple `CASE x WHEN v THEN …` into searched form (`x = v`), so the evaluator only handles boolean conditions. NULL/false conditions skip, matching SQL semantics.
New Expr::Cast covering every sensible (source, target) pair across our type set: numerics convert across int/float/numeric widths (float/numeric -> int rounds, narrowing range-checks); text parses to numbers, bool, date, timestamp, numeric, json; anything renders to text; date<->timestamp convert (timestamp->date drops the time, date->timestamp is midnight UTC). NULL casts to NULL; unconvertible pairs are a clear type error. The binder reuses the DDL type mapping (bind_data_type, now pub(super)). Coverage: eval_cast unit tests + a cast.slt spec across the matrix.
- TIME (without time zone): i64 microseconds since midnight; rides the i64 codec (order-preserving key). `TIME 'HH:MM:SS'` literals, TIMETZ rejected. - UUID: 16 bytes (uuid crate); raw-bytes key already sorts like Postgres orders uuids. `UUID '…'` literals. Both flow through codec, comparison, the CAST matrix (text<->), concat stringify, the binder type/literal paths, coercion, and server render. Coverage: temporal time unit tests, plus types.slt sections for TIME (literals, ordering, column, cast) and UUID (literal, byte-ordered PK, cast normalization).
Add E2E rejection tests: out-of-range/garbage TIME and malformed UUID literals (and casts) error at bind time, matching the parse-side validation in temporal::parse_time and Uuid::parse_str.
resolve_scalar_func mixed name resolution with ad-hoc, per-function arity checks (ABS/NULLIF inline, others in grouped arms, REPEAT separate) — easy to forget when adding a function. Replace it with a single name -> (ScalarFunc, Arity) table plus one uniform arity validation, so adding a function is one row. MAX/MIN keep their aggregate special-case. Also extract eval_nullif so eval_call is uniformly helper-dispatched. No behavior change.
INTERVAL stores months/days/microseconds separately (a month isn't a fixed number of days), so '1 mon' and '30 days' store and read back distinctly. Comparison and the order-preserving key use a normalized i128 micro count (30-day months, 24-h days), so '1 mon' = '30 days' — matching Postgres `=`. Value codec keeps the 3 components verbatim (16 B, new codec/interval.rs); key codec encodes the normalized value. Parsing/formatting reuse the pg_interval crate (already in the tree) via our own serde-friendly Interval type. `INTERVAL '…'` literals (postgres / SQL / ISO-8601 forms, plus `INTERVAL '5' DAY`), CAST text<->interval, and render all wired. Coverage: temporal unit tests (normalize-equal-but-distinct, format) and a types.slt INTERVAL section.
Add ARRAY (single element type, no nesting in v1). LogicalType becomes
recursive (Array(Box<LogicalType>)) and loses Copy; Value::decode and
eval_cast now take &LogicalType. Value::Array(Vec<Value>):
- codec: u32 count + each element (value form); element keys concatenated
(element-wise ordering, correct for fixed-width elements); recursive
decode using the column's element type.
- comparison: element-wise (a prefix sorts before a longer array);
equality coerces element types (Int32 vs Int64).
- ARRAY[...] literals via a runtime Expr::Array; {a,b} rendering; element
coercion to the column's element type; NULL elements rejected.
- size guard now caps an array's encoded size to a page too.
Nested array types are rejected. Coverage: codec roundtrip + element-wise
key-order unit tests, and a thorough array.slt (round-trip, literals,
comparison, text arrays, nested-rejected, wrong-element-type, null-element).
Per-feature specs exercise features in isolation; add queries.slt that combines them in realistic queries — LIKE + comparison + AND, projections stacking || + CASE, string-function + CAST, COALESCE over nullable columns, LIMIT over a filtered set, and BETWEEN + NOT IN + IS NOT NULL together. Closes the integration gap where feature interactions weren't covered.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Broad expansion of SQL feature coverage, built type-by-type with per-feature spec tests.
Types
Expressions & predicates
||concat, CASE, CAST /::(full matrix)CONCAT, SUBSTRING, TRIM, LENGTH/REPEAT/UPPER/LOWER
Hardening & refactors
Tests
11 per-feature spec files plus an integration spec combining features in
realistic queries. Full workspace suite + clippy
-D warningsclean.🤖 Generated with Claude Code