Releases: lance-format/lance-duckdb
Releases · lance-format/lance-duckdb
v0.4.1
Added
lance_vector_searchnow supports thenprobsandrefine_factorparameters.
Fixed
- Fixed vector search to accept query vectors as DuckDB ARRAY types (e.g.
FLOAT[dim],DOUBLE[dim]) in addition to LIST types. - Fixed
COPY ... (FORMAT lance)to normalize LIST vector columns so they can be used with vector search and indexing.
Commits
- fix: Fix DIM support in vertor search by @Xuanwo in #119
- feat: Add nprobs and refine_factor support by @Xuanwo in #120
- Bump to version v0.4.1 by @Xuanwo in #121
Full Changelog: v0.4.0...v0.4.1
v0.4.0
Added
- Transactional
DELETEsupport (honorsBEGIN/COMMIT/ROLLBACK). - Support
UPDATEwithout aWHEREclause to update all rows. - Predicate pushdown for string functions:
starts_with,ends_with,contains. - Support
CREATE INDEX ... ON ns.main.tbl(col) USING BTREEon attached namespaces.
Changed
- Breaking: Object store credential/configuration now uses
CREATE SECRET (TYPE LANCE, ...)(instead ofTYPE S3). - Aligned directory and REST namespace behavior.
Fixed
UPDATE ... SET col = DEFAULTnow works correctly.
Commits
- docs: Polish our project goal by @Xuanwo in #106
- feat: Allow update the full table without where by @Xuanwo in #107
- fix: Fix UPDATE with DEFAULT support by @Xuanwo in #108
- feat: Add starts_with/ends_with/contains pushdown by @Xuanwo in #110
- feat: Add transaction support for delete by @Xuanwo in #112
- feat: Add support for CREATE INDEX ... ON ns.main.tbl(col) USING BTREE by @Xuanwo in #111
- refactor!: Use lance secrets instead by @Xuanwo in #113
- refactor: Align namespace behavior between dir and rest by @Xuanwo in #114
- Bump to verson 0.4.0 by @Xuanwo in #116
Full Changelog: v0.3.0...v0.4.0
v0.3.0
Added
COPY ... TO ... (FORMAT lance, ...)writer for creating and appending Lance datasets.- Directory and REST namespaces via
ATTACH ... (TYPE LANCE, ...). - DDL support for attached namespaces:
CREATE TABLE,CTAS,DROP TABLE,ALTER TABLE(schema evolution and comments). - DML support for attached namespaces:
INSERT,UPDATE,DELETE,TRUNCATE TABLE. - Index DDL on dataset URIs:
CREATE INDEX,SHOW INDEXES,DROP INDEX(vector, scalar, and full-text). - Additional scan pushdown and planning improvements:
LIMIT/OFFSET,TABLESAMPLE SYSTEM,LIKE,regexp_matches,
IS (NOT) DISTINCT FROM, and expanded type coverage (including timestamp/decimal/struct filters). - DuckDB
rowid/ Lance_rowidsupport for scans, enabling point-lookup and rowid-based optimizations. - Basic scan statistics for improved planning.
Changed
- Scan and maintenance helper table functions are internal-only; the user-facing surface is replacement scan, namespaces,
search functions, and thelancecopy format. - Improved parallelism for writing datasets.
- Refined test coverage and organization for sqllogictests.
- test: Add big ann tiny test by @Xuanwo in #50
- feat: Add basic lance write support by @Xuanwo in #53
- chore: Set dev panic=abort to avoid FFI unwind by @Xuanwo in #55
- docs: add DuckDB write examples by @Xuanwo in #56
- refactor: Support multi threads writing by @Xuanwo in #57
- feat: Add filter for timestamp/decimal/struct by @Xuanwo in #58
- feat: Add Lance Namespace support by @Xuanwo in #54
- feat: Add CREATE TABLE support by @Xuanwo in #64
- feat: Add INSERT INTO VALUES support by @Xuanwo in #65
- feat: Add DROP TABLE support by @Xuanwo in #66
- refactor: Always call lance namespace instead by @Xuanwo in #68
- feat: Add DELETE FROM support by @Xuanwo in #69
- feat: Add UPDATE support by @Xuanwo in #70
- feat: Add TRUNCATE TABLE support by @Xuanwo in #72
- refactor: Cleanup duplicated logic in DELETE by @Xuanwo in #73
- feat: Add pushdown support for LIMIT and OFFSET by @Xuanwo in #74
- feat: Add CREATE/DROP INDEX support by @Xuanwo in #76
- feat: Add ALTER TABLE ADD/DROP/ALTER COLUMN support by @Xuanwo in #77
- feat: Add LIKE pushdown support by @Xuanwo in #78
- feat: Add basic statistics support by @Xuanwo in #81
- feat: Refactor SHOW INDEXES by @Xuanwo in #82
- feat: Add take rowid supports by @Xuanwo in #85
- feat: Implement pushdown for regexp_matches by @Xuanwo in #91
- feat: Implement TABLESAMPLE SYSTEM for lance by @Xuanwo in #87
- feat: Add duckdb native rowid support by @Xuanwo in #93
- feat: Add IS (NOT) DISTINCT FROM pushdown support by @Xuanwo in #94
- refactor: Don't expose REINDEX as it's not part of duckdb SQL by @Xuanwo in #95
- docs: Add a full references to SQL by @Xuanwo in #97
- refactor: Never expose unneed table functions by @Xuanwo in #98
- ci: Refactor sqllogictests by @Xuanwo in #96
- Bump to version 0.3.0 by @Xuanwo in #99
Full Changelog: v0.2.0...v0.3.0
v0.2.0
Added
- Filter and projection pushdown for scans on Lance datasets.
- S3 authentication via DuckDB Secrets for
s3://...dataset paths. EXPLAIN (FORMAT JSON)diagnostics for Lance scans (bind-time and runtime plan details).lance_vector_search(path, vector_column, vector, ...)table function.lance_fts(path, text_column, query, ...)table function.lance_hybrid_search(path, vector_column, vector, text_column, query, ...)table function.
Changed
- Optimized
SELECT COUNT(*)on Lance datasets. - Improved error propagation across the Rust FFI boundary.
- Upgraded the Lance dependency to
v1.0.0(including Hugging Face backend support).
Detailed PRs
- feat: Support filter pushdown to lance by @Xuanwo in #24
- feat: Bump lance to version 1.0 by @Xuanwo in #23
- refactor: Handling ffi errors correctly by @Xuanwo in #25
- feat: Add last error API for ffi by @Xuanwo in #26
- docs: Add docs for FFI ownership by @Xuanwo in #27
- docs: Add docs for install from community by @Xuanwo in #28
- feat: Optimize for SELECT COUNT(*) by @Xuanwo in #29
- test: Add more push down support by @Xuanwo in #30
- feat: Enable huggingface support by @Xuanwo in #33
- docs: Polish README by @Xuanwo in #34
- test: Add TPC-H test cases by @Xuanwo in #32
- refactor: Build a simple filter expr ir to replace SQL by @Xuanwo in #38
- feat: Add duckdb secrets support by @Xuanwo in #37
- feat: Add big ann test cases by @Xuanwo in #39
- docs: Add section for formatting by @Xuanwo in #40
- feat: Add explain support for lance by @Xuanwo in #43
- feat: Add lance_search support for {vector,full-text,fuzzy} search by @Xuanwo in #45
- refactor: Re-org the cpp and rust code by @Xuanwo in #46
- chore: Cleanup old lance_knn functions by @Xuanwo in #47
- refactor: Redesign lance's search API by @Xuanwo in #48
- Bump to version 0.2.0 by @Xuanwo in #49
Full Changelog: v0.1.0...v0.2.0
v0.1.0
What's Changed
- feat: add basic read support and test by @Xuanwo in #1
- feat: Implement replacement rule correctly by @Xuanwo in #2
- refactor: Migrate to cpp based duckdb layout by @Xuanwo in #5
- feat: make
select * from x.lancework by @Xuanwo in #8 - docs: Polish README and AGENTS.md by @Xuanwo in #9
- ci: Remove not needed protoc wrapper by @Xuanwo in #11
- feat: Support duckdb 1.4.3 instead by @Xuanwo in #10
- refactor: Cleanup not used dead code by @Xuanwo in #14
- feat: Enable global runtime for duckdb by @Xuanwo in #16
- ci: Don't build parquet extension by @Xuanwo in #17
- ci: Enable mold for linking by @Xuanwo in #18
- ci: Include build config in to cache key by @Xuanwo in #19
- feat: Support concurrent executing at fragments level by @Xuanwo in #20
- chore: prepare v0.1.0 by @Xuanwo in #21
- ci: Try fix distribution by @Xuanwo in #22
New Contributors
Full Changelog: https://github.com/lance-format/lance-duckdb/commits/v0.1.0