Initial memtable-based implementation - #1

Merged

tjgreen42 merged 78 commits into

tj-initial-implementation

Sep 13, 2025

Commits on Sep 5, 2025

Initial memtable-based implementation
Todd J. Green
committed
55a96bb
PR feedback; give Claude Code access to PRs
Todd J. Green
committed
1b49139
Address more PR feedback
Todd J. Green
committed
5a4c0e1
Fix critical posting list storage and retrieval bug
Todd J. Green
committed
e39283c
Address Sven's LIMIT pushdown safety concern
Todd J. Green
committed
e210885
Fix sanitizer crashes in LIMIT pushdown implementation
Todd J. Green
committed
6f0a441
Add transaction state safety check to query limit cleanup
Todd J. Green
committed
20cc9c6
Fix sanitizer test failures by correcting log levels
Todd J. Green
committed
a15715d
Temporarily disable LIMIT pushdown to debug sanitizer BM25 issues
Todd J. Green
committed
946b3c8
Fix BM25 type consistency between metapage and runtime calculations
Todd J. Green
committed
c592497
Replace avg_doc_length with total_len approach for precision stability
Todd J. Green
committed
935c486
Implement comprehensive BM25 precision fixes for sanitizer stability
Todd J. Green
committed
d65baf1
Fix division by zero in BM25 calculations
Todd J. Green
committed
74e9afd
Fix BM25 scoring precision in sanitizer builds
Todd J. Green
committed
e49bdf9

Commits on Sep 6, 2025

Add comprehensive debug logging to trace BM25 calculation differences in sanitizer
Todd J. Green
committed
8538356
Add debug logging to track query term frequencies in to_tpvector
Todd J. Green
committed
b97bbde
Update test expected values to match correct BM25 calculations
Todd J. Green
committed
6230cdc
Fix remaining test value differences for sanitizer build
Todd J. Green
committed
8d2ecfe
Fix final test value differences in strings and limits tests
Todd J. Green
committed
fdfe23f
Fix limits test for sanitizer environment
Todd J. Green
committed
1d1a99e
Remove variable LOG duration times from limits test
Todd J. Green
committed
0cc3fa9
Fix limits test by disabling log_duration early
Todd J. Green
committed
fd415b2
Re-enable LIMIT optimization after fixing tests
Todd J. Green
committed
65ba508
Disable LIMIT optimization to prevent sanitizer crash
Todd J. Green
committed
ab37e3e
Fix posting list rebuilding and memory allocation issues
Todd J. Green
committed
b09d5ea
Fix use-after-poison sanitizer error in posting.c
Todd J. Green
committed
715e8ae

Commits on Sep 7, 2025

Fix undefined behavior in posting list rebuild
Todd J. Green
committed
337188d
Fix sanitizer test failures and update CI to test PG 15, 16, 17
Todd J. Green
committed
d6046b7
Fix PostgreSQL 15/16 compatibility for hash functions
Todd J. Green
committed
56e0e45
Fix CI workflow to trigger on master branch instead of main
Todd J. Green
committed
483988c
Fix PostgreSQL install path in CI
Todd J. Green
committed
dc141cc
Fix expected test output for hash table scan fix and PG version differences
Todd J. Green
committed
48bfecd
Fix another test warning in alternate expected output
Todd J. Green
committed
099b218
Remove memory_limits.sh from CI shell tests
Todd J. Green
committed
819a0fa
Swap limits expected output files for version compatibility
Todd J. Green
committed
3829fca
Revert swap - use original file order for limits test
Todd J. Green
committed
28ce677
Remove EXPLAIN tests that vary by PostgreSQL version
Todd J. Green
committed
f013d51
Fix blank line differences in expected test output
Todd J. Green
committed
7ad070e
Restrict sanitizer tests to PG17 only
Todd J. Green
committed
ecc7586

Commits on Sep 10, 2025

Fix DSA persistence issues and add BM25 validation tools

Todd J. Green
and
claude
committed
74c94e9
Fix heap-use-after-free in tp_destroy_index_dsa
Todd J. Green
committed
1858cda
Fix heap-use-after-free by using TopMemoryContext for DSA control structures
Todd J. Green
committed
ac55e58
Fix DSM segment size error by adding cache check in cleanup
Todd J. Green
committed
7bd8f5d

Commits on Sep 11, 2025

Fix BM25 scoring and DSM/DSA persistence issues
Todd J. Green
committed
ff587ae
Clean up debugging scripts
Todd J. Green
committed
301fdc5
Limit support to PostgreSQL 17+
Todd J. Green
committed
f8846ce
Remove pointer address from DEBUG output
Todd J. Green
committed
ce86af6
Fix test stability: remove pointer addresses from debug output
Todd J. Green
committed
c1ae9a6
Fix limits test expected output for CI
Todd J. Green
committed
1648627
Fix limits test for CI environment compatibility
Todd J. Green
committed
6419e4e
Make limits test resilient to CI environment differences
Todd J. Green
committed
069aaca
Suppress LOG statements in CI for limits test
Todd J. Green
committed
818a7d7
Simplify limits.sql test for better reliability
Todd J. Green
committed
cb4d660
Fix limits test and implement progress reporting

Todd J. Green
and
claude
committed
6c3b9d9
Fix hash sequence scan registration/deregistration mismatch
Todd J. Green
committed
c13c372
Clean up TODO items in memtable.c
Todd J. Green
committed
615a324
Modernize header files to use #pragma once
Todd J. Green
committed
214f326

Commits on Sep 12, 2025

Clean up all TODO items and unused code in posting.c
Todd J. Green
committed
5674ff6
Remove tp_posting_init_shared_state no-op and clean up historical comments
Todd J. Green
committed
cdcc2c8
Refactor stringtable to use PostgreSQL's dshash for improved concurrency
Todd J. Green
committed
a3b69f4
Implement zero-allocation string lookups with variant wrapper approach
Todd J. Green
committed
7c1b424
Refactor metapage functionality into separate module for better code organization

Todd J. Green
and
claude
committed
f86552e
Implement PostgreSQL-style header organization with clang-format
Todd J. Green
committed
c047841
Test pre-commit hook
Todd J. Green
committed
a97ca01
Simplify code formatting and add pre-commit setup
Todd J. Green
committed
d698846
Fix clang-format language setting to C (not Cpp)
Todd J. Green
committed
3167366
Fix .clang-format for older clang-format versions
Todd J. Green
committed
54faddc
Simplify and improve sanitizer workflow
Todd J. Green
committed
9fa418f
Fix include formatting: treat self-includes like other local headers
Todd J. Green
committed
419e985
Refactor code organization and improve TpVector structure
Todd J. Green
committed
86af433
Clean up paranoid checks in index.c and fix sanitizer CI
Todd J. Green
committed
0b5a035
Fix function name in warning message and clean up comment
Todd J. Green
committed
c6a75cc
Simplify LIMIT implementation by replacing hash table with simple structure

Todd J. Green
and
claude
committed
c961fcf
Disable memory leak detection in sanitizer CI to avoid PostgreSQL core leaks

Todd J. Green
and
claude
committed
1c8ca60
Remove all debug messages from codebase for cleaner production code

Todd J. Green
and
claude
committed
866345c
Refactor code organization and remove unnecessary abstractions
Todd J. Green
committed
b7d839c
Enable unused code analysis and fix all warnings
Todd J. Green
committed
ddb6e91
Enable unused code analysis and fix all warnings
Todd J. Green
committed
002d500