Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 19 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,9 +129,25 @@ generic plan, `DEALLOCATE` and prepare the statement again. A newly planned
query can choose the correct sequential fallback, while the cached plan is
rejected to avoid incorrect index results.

Boolean filtering and BM25 ranking are separate scan modes. A query combining
`WHERE content @@ ...` with `ORDER BY content <@> ...` cannot use one BM25
index scan for both operations.
Boolean filtering can be combined with BM25 ranking in one index scan:

```sql
SELECT * FROM documents
WHERE content @@ to_tsquery('english', 'postgres & !mysql')
ORDER BY content <@> 'database system'
LIMIT 5;
```

The index produces candidates in BM25 order. If many candidates fail the
Boolean predicate, it materializes the Boolean matches once. When their CTID
lookup fits within `work_mem`, later ranked candidates are filtered in the
index before PostgreSQL reads their table rows; otherwise filtering continues
through PostgreSQL's heap recheck. Phrase, prefix, and weight conditions may
still require heap rechecks for exact `tsquery` semantics.

For indexes with 100,000 or more documents, PostgreSQL uses a full scan and sort
for combined Boolean ranking so matches beyond the bounded ranking window
cannot be assigned an incorrect zero score.

### Verifying Index Usage

Expand Down
15 changes: 13 additions & 2 deletions src/access/am.h
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,18 @@ typedef struct TpScanOpaqueData
bool is_boolean_scan;
bool boolean_recheck;
BufFile *boolean_results;
Oid index_oid; /* Index OID */

/*
* Combined Boolean + ranked scans evaluate the Boolean query once.
* boolean_matches keeps every matching CTID for the zero-score tail, and
* boolean_matched_ctids rejects ranked candidates before PostgreSQL
* fetches their heap tuples. The lookup set stays NULL when the match
* set does not fit in work_mem.
*/
BufFile *boolean_matches;
struct HTAB *boolean_matched_ctids;

Oid index_oid; /* Index OID */

/* Scan results state */
ItemPointer result_ctids; /* Array of matching CTIDs */
Expand All @@ -47,7 +58,7 @@ typedef struct TpScanOpaqueData
int limit; /* Query LIMIT value, -1 if none */
int max_results_used; /* Internal limit used for current batch */

/* CTIDs already emitted; used across limit-doubling re-execs. */
/* Ranked CTIDs already emitted; bounded by TP_MAX_QUERY_LIMIT. */
struct HTAB *returned_ctids;
} TpScanOpaqueData;

Expand Down
Loading
Loading