Skip to content

Match the vector, bucket and async pages to the engine (#499) - #508

Merged
robfrank merged 2 commits into
ArcadeData:mainfrom
tae898:docs/499-vector-bucket-async-wording
Oct 28, 2026
Merged

robfrank merged 2 commits into
ArcadeData:mainfrom
tae898:docs/499-vector-bucket-async-wording

Conversation

@tae898

@tae898 tae898 commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Fixes #499. Items 1, 2, 3, 5 and 6 are addressed here; item 4 was already fixed on main (docs commits 56669e1 and c53819e, after the 26.11.1 PRODUCT search) and is left alone. Line numbers below are the ones on main 6024918.

1. Quantization speed claims. We have no measured figure to publish, so no number is added. concepts/vector-search.adoc (the INT8 and BINARY cells of the quantization table, the TIP below it, the 100K x 384 example table) and how-to/data-modeling/vector-embeddings.adoc (L41, L69) now say that latency is lower than NONE and that the gain depends on the data and the recall target, and link quantization-performance, the section that explains the mechanism. That section is renamed "Why INT8 can be faster" (same anchor). I kept its 500K x 384 latency table, because it is the page's own measurement, but it is now introduced as one setup rather than a general 2.5x; if you would rather drop the table, say so. The same claims were on reference/java-api/java-vectors.adoc (L41, L86, the comparison table, the INT8 bullet list), so I changed them there too. The DOT_PRODUCT "10-15% faster than COSINE" at vector-search L52 is a different claim and is untouched.

2. efSearch. The "adaptive two-pass strategy" section is now "efSearch and the Default Beam" (anchor adaptive-efsearch kept). It says the search runs once with a beam of max(k, 100) under 10,000 nodes and max(2 x k, 20) above, that the larger-graph beam is deliberately narrow, that there is no second pass, that a short result is topped up by the brute-force scan, and that efSearch is how you ask for more recall. This matches LSMVectorIndex.searchNeighbors on main (the comment at L8635 explains why the second pass was removed). The "Adaptive (default)" paragraph and TIP further down, the settings-table default, the 1,000-vector remark at L225 (which cited the adaptive strategy's threshold), reference/extended-functions/vector.adoc and the vector tutorial no longer call the default adaptive. The default itself is discussed in ArcadeData/arcadedb#10704.

3. Bucket configuration (how-to/operations/performance-tuning.adoc). Leads with CREATE DOCUMENT TYPE HighWriteType BUCKETS 8 and a count equal to, or a multiple of, arcadedb.asyncWorkerThreads. ALTER TYPE ... BUCKET +... follows, with the note that only records created afterwards use the new bucket and existing records are not moved.

5. Async record callbacks (reference/java-api/java-ref-database-async.adoc). createRecord, updateRecord and deleteRecord now say the callback fires once the write is applied to the worker's current batch, that this is not yet durable (a later failure in the same batch rolls the whole batch back), and point to onOk() and waitCompletion(). A new section documents onOk(OkCallback) from DatabaseAsyncExecutor on main: it fires when a worker's batch commits (the setCommitEvery boundary, a durability setting change, waitCompletion()), not per record; it runs on the committing worker's thread and must be thread-safe and cheap; and "committed" means WAL-durable, with no fsync unless setTransactionSync is YES_NOMETADATA or YES_FULL.

6. Vector bulk-load tip (vector-embeddings.adoc, L307). Names mutationsBeforeRebuild and inactivityRebuildTimeoutMs, says a pause longer than the timeout can rebuild regardless, and ends with REBUILD INDEX (linked to its reference, with the backtick-quoted index name Document[embedding] as that page requires). GraphBatch loads need neither, as the NOTE below already says.

Checked locally. npx -y asciidoctor@3.0.4 --failure-level WARN -o /dev/null <page> is clean for the six pages that are complete documents. java-ref-database-async.adoc is a fragment of the book (its first heading is already level 5 under a level 3 title), so it warns "section title out of sequence" for every ====== heading on main as well (11 warnings on main, 12 here, one for the new onOk heading, the same convention as its neighbours). No other warning. I did not run the Antora build.

🤖 Generated with Claude Code

…ata#499)

- Quantization speed: drop the unmeasured 10-15%, 15-20%, 2.5x and ~3x claims from the vector pages and say the effect depends on the data and the recall target.
- efSearch: describe the single-pass default beam (max(k, 100) under 10,000 nodes, max(2 x k, 20) above) and drop the second pass and "without requiring any tuning".
- Bucket configuration: lead with BUCKETS at CREATE TYPE; ALTER TYPE only helps records created afterwards.
- Async createRecord/updateRecord/deleteRecord: the callback fires when the write is applied to the worker's batch, not when it is durable; document onOk().
- vector-embeddings: the bulk-load tip names REBUILD INDEX and inactivityRebuildTimeoutMs.

Fixes ArcadeData#499

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@mergify

mergify Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

This pull request does not currently match the merge queue conditions, so it cannot be queued from here. The box comes back if it matches again.

…rge graph since 26.11.1

Engine commit fb33b26814 (ArcadeData/arcadedb#10704) raised the floor
from 20 to 100, so the large-graph default is no longer narrow. Also
drops an unverified claim about adding buckets mid-load and plainer
wording in the quantization benchmark intro.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@robfrank
robfrank merged commit 7313fdc into ArcadeData:main Oct 28, 2026
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Docs: six stale or contradictory statements in vector-search, vector-embeddings, performance-tuning and the async reference

2 participants