Repository navigation
Match the vector, bucket and async pages to the engine (#499) - #508
Merged
robfrank merged 2 commits intoOct 28, 2026
Merged
Conversation
…ata#499) - Quantization speed: drop the unmeasured 10-15%, 15-20%, 2.5x and ~3x claims from the vector pages and say the effect depends on the data and the recall target. - efSearch: describe the single-pass default beam (max(k, 100) under 10,000 nodes, max(2 x k, 20) above) and drop the second pass and "without requiring any tuning". - Bucket configuration: lead with BUCKETS at CREATE TYPE; ALTER TYPE only helps records created afterwards. - Async createRecord/updateRecord/deleteRecord: the callback fires when the write is applied to the worker's batch, not when it is durable; document onOk(). - vector-embeddings: the bulk-load tip names REBUILD INDEX and inactivityRebuildTimeoutMs. Fixes ArcadeData#499 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Contributor
|
This pull request does not currently match the merge queue conditions, so it cannot be queued from here. The box comes back if it matches again. |
…rge graph since 26.11.1 Engine commit fb33b26814 (ArcadeData/arcadedb#10704) raised the floor from 20 to 100, so the large-graph default is no longer narrow. Also drops an unverified claim about adding buckets mid-load and plainer wording in the quantization benchmark intro. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #499. Items 1, 2, 3, 5 and 6 are addressed here; item 4 was already fixed on main (docs commits 56669e1 and c53819e, after the 26.11.1 PRODUCT search) and is left alone. Line numbers below are the ones on main 6024918.
1. Quantization speed claims. We have no measured figure to publish, so no number is added.
concepts/vector-search.adoc(the INT8 and BINARY cells of the quantization table, the TIP below it, the 100K x 384 example table) andhow-to/data-modeling/vector-embeddings.adoc(L41, L69) now say that latency is lower than NONE and that the gain depends on the data and the recall target, and linkquantization-performance, the section that explains the mechanism. That section is renamed "Why INT8 can be faster" (same anchor). I kept its 500K x 384 latency table, because it is the page's own measurement, but it is now introduced as one setup rather than a general 2.5x; if you would rather drop the table, say so. The same claims were onreference/java-api/java-vectors.adoc(L41, L86, the comparison table, the INT8 bullet list), so I changed them there too. The DOT_PRODUCT "10-15% faster than COSINE" at vector-search L52 is a different claim and is untouched.2. efSearch. The "adaptive two-pass strategy" section is now "efSearch and the Default Beam" (anchor
adaptive-efsearchkept). It says the search runs once with a beam ofmax(k, 100)under 10,000 nodes andmax(2 x k, 20)above, that the larger-graph beam is deliberately narrow, that there is no second pass, that a short result is topped up by the brute-force scan, and thatefSearchis how you ask for more recall. This matchesLSMVectorIndex.searchNeighborson main (the comment at L8635 explains why the second pass was removed). The "Adaptive (default)" paragraph and TIP further down, the settings-table default, the 1,000-vector remark at L225 (which cited the adaptive strategy's threshold),reference/extended-functions/vector.adocand the vector tutorial no longer call the default adaptive. The default itself is discussed in ArcadeData/arcadedb#10704.3. Bucket configuration (
how-to/operations/performance-tuning.adoc). Leads withCREATE DOCUMENT TYPE HighWriteType BUCKETS 8and a count equal to, or a multiple of,arcadedb.asyncWorkerThreads.ALTER TYPE ... BUCKET +...follows, with the note that only records created afterwards use the new bucket and existing records are not moved.5. Async record callbacks (
reference/java-api/java-ref-database-async.adoc).createRecord,updateRecordanddeleteRecordnow say the callback fires once the write is applied to the worker's current batch, that this is not yet durable (a later failure in the same batch rolls the whole batch back), and point toonOk()andwaitCompletion(). A new section documentsonOk(OkCallback)fromDatabaseAsyncExecutoron main: it fires when a worker's batch commits (thesetCommitEveryboundary, a durability setting change,waitCompletion()), not per record; it runs on the committing worker's thread and must be thread-safe and cheap; and "committed" means WAL-durable, with nofsyncunlesssetTransactionSyncisYES_NOMETADATAorYES_FULL.6. Vector bulk-load tip (
vector-embeddings.adoc, L307). NamesmutationsBeforeRebuildandinactivityRebuildTimeoutMs, says a pause longer than the timeout can rebuild regardless, and ends withREBUILD INDEX(linked to its reference, with the backtick-quoted index nameDocument[embedding]as that page requires).GraphBatchloads need neither, as the NOTE below already says.Checked locally.
npx -y asciidoctor@3.0.4 --failure-level WARN -o /dev/null <page>is clean for the six pages that are complete documents.java-ref-database-async.adocis a fragment of the book (its first heading is already level 5 under a level 3 title), so it warns "section title out of sequence" for every======heading on main as well (11 warnings on main, 12 here, one for the newonOkheading, the same convention as its neighbours). No other warning. I did not run the Antora build.🤖 Generated with Claude Code