You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 7313fdc
Browse filesBrowse the repository at this point in the historyBrowse files
| **INT8** (recommended) | 4x (75% savings) | Lower latency than NONE; the gain depends on the data and the recall target (see <<quantization-performance>>) | 95-98% | Best balance of speed and accuracy for most workloads
68
+
| **BINARY** | 32x (97% savings) | Lower latency than NONE; the gain depends on the data and the recall target (see <<quantization-performance>>) | 85-92% | Massive datasets, approximate search with reranking
69
69
| **PRODUCT** | 16-64x for the PQ codes in memory (the stored vectors are still read to re-rank) | Search walks on the PQ codes, then re-ranks on the stored vectors | Close to NONE with the default re-rank | Very large datasets (100K+), enables zero-disk-I/O graph construction. INT8 is the better default unless the PQ codes are what you need
70
70
|===
71
71
72
-
TIP: **Use INT8 quantization for most use cases.** It provides 4x memory savings with minimal accuracy loss and significantly faster search. Only use NONE for very small datasets where maximum precision matters.
72
+
TIP: **Use INT8 quantization for most use cases.** It provides 4x memory savings with minimal accuracy loss and usually lower search latency (see <<quantization-performance>>). Only use NONE for very small datasets where maximum precision matters.
73
73
74
74
[[quantization-performance]]
75
-
===== Why INT8 is faster
75
+
===== Why INT8 can be faster
76
76
77
-
Quantization doesn't just save memory -- it fundamentally changes how vectors are read during search.
77
+
Besides saving memory, quantization changes how vectors are read during search.
78
78
79
79
Without quantization (`NONE`), each node visited during graph traversal requires a full document lookup: read the record from disk, deserialize it, and extract the vector property. With INT8, vectors are stored in compact contiguous index pages and read directly -- no document deserialization needed.
80
80
81
-
In benchmarks with 500K 384-dimensional vectors (matching the `all-MiniLM-L6-v2` embedding model), INT8 reduces search latency by **2.5x** compared to NONE:
81
+
One benchmark with 500K 384-dimensional vectors (matching the `all-MiniLM-L6-v2` embedding model) measured the latencies below. Treat them as one setup, not a general figure. The gap between INT8 and NONE depends on the data, on whether the NONE vectors are served from memory or read from documents, and on the recall target, so measure it on your own data:
82
82
83
83
[cols="1,1,1,1"]
84
84
|===
@@ -88,7 +88,7 @@ In benchmarks with 500K 384-dimensional vectors (matching the `all-MiniLM-L6-v2`
The difference becomes even more significant under memory pressure. With NONE quantization, vector data is 4x larger, evicting more data from memory caches and forcing real disk I/O. INT8 keeps the working set small enough to stay in memory even with constrained resources.
91
+
The difference can be larger under memory pressure. With NONE quantization, vector data is 4x larger, evicting more data from memory caches and forcing real disk I/O. INT8 keeps the working set small enough to stay in memory even with constrained resources.
92
92
93
93
Quantization is transparent -- queries work identically regardless of quantization setting. The index automatically quantizes on insert and dequantizes on retrieval.
94
94
@@ -110,11 +110,11 @@ NOTE: A rebuild replaces the graph and its PQ codes a moment apart. Before v26.1
110
110
.Memory and performance example: 100K vectors with 384 dimensions
111
111
[cols="1,1,1"]
112
112
|===
113
-
| Quantization | Vector memory | Search speed
113
+
| Quantization | Vector memory | Search latency
114
114
115
115
| NONE | 156 MB | Baseline
116
-
| INT8 | 39 MB | ~2.5x faster
117
-
| BINARY | 5 MB | ~3x faster (lower recall)
116
+
| INT8 | 39 MB | Lower than NONE; depends on the data and the recall target
117
+
| BINARY | 5 MB | Lower than NONE, with lower recall; depends on the data and the rerank
118
118
|===
119
119
120
120
[[vector-encoding]]
@@ -205,7 +205,7 @@ To migrate an existing populated type to external storage in one step, use `ALTE
| `encoding` | `FLOAT32` | Wire / document-storage encoding: FLOAT32 (`ARRAY_OF_FLOATS` property) or INT8 (`BINARY` property, byte per dim, 4x smaller payload). See <<vector-encoding,Vector Encoding>>.
207
207
| `quantization` | `NONE` | Index-internal compression: NONE, INT8, BINARY, or PRODUCT. INT8 is recommended for most use cases.
208
-
| `efSearch` | adaptive | Search beam width at query time. Controls recall vs speed trade-off. See <<adaptive-efsearch>>.
208
+
| `efSearch` | sized from the graph | Search beam width at query time. Controls recall vs speed trade-off. See <<adaptive-efsearch>>.
209
209
| `rerankOversample` | 10 | `PRODUCT` only: candidates per requested neighbor that a search re-ranks on the stored vectors, which makes the returned distances exact. `0` returns the PQ scores alone, without reading a stored vector. A query overrides it with the `rerank` option of `vector.neighbors`. See <<product-quantization-search>>. Since 26.11.1.
210
210
| `pqTrainingLimit` | 128000 | `PRODUCT` only: vectors (a random sample) the PQ codebook is trained on, at most 128000.
211
211
| `fusedPQ` | false | `PRODUCT` only: store the PQ codes of each node's neighbors in the graph file (jvector FusedPQ). Makes the graph file several times larger; useful only when the PQ codes cannot stay in memory. Since 26.11.1.
@@ -222,8 +222,8 @@ To migrate an existing populated type to external storage in one step, use `ALTE
222
222
A rebuild of the graph is always a full one -- there is no incremental update, so any rebuild reindexes every
223
223
vector currently in the index, not just the ones written since the last build. `close()` normally rebuilds and
224
224
persists the graph before shutting down, so a later reopen finds a ready-to-search graph rather than rebuilding on
225
-
first open. For an index of 1,000 vectors or more (the same threshold `efSearch`'s adaptive strategy and
226
-
mutation-triggered rebuilds use), a session that closes with unpersisted writes pending instead defers that
225
+
first open. For an index of 1,000 vectors or more (the same threshold mutation-triggered
226
+
rebuilds use), a session that closes with unpersisted writes pending instead defers that
227
227
rebuild rather than blocking `close()` on it: the vectors themselves are already durably persisted independently
228
228
of the graph, so nothing is lost, but the graph's on-disk topology is left stale or absent. The deferred rebuild
229
229
then runs the next time that index is actually searched, after any reopen. A session that closes and never
@@ -441,18 +441,16 @@ so `CHECK DATABASE FIX` (or `REBUILD INDEX <name>`) rebuilds it and lifts the re
441
441
====
442
442
443
443
[[adaptive-efsearch]]
444
-
==== efSearch and Adaptive Search
444
+
==== efSearch and the Default Beam
445
445
446
-
The `efSearch` parameter controls how many candidate nodes the search explores in the vector graph. Higher values find more accurate results but take longer.
446
+
The `efSearch` parameter is the beam width of the graph search: how many candidate nodes the search keeps while it walks the vector graph. Higher values find more accurate results but take longer. The beam is never narrower than `k`.
447
447
448
-
When `efSearch` is not explicitly set (either on the index or per-query), ArcadeDB uses an **adaptive two-pass strategy**:
448
+
When `efSearch` is not set (neither on the index nor perquery), ArcadeDB sizes the beam from the graph and runs the search once:
449
449
450
-
1. **First pass** -- Uses a moderate beam width (`2 × k`), which is sufficient for most queries on well-clustered data.
451
-
2. **Second pass** -- If the first pass returns insufficient results, the search automatically widens the beam to `10 × k`.
450
+
1. For a graph under 10,000 nodes the beam is `max(k, 100)`.
451
+
2. For a larger graph the beam is `max(2 × k, 100)`. Before 26.11.1 it was `max(2 × k, 20)`, so a 10-neighbor query on a large graph walked a beam of 20 and lost recall.
452
452
453
-
For small indexes (< 10K vectors), the full default efSearch is always used since the cost is negligible.
454
-
455
-
This adaptive behavior gives you fast queries on easy lookups while still maintaining recall on harder queries -- without requiring any tuning.
453
+
There is no second pass that widens the beam when a search comes up short: a wider beam would visit the same nodes. A search that still returns fewer than `k` results is topped up by a brute-force scan instead. The default beam does not adapt to hard queries. If recall matters, set `efSearch` explicitly (see below): a larger value trades latency for recall.
456
454
457
455
===== Setting efSearch
458
456
@@ -483,9 +481,9 @@ CREATE INDEX ON Doc (embedding) LSM_VECTOR METADATA {
483
481
}
484
482
----
485
483
486
-
*Adaptive* (default) -- when neither per-query nor per-index efSearch is specified, the adaptive strategy described above is used.
484
+
*Default beam* -- when neither per-query nor per-index efSearch is specified, the beam is sized from the graph as described above.
487
485
488
-
TIP: For most workloads, the adaptive default works well. Only set efSearch explicitly if you need consistently high recall regardless of query difficulty, or if you have strict latency requirements.
486
+
TIP: The default beam suits most workloads. If recall matters, set efSearch explicitly and raise it until the recall you measure on your own queries is enough. Set it lower only if you have strict latency requirements. On a version before 26.11.1, the default beam of a graph of 10,000 nodes or more is narrow, so set efSearch there when recall matters.
Copy file name to clipboardExpand all lines: src/main/asciidoc/how-to/data-modeling/vector-embeddings.adoc
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -38,7 +38,7 @@ CREATE INDEX ON Document (embedding) LSM_VECTOR METADATA {
38
38
}
39
39
----
40
40
41
-
INT8 quantization is recommended for all production workloads. It provides 2.5x faster search and 4x lower memory usage with negligible accuracy loss (see <<quantization-performance>>). Only omit quantization for very small datasets (< 10K vectors) where maximum precision matters.
41
+
INT8 quantization is recommended for all production workloads. It provides 4x lower memory usage with negligible accuracy loss, and it reads vectors from index pages instead of documents, which usually lowers search latency; how much depends on the data and the recall target (see <<quantization-performance>>). Only omit quantization for very small datasets (< 10K vectors) where maximum precision matters.
42
42
43
43
Production-ready index with additional tuning:
44
44
@@ -66,7 +66,7 @@ CREATE INDEX ON Document (embedding) LSM_VECTOR METADATA {
66
66
67
67
===== Quantization Trade-offs
68
68
69
-
Use INT8 quantization for most use cases. It provides 4x memory savings with minimal accuracy loss and significantly faster ingestion and search:
69
+
Use INT8 quantization for most use cases. It provides 4x memory savings with minimal accuracy loss and usually lower search latency (see <<quantization-performance>>):
70
70
71
71
* **< 10K vectors**: `NONE` is fine, but `INT8` works well too
TIP: For large bulk loads, increase `mutationsBeforeRebuild` to delay index rebuilds until after the load completes, then trigger a rebuild.
307
+
TIP: For a large load through SQL or the record API, increase `mutationsBeforeRebuild` and `inactivityRebuildTimeoutMs` so the graph is not rebuilt part-way (a pause longer than `inactivityRebuildTimeoutMs`, 15 seconds by default, can trigger a rebuild whatever `mutationsBeforeRebuild` is), then run <<sql-rebuild-index,`REBUILD INDEX`>> once the load is finished, for example `+REBUILD INDEX `Document[embedding]`+`. `GraphBatch` loads need neither.
308
308
309
309
NOTE: Since v26.10.1 a bulk load through `GraphBatch` - which is what `GraphImporter`, the `arcadedb-integration`
310
310
importers and the HTTP batch endpoint use - needs none of that tuning. ArcadeDB holds the automatic graph rebuild off
Copy file name to clipboardExpand all lines: src/main/asciidoc/how-to/operations/performance-tuning.adoc
+8-1Lines changed: 8 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -126,7 +126,14 @@ difference between the two forms being several times apart in latency.
126
126
127
127
ArcadeDB distributes records across buckets for parallel access. By default a new type gets one bucket (the `arcadedb.typeDefaultBuckets` setting is `1`); set a higher value, or pass the number of buckets when creating the type, to get more.
128
128
129
-
For write-heavy types, add buckets to the type. `ALTER TYPE` adds and removes buckets by name (`+` adds, `-` removes), and a bucket that does not exist is created:
129
+
For write-heavy types, choose the number of buckets when you create the type, with `BUCKETS`. For asynchronous loads, use a count equal to, or a multiple of, the async parallel level (`arcadedb.asyncWorkerThreads`, cores minus 1 by default):
130
+
131
+
[source,sql]
132
+
----
133
+
CREATE DOCUMENT TYPE HighWriteType BUCKETS 8
134
+
----
135
+
136
+
You can also add a bucket to an existing type later, but only records created afterwards go into it, and existing records are not moved. `ALTER TYPE` adds and removes buckets by name (`+` adds, `-` removes), and a bucket that does not exist is created:
* `index-spec` - Index specification as `'TypeName[propertyName]'`
222
222
* `query-vector` - Query vector or key to look up
223
223
* `k` - Number of neighbors to return. Must be a non-negative integer: `0` returns no rows (like `LIMIT 0`), a negative or fractional value is rejected with an error. The same rule applies to the `k` of `vector.sparseNeighbors()`, `vector.discover()`, `vector.recommend()` and `vector.rerank()`. Since 26.11.1 (a negative `k` returned no rows before).
224
-
* `efSearch` - (optional, positional) Search beam width. Controls the trade-off between recall and speed. Higher values improve recall but increase latency. When omitted, the index uses <<adaptive-efsearch,adaptive efSearch>>.
224
+
* `efSearch` - (optional, positional) Search beam width. Controls the trade-off between recall and speed. Higher values improve recall but increase latency. When omitted, the index uses its <<adaptive-efsearch,default beam>>, sized from the graph.
225
225
226
226
*Options map* (optional, alternative to the positional `efSearch` argument). An option of type Integer accepts a whole number only (`2` or `2.0`): a fractional value such as `2.5` is rejected with an error instead of being truncated, since 26.11.1.
227
227
@@ -249,7 +249,7 @@ Cypher `SEARCH` clause, which is tested by the index the same way.
Create a record in async way. Once the record is created in the database, the callback will be executed.
358
+
Create a record in async way. Once the record is applied to the worker's current batch, the callback will be executed. This is not yet durable: if a later operation in the same batch fails, the whole batch rolls back, and the record with it, even though the callback already fired. For a signal that fires only once the batch has committed, register <<asynconok,`onOk()`>> on the executor or call `waitCompletion()`.
359
359
This method returns immediately.
360
360
The result can be managed in the NewRecordCallback callback and errors in ErrorCallback callback.
Updates a record in async way. Once the record is updated in the database, the callback will be executed.
388
+
Updates a record in async way. Once the update is applied to the worker's current batch, the callback will be executed. This is not yet durable: if a later operation in the same batch fails, the whole batch rolls back, and the update with it, even though the callback already fired. For a signal that fires only once the batch has committed, register <<asynconok,`onOk()`>> on the executor or call `waitCompletion()`.
389
389
This method returns immediately.
390
390
The result can be managed in the UpdatedRecordCallback callback and errors in ErrorCallback callback.
Deletes a record in async way. Once the record is deleted from the database, the callback will be executed.
417
+
Deletes a record in async way. Once the delete is applied to the worker's current batch, the callback will be executed. This is not yet durable: if a later operation in the same batch fails, the whole batch rolls back, and the delete with it, even though the callback already fired. For a signal that fires only once the batch has committed, register <<asynconok,`onOk()`>> on the executor or call `waitCompletion()`.
418
418
This method returns immediately.
419
419
The result can be managed in the DeletedRecordCallback callback and errors in ErrorCallback callback.
420
420
@@ -437,3 +437,31 @@ Example on inserting a vertex asynchronously.
Registers a global callback that fires every time a worker's batch transaction has committed: at each `setCommitEvery()` boundary, when a durability setting (<<asyncdurability,`setTransactionUseWAL()` and `setTransactionSync()`>>) changes mid-batch, and when `waitCompletion()` commits the pending batches. This is the durability signal for `createRecord()`, `updateRecord()` and `deleteRecord()`: their own callbacks fire as soon as the write is applied to the worker's current batch, which a later failure in the same batch can still roll back.
v -> { System.out.println("Record " + v.toJSON() + " applied, not yet durable"); });
462
+
database.async().waitCompletion();
463
+
```
464
+
465
+
The callback is not tied to one record: it tells you a batch has committed, not which records were in it. Each worker commits on its own, so the callback runs on the committing worker's thread, possibly from several threads at once. It must be thread-safe and cheap, because a slow callback slows that worker down. A worker also calls it on shutdown even when it had nothing left to commit, so a call at shutdown is not proof that a particular write became durable at that moment.
466
+
467
+
NOTE: "Committed" here means durable in the write-ahead log and safe to replay on the next open. With the default `setTransactionSync(NO)` there is no `fsync` at that point, so the write can still be lost to a power loss or an operating system crash (not to a JVM crash or restart). Set `setTransactionSync(YES_NOMETADATA)` or `YES_FULL` if you need that guarantee; the timing of this callback does not change with it.
0 commit comments