Hi team,
I noticed some ambiguity in the documentation regarding the parallel configuration parameters, which could lead to confusion when setting up performance-related options.
Specifically:
From my observations, SearchParams::num_threads appears to control inter-query parallelism (i.e., parallel processing across multiple queries). However, it’s unclear what IndexBuildParams::num_workers represents. For example:
- Does num_workers control intra-query parallelism (parallelizing a single index build task), or another type of parallelism?
- Is there any interaction between num_threads and num_workers? For instance, do their values multiply to determine total threads/workers used, or are they independent?
- When searching, under condition that numa is disabled (
QUAKE_USE_NUMA=OFF) and inter-query parallelism is disabled (num_threads=1), if I want to use two cores for intra-query parallelism during search, should num_workers be set to 1 or 2?
- When building, if I want to use only two cores for the build process, how should I configure it?
Could we add explicit explanations in the code comments (for both parameters) and update the README to clarify?
This would help users avoid misconfigurations and better optimize parallel performance.
Thanks!
Hi team,
I noticed some ambiguity in the documentation regarding the parallel configuration parameters, which could lead to confusion when setting up performance-related options.
Specifically:
From my observations, SearchParams::num_threads appears to control inter-query parallelism (i.e., parallel processing across multiple queries). However, it’s unclear what IndexBuildParams::num_workers represents. For example:
QUAKE_USE_NUMA=OFF) and inter-query parallelism is disabled (num_threads=1), if I want to use two cores for intra-query parallelism during search, shouldnum_workersbe set to 1 or 2?Could we add explicit explanations in the code comments (for both parameters) and update the README to clarify?
This would help users avoid misconfigurations and better optimize parallel performance.
Thanks!