How should the parameters be configured to achieve a smaller search time?[Solved]-> warmup-question remained

[Ai-yang-dev]

Current configuration：

GPU：4090×1
embedding_model：BAAI/bge-large-zh-v1.5
data size：180M
beam_width：10

The remaining parameters are all set to their default values.

But, I have conducted an experiment and found that the search takes approximately 15 to 30 seconds. I'm not sure where I went wrong and would appreciate any help in resolving this issue. @yichuan-w

[Ai-yang-dev]

This is my operation log.

[yichuan-w]

data size：180M
You mean 180MB text?

[Ai-yang-dev]

data size：180M You mean 180MB text?

yes，text size about 180M，and Leann Builder's parameters is (backend_name="hnsw", embedding_model="BAAI/bge-large-zh-v1.5")

[yichuan-w]

Interesting, it should not be that slow, one thing is that in our paper we use a smaller model, which is around 100M, but bge-large is around 300M, but it should still finish within 5 seconds, I think
And you use all the default args, right?

[yichuan-w]

Can you share the script to reproduce this? Or even the data, if you want, I will investigate

[yichuan-w]

This is an emergency performance issue; we need to figure it out ASAP @andylizf @ASuresh0524 @yichuan-w

[Ai-yang-dev]

Interesting, it should not be that slow, one thing is that in our paper we use a smaller model, which is around 100M, but bge-large is around 300M, but it should still finish within 5 seconds, I think And you use all the default args, right?

Will chunks count have any impact on the search speed?

[Ai-yang-dev]

Can you share the script to reproduce this? Or even the data, if you want, I will investigate

test.py
This is my test script.

The dataset uses CRUD-RAG，This is an open-source dataset.（https://github.com/IAAR-Shanghai/CRUD_RAG/tree/main/data/80000_docs）

[yichuan-w]

Will chunks count have any impact on the search speed?
90K is okay, not that large

I will try to reproduce tmrr, and a quick fix is that you can use a smaller embedding model or switch to our diskann backend, which is much faster, but i would recomend use the latter.

[Ai-yang-dev]

Will chunks count have any impact on the search speed?
90K is okay, not that large

I will try to reproduce tmrr, and a quick fix is that you can use a smaller embedding model or switch to our diskann backend, which is much faster, but i would recomend use the latter.

There are also reasons related to the project. Currently, we are using hnsw for search in es. We do not wish to change this configuration for the time being. Due to the large size of our library, we hope to use LEANN to alleviate the storage pressure.

[Ai-yang-dev]

Will chunks count have any impact on the search speed?
90K is okay, not that large

I will try to reproduce tmrr, and a quick fix is that you can use a smaller embedding model or switch to our diskann backend, which is much faster, but i would recomend use the latter.

Could you please check if your system can reproduce the same performance issue as mine? It's important. @yichuan-w

[yichuan-w]

I am reproducing, btw, can you try to use
results = searcher.search(test_query, top_k=10, complexity=32)

This is my initial test result