LEANN search latency problem #182
Description
This is the log of running claude code, it can run stably, but it do have some problems of the embedding server start problem, which we might need to solve. cc @andylizf @ASuresh0524
https://github.com/yichuan-w/LEANN/blob/main/packages/leann-mcp/README.md
Here is the log
(vllm) ➜ vllm git:(main) ✗ claude
> /mcp
⎿ MCP dialog dismissed
> please tell me whats the page size in vllm use leann search to localize
leann-server - leann_search (MCP)(index_name: "my-repo", query: "page size configuration block size", top_k: 10, show_metadata: true, complexity: 32)
⎿ ⚠️ Embedding server failed: Failed to compute embeddings via server: Resource temporarily unavailable
⏭️ Falling back to direct model loading...
Search results for 'page size configuration block size' (top 10):
… +139 lines (ctrl+o to expand)
● Great! I found several relevant files. Let me read the key files to understand the page size configuration:
● Read(vllm/v1/core/kv_cache_utils.py)
⎿ Read 1473 lines
● Search(pattern: "block_size|page_size|BLOCK_SIZE", path: "vllm", type: "py", output_mode: "content", head_limit: 30)
⎿ Found 30 lines (ctrl+o to expand)
● Let me search for more specific information about the default page size value:
● leann-server - leann_search (MCP)(index_name: "my-repo", query: "default page size block size 16 configuration cache config", top_k: 10, show_metadata: true, complexity: 32)
⎿ ⚠️ Embedding server failed: Failed to compute embeddings via server: Resource temporarily unavailable
⏭️ Falling back to direct model loading...
Search results for 'default page size block size 16 configuration cache config' (top 10):
… +138 lines (ctrl+o to expand)