-
Notifications
You must be signed in to change notification settings - Fork 747
Open
Description
Summary
Currently, leann list scans directories to discover indexes, which can be slow for large directory trees. A global registry system would make index discovery O(1) by tracking all index paths centrally.
Proposed Solution
Implement a global registry at ~/.leann/indexes.json that stores the paths of all LEANN indexes:
{
"indexes": [
{
"name": "my-docs",
"path": "/Users/foo/projects/my-project/.leann/indexes/my-docs",
"type": "cli",
"created_at": "2024-12-25T10:00:00Z"
},
{
"name": "app-index",
"path": "/Users/foo/apps/data/app-index.leann",
"type": "app",
"created_at": "2024-12-25T11:00:00Z"
}
]
}Implementation Steps
- Register on build: When
leann buildcreates an index, automatically register it in the global registry - Register on API usage: When apps create indexes via the Python API, register them as well
- Update
leann list: Read from registry instead of scanning directories - Cleanup stale entries: Validate paths exist and remove stale entries periodically
- Migration: First run should scan existing indexes and populate the registry
Benefits
leann listbecomes O(1) instead of O(n) directory scan- Works correctly even when running from
$HOMEor other large directories - Provides a single source of truth for all LEANN indexes
Related
- Follow-up from PR fix: limit depth of index discovery to prevent slow scans #196 (reviewer suggestion by @andylizf)
- Fixes leann list scans all the things #122 (long-term solution)
Metadata
Metadata
Assignees
Labels
No labels