Skip to content

feat: add indexing queue support for sequential codebase processing #181 - #208

Merged
Aharshi3614 merged 3 commits into
Aharshi3614:mainfrom
Aryanbuha890:issue-181-indexing-queue
Aug 4, 2026
Merged

feat: add indexing queue support for sequential codebase processing #181#208
Aharshi3614 merged 3 commits into
Aharshi3614:mainfrom
Aryanbuha890:issue-181-indexing-queue

Conversation

@Aryanbuha890

Copy link
Copy Markdown
Contributor

Summary

This PR implements an indexing queue system to sequentially process codebase uploads and re-indexing runs instead of rejecting concurrent requests, closing #181.

Changes

  • Backend Indexing Queue:
    • Implemented indexing_queue and jobs_history in main.py.
    • Added a background worker thread (queue_worker) running sequentially. It handles temp ZIP storage using unique names (temp_upload_<job_id>.zip to prevent overwrite conflicts), path-traversal validation, file checks, and executes index_directory sequentially.
    • Handled failed jobs gracefully (logs errors, updates job status to failed, and continues to the next job in queue).
    • Exposed a GET /index/queue endpoint.
  • Queue Status UI:
    • Updated SettingsPanel.jsx to poll /index/queue every 3 seconds while open.
    • Displayed a queued jobs list displaying job names and status tags (PENDING, RUNNING, COMPLETED, FAILED with error text).
  • Unit Tests:
    • Updated existing tests in test_upload.py to match queueing expectations.
    • Added tests/test_queue.py verifying queue retrieval, concurrent job queueing, and graceful error handling.

Closes #181

@Aharshi3614
Aharshi3614 merged commit 226f766 into Aharshi3614:main Aug 4, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Indexing Queue Support

2 participants