Skip to content

Task queue persist dead letter queue to file and expose /api/tasks/dead letter endpoint#421

Open
clintjeff2 wants to merge 2 commits into
Bitcoindefi:mainfrom
clintjeff2:Task-queue-persist-dead-letter-queue-to-file-and-expose-/api/tasks/dead-letter-endpoint-#192
Open

Task queue persist dead letter queue to file and expose /api/tasks/dead letter endpoint#421
clintjeff2 wants to merge 2 commits into
Bitcoindefi:mainfrom
clintjeff2:Task-queue-persist-dead-letter-queue-to-file-and-expose-/api/tasks/dead-letter-endpoint-#192

Conversation

@clintjeff2

Copy link
Copy Markdown
Contributor

Motivation

  • Make the task dead-letter queue (DLQ) durable so failed tasks survive process restarts for debugging and recovery.
  • Provide simple HTTP endpoints to inspect, retry, and discard DLQ entries.
  • Ensure DLQ writes are atomic to avoid partial/corrupt files.

Description

  • Added DLQ persistence to lib/agent-runtime/task-queue.ts including a DeadLetterTaskEntry shape, persistDeadLetterQueue() and loadDeadLetterQueue() helpers, and support for overriding the path via TASK_DLQ_FILE (default /.data/task-dlq.json).
  • Persist DLQ on state changes and hydrate the in-memory queue at startup by calling loadDeadLetterQueue(); writes use a temp file + rename for atomic updates.
  • Exposed a paginated listing endpoint GET /api/tasks/dead-letter that returns entries and pagination metadata via app/api/tasks/dead-letter/route.ts.
  • Implemented POST /api/tasks/dead-letter/[id]/retry to move a DLQ task back to pending and DELETE /api/tasks/dead-letter/[id] to permanently discard an entry via app/api/tasks/dead-letter/[id]/retry/route.ts and app/api/tasks/dead-letter/[id]/route.ts.
  • Added listDeadLetterEntries() and discardDeadLetterTask() helpers and wired persistence calls into failTask() and retryDeadLetterTask().
  • Extended __tests__/api/tasks.test.ts to set up a temp TASK_DLQ_FILE, verify file persistence/reload, and exercise the new list/retry/delete endpoints.

Testing

  • Ran the focused tests for the change with npm test -- __tests__/api/tasks.test.ts, and they passed (6 tests passed).
  • Ran npm test which completed but failed unrelated suites due to a missing dependency (lru-cache) in other parts of the repo; this is unrelated to the DLQ changes.
  • Ran lint/type checks and npx tsc --noEmit; these surfaced pre-existing unrelated lint/type errors and missing packages (lru-cache, @wagmi/connectors) but did not indicate issues with the DLQ functionality.

Closes #192

@sonarqubecloud

Copy link
Copy Markdown

@clintjeff2

Copy link
Copy Markdown
Contributor Author

@leocagli, review and merge.

@clintjeff2

Copy link
Copy Markdown
Contributor Author

@leocagli, please review and merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Task queue: persist dead-letter queue to file and expose /api/tasks/dead-letter endpoint

1 participant