Problem Statement
Outbound webhooks sent to external integrations (e.g., Discord webhooks, personal dashboards) often fail due to transient network glitches, short-lived downtime on the target server, or temporary rate limits.
Currently, DevTrack records these failures in the database but takes no automated action. If a webhook fails, the event is permanently lost to the external system unless a user manually navigates to the Settings page, views the webhook's delivery history, and clicks the "Retry Delivery" button. This manual requirement makes DevTrack's webhook integration fragile and unreliable for developers.
Proposed Solution
Implement an automated, database-backed background retry mechanism with an exponential backoff strategy:
- Schema Updates: Add tracking fields to the
webhook_deliveries table (or create a dedicated webhook_retry_queue table):
retry_count: Integer (defaults to 0, max 3).next_retry_at: Timestamptz (nullable).status: Text (e.g., 'delivered', 'failed', 'retrying', 'exhausted').
- Failure Capture: If an outbound webhook dispatch fails with a transient error (e.g., network timeout, socket hang-up, or HTTP status
429 Too Many Requests / 5xx Server Error):
- Calculate the next retry time based on the number of attempts:
- Attempt 1: Retry in 2 minutes.
- Attempt 2: Retry in 15 minutes.
- Attempt 3: Retry in 60 minutes.
- Update
next_retry_at and increment retry_count.
- Background Worker: Create a new cron route
/api/cron/webhooks/retry (triggered periodically, e.g., every 5 minutes in vercel.json). This route will:
- Query all pending deliveries where
next_retry_at <= now() and retry_count < 3.
- Dispatch the webhook payload.
- On success, mark the delivery as
'delivered' and clear the retry fields.
- On failure, recalculate backoff or mark as
'exhausted' if the limit is reached.
- UI Updates: Display the retry status (e.g., "Retrying in 5 mins", "Retries Exhausted") in the
WebhookManager dashboard UI.
Feature Area
New Feature Area
Alternatives Considered
- In-Memory Queue (e.g., BullMQ or async queues): Rejected because DevTrack is designed to be deployed on serverless environments (like Vercel). Serverless instances are ephemeral and terminate quickly, which would cause pending in-memory retries to be lost.
- Adding a hard Redis queue dependency: While robust, Upstash Redis is optional in DevTrack's architecture. Implementing a simple, query-bounded queue in PostgreSQL (via Supabase) ensures it works out-of-the-box for all self-hosted and default deployments.
Acceptance Criteria
- Failed webhooks due to transient network or 5xx server issues automatically compute and schedule a
next_retry_at timestamp in the database.
- Webhooks that fail with client-side errors (e.g.,
400 Bad Request, 401 Unauthorized, 404 Not Found) are not retried, as they indicate a configuration mismatch rather than a transient error.
- The
/api/cron/webhooks/retry route successfully processes overdue retries and respects the configured backoff intervals.
- Deliveries that fail after 3 attempts are marked as
'exhausted' and stop scheduling further retries.
- The frontend webhook details view shows active retry attempts and countdown labels.
Additional Context
No response
Problem Statement
Outbound webhooks sent to external integrations (e.g., Discord webhooks, personal dashboards) often fail due to transient network glitches, short-lived downtime on the target server, or temporary rate limits.
Currently, DevTrack records these failures in the database but takes no automated action. If a webhook fails, the event is permanently lost to the external system unless a user manually navigates to the Settings page, views the webhook's delivery history, and clicks the "Retry Delivery" button. This manual requirement makes DevTrack's webhook integration fragile and unreliable for developers.
Proposed Solution
Implement an automated, database-backed background retry mechanism with an exponential backoff strategy:
webhook_deliveriestable (or create a dedicatedwebhook_retry_queuetable):retry_count: Integer (defaults to 0, max 3).next_retry_at: Timestamptz (nullable).status: Text (e.g.,'delivered','failed','retrying','exhausted').429 Too Many Requests/5xx Server Error):next_retry_atand incrementretry_count./api/cron/webhooks/retry(triggered periodically, e.g., every 5 minutes invercel.json). This route will:next_retry_at <= now()andretry_count < 3.'delivered'and clear the retry fields.'exhausted'if the limit is reached.WebhookManagerdashboard UI.Feature Area
New Feature Area
Alternatives Considered
Acceptance Criteria
next_retry_attimestamp in the database.400 Bad Request,401 Unauthorized,404 Not Found) are not retried, as they indicate a configuration mismatch rather than a transient error./api/cron/webhooks/retryroute successfully processes overdue retries and respects the configured backoff intervals.'exhausted'and stop scheduling further retries.Additional Context
No response