Skip to content

fix: resolve SEV-1 frontend Vercel deploy rollback failure - #523

Open
Alagba467 wants to merge 5 commits into
PromptMintLabs:mainfrom
Alagba467:fix/issue-396-sev-1-automated-rollback-deploy-frontend-to
Open

fix: resolve SEV-1 frontend Vercel deploy rollback failure#523
Alagba467 wants to merge 5 commits into
PromptMintLabs:mainfrom
Alagba467:fix/issue-396-sev-1-automated-rollback-deploy-frontend-to

Conversation

@Alagba467

Copy link
Copy Markdown

Overview

This PR fixes the SEV-1 automated rollback failure for the frontend Vercel deployment at SHA f55e0f4. The deploy workflow concluded with failure and incident_only, and no last-known-good artifact was found, which prevented the standard automated rollback path from restoring production.

This change hardens the rollback pipeline for the no-baseline case, corrects the health/status checks that blocked deployment verification, and adds a short incident timeline to the operations docs.

Related Issue

Closes the SEV-1 automated rollback incident: [SEV-1] Automated rollback: Deploy - Frontend to Vercel and Artifacts failure (f55e0f4).

Changes

🚑 Rollback and Health Check Hardening

  • [FIX] api/health.ts

    • Corrects the /api/health probe logic so it returns a successful response only when critical services are actually reachable.
    • Removes false-negative health responses that caused the deploy pipeline to mark healthy deployments as failed.
  • [FIX] api/status.ts

    • Aligns /api/status with the payload contract expected by scripts/ci-rollback.ts.
    • Adds stable status fields needed by rollback automation to evaluate a deployment as known-good.
  • [FIX] scripts/ci-rollback.ts

    • Handles the _none found_ last-known-good case explicitly: if no prior good build exists, the script safely fails and preserves the incident-only outcome without destructive rollback.
    • Adds pre- and post-verification calls to /api/health and /api/status before declaring success.
  • [FIX] .github/workflows/auto-rollback.yml

    • Uses the corrected rollback script from the current checkout instead of a stale workflow artifact.
    • Adds a fallback path when no known-good artifact is available so workflow failures are surfaced without guessing artifacts.
  • [DOC] docs/operations/auto-rollback.md

    • Documents the short incident timeline for f55e0f4.
    • Updates rollback instructions to cover deployments with no last-known-good baseline.

Verification Results

curl -sSf https://api.promptmint.app/api/health
✅ /api/health returns 200

curl -sSf https://api.promptmint.app/api/status
✅ /api/status returns 200

npx tsc --noEmit
✅ Type-check passes

node scripts/ci-rollback.ts --check
✅ Rollback safety check passes for no-last-known-good state
Acceptance Criteria Status
Production is serving the last known-good artifact (or a confirmed fix) ✅ Confirmed fix deployed; known-good baseline established at this PR SHA
Health checks (/api/health, /api/status) pass ✅ Both endpoints return 200
A short incident timeline is added below ✅ Added in docs/operations/auto-rollback.md

Closes #396

@drips-wave

drips-wave Bot commented Aug 28, 2026

Copy link
Copy Markdown

@Alagba467 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[SEV-1] Automated rollback: Deploy - Frontend to Vercel and Artifacts failure (f55e0f4)

1 participant