Skip to content

ci(e2e): prod LIVE E2E via on-the-fly minted cohort account #174

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mastermanas805 merged 1 commit into from
Jun 5, 2026
Merged

ci(e2e): prod LIVE E2E via on-the-fly minted cohort account #174

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
192 changes: 192 additions & 0 deletions .github/workflows/e2e-prod.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,192 @@
# Real-backend (LIVE) E2E against PRODUCTION (api.instanode.dev) using an
# ephemeral, cohort-scoped account minted on the fly. This is the prod sibling
# of e2e-live.yml (which targets STAGING) — DO NOT delete that one.
#
# Plan: docs/sessions/2026-06-04 TEST-ACCOUNTS-AND-NR-SYNTHETICS-PLAN.md.
#
# WHY this is safe to run against prod (and e2e-live.yml is not):
# - The api guards a mint endpoint (PR #260) that creates an account with
# is_test_cohort=true. The live worker skip-guards neuter
# billing/churn/email/quota for that team, so a LIVE run can never charge a
# card, burn a real quota budget, send a "we miss you" email, or churn a
# real customer.
# - The account + every resource it creates is reaped: this job DELETEs the
# minted account (DELETE /internal/e2e/account/{team_id}) AND runs the
# per-run ledger reaper (npm run reap:live) in an `if: always()` teardown.
# The reaper exits non-zero on any leak, failing the job loudly (rule 24).
# - cohort.ts assertSafeApiTarget() ALLOWS a prod E2E_API_URL only when a mint
# token / minted session is present (a sanctioned run); an un-tokened prod
# target is still refused, so a stray invocation can never hammer prod.
#
# HOW it mints/runs/reaps:
# 1. MINT — POST https://api.instanode.dev/internal/e2e/account with header
# X-E2E-Token: $E2E_ACCOUNT_TOKEN and body {"tier":"pro"} →
# {team_id, user_id, email, tier, session_jwt, expires_at}. The
# session_jwt + team_id are masked and exported to later steps.
# 2. RUN — E2E_LIVE=1 E2E_API_URL=https://api.instanode.dev
# E2E_SESSION_JWT=<minted> npx playwright test
# --config=playwright.live.config.ts. The authed legs use the
# minted account (cohort.ts mintedSession()); anon legs run as-is.
# 3. REAP — (always) DELETE the minted account, then npm run reap:live to
# sweep any spec-created resources from the on-disk ledger.
#
# Triggers:
# - workflow_dispatch (operator on demand).
# - schedule every 30 min (continuous prod integration signal).
# - repository_dispatch type `e2e-prod-from-deploy` (post-deploy hook the api
# repo can fire after a prod rollout).
#
# Guard: if secrets.E2E_ACCOUNT_TOKEN is empty (not yet configured) the job
# no-ops cleanly with a ::notice:: — it NEVER reds when unconfigured. The
# workflow ships before the secret exists and goes green only once the operator
# sets E2E_ACCOUNT_TOKEN and the api mint endpoint is deployed.

name: E2E LIVE (prod, minted account)

on:
workflow_dispatch: {}
schedule:
# Every 30 minutes — continuous prod integration signal.
- cron: '*/30 * * * *'
repository_dispatch:
types: [e2e-prod-from-deploy]

concurrency:
# One prod LIVE run at a time: they mint a real cohort account + create real
# resources; overlapping runs could interleave ledger writes / dedup state.
group: e2e-prod-${{ github.workflow }}
cancel-in-progress: false

permissions:
contents: read

jobs:
e2e-prod:
name: LIVE against prod via minted account + reap
runs-on: ubuntu-latest
timeout-minutes: 15
env:
# Fixed prod target — this workflow is prod-only by design.
E2E_API_URL: https://api.instanode.dev
E2E_LIVE_RUN_ID: ${{ github.run_id }}
# The mint-endpoint guard token. Empty until the operator configures it →
# the gate step below no-ops the job cleanly.
E2E_ACCOUNT_TOKEN: ${{ secrets.E2E_ACCOUNT_TOKEN }}
steps:
- name: Gate on configured mint token
# No token configured → no-op cleanly (never a false red). Sets RUN=0
# so every later step is skipped.
run: |
set -euo pipefail
if [ -z "${E2E_ACCOUNT_TOKEN:-}" ]; then
echo "::notice::secrets.E2E_ACCOUNT_TOKEN not configured — skipping prod LIVE E2E (no-op)."
echo "RUN=0" >> "$GITHUB_ENV"
else
echo "RUN=1" >> "$GITHUB_ENV"
fi

- uses: actions/checkout@v6
if: env.RUN == '1'

- uses: actions/setup-node@v6
if: env.RUN == '1'
with:
node-version: '22'
cache: 'npm'

- name: Install deps
if: env.RUN == '1'
run: npm ci

- name: Install Chromium
if: env.RUN == '1'
run: npx playwright install --with-deps chromium

- name: Mint ephemeral cohort account
id: mint
if: env.RUN == '1'
# POST the guarded mint endpoint → capture session_jwt + team_id, mask
# them, and export to later steps. Fails the job (non-2xx) so a broken
# mint endpoint surfaces immediately rather than running un-authed.
run: |
set -euo pipefail
resp="$(curl -sS -w '\n%{http_code}' \
-X POST "${E2E_API_URL}/internal/e2e/account" \
-H "X-E2E-Token: ${E2E_ACCOUNT_TOKEN}" \
-H 'Content-Type: application/json' \
-d '{"tier":"pro"}')"
code="$(printf '%s' "$resp" | tail -n1)"
body="$(printf '%s' "$resp" | sed '$d')"
if [ "$code" != "200" ]; then
echo "::error::mint endpoint returned HTTP $code (expected 200). Body: $body"
exit 1
fi
jwt="$(printf '%s' "$body" | jq -r '.session_jwt // empty')"
team="$(printf '%s' "$body" | jq -r '.team_id // empty')"
email="$(printf '%s' "$body" | jq -r '.email // empty')"
tier="$(printf '%s' "$body" | jq -r '.tier // empty')"
if [ -z "$jwt" ] || [ -z "$team" ]; then
echo "::error::mint response missing session_jwt or team_id. Body: $body"
exit 1
fi
# Mask the secrets so they never appear in logs.
echo "::add-mask::$jwt"
echo "::add-mask::$team"
# session_jwt + team_id are secret-ish → env only (not step outputs).
# team_id is also a non-secret output for the reap step's `if`.
{
echo "MINTED_SESSION_JWT=$jwt"
echo "MINTED_TEAM_ID=$team"
echo "MINTED_EMAIL=$email"
echo "MINTED_TIER=$tier"
} >> "$GITHUB_ENV"
echo "minted=1" >> "$GITHUB_OUTPUT"
echo "Minted cohort account (tier=$tier) — session + team_id masked."

- name: Run LIVE E2E against prod (minted account)
if: env.RUN == '1' && steps.mint.outputs.minted == '1'
env:
E2E_LIVE: '1'
# The minted account drives the authed legs (cohort.ts mintedSession);
# anon legs run as-is. assertSafeApiTarget() permits the prod target
# because E2E_SESSION_JWT is present (a sanctioned run).
E2E_SESSION_JWT: ${{ env.MINTED_SESSION_JWT }}
E2E_TEAM_ID: ${{ env.MINTED_TEAM_ID }}
E2E_ACCOUNT_EMAIL: ${{ env.MINTED_EMAIL }}
E2E_ACCOUNT_TIER: ${{ env.MINTED_TIER }}
run: npm run test:e2e:live

- name: Reap minted account (teardown)
# ALWAYS runs (even on test failure/cancel) so the minted account + its
# resources are deleted out-of-band. Idempotent: 404 == already gone.
if: always() && env.RUN == '1' && env.MINTED_TEAM_ID != ''
run: |
set -euo pipefail
code="$(curl -sS -o /dev/null -w '%{http_code}' \
-X DELETE "${E2E_API_URL}/internal/e2e/account/${MINTED_TEAM_ID}" \
-H "X-E2E-Token: ${E2E_ACCOUNT_TOKEN}")"
case "$code" in
200|202|204|404|410)
echo "Reaped minted account (HTTP $code)." ;;
*)
echo "::error::DELETE minted account returned HTTP $code — possible leak."
exit 1 ;;
esac

- name: Reap cohort resources from ledger (teardown)
# The per-run ledger reaper sweeps any resource a spec created. Exits
# non-zero on any leak, failing the job loudly (rule 24).
if: always() && env.RUN == '1'
run: npm run reap:live

- name: Upload LIVE trace + ledger on failure
if: failure() && env.RUN == '1'
uses: actions/upload-artifact@v4
with:
name: e2e-prod-trace-${{ github.run_id }}
path: |
test-results/
playwright-report-live/
e2e/.cleanup-ledger.json
if-no-files-found: ignore
retention-days: 14
97 changes: 94 additions & 3 deletions e2e/cohort.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,12 @@
// real quota budget, or attempt a real charge.
//
// This is the instanode-web side ONLY. The backend `is_test_cohort` column +
// the guards that read it are intentionally NOT in this PR (one-tree
// discipline) — see the PR body's follow-up note. Until those guards exist,
// LIVE runs MUST target STAGING, never prod.
// the guards that read it ship in the api/worker tree (PR #260): a minted team
// is `is_test_cohort=true`, and the live worker skip-guards neuter
// billing/churn/email/quota for it. With those guards + the mint endpoint
// (cohort-scoped) + the reaper all live, a SANCTIONED minted-account run MAY
// target prod (see assertSafeApiTarget below). An un-sanctioned/un-tokened run
// against prod is still REFUSED so a stray invocation can never hammer prod.
//
// The contract the backend guards will key on (kept here as the single source
// of truth for the string the two repos share):
Expand Down Expand Up @@ -76,3 +79,91 @@
export function isCohortBranded(value: string | null | undefined): boolean {
return !!value && value.includes(COHORT_MARKER)
}

// ── Prod-target safety (item 3) ──────────────────────────────────────────────
// LIVE specs create REAL backend resources. Originally cohort.ts refused any
// prod E2E_API_URL outright (only STAGING was safe). Now that the backend
// `is_test_cohort` skip-guards (PR #260), the cohort-scoped mint endpoint, and
// the reaper are all live, a prod run is safe IFF it is a SANCTIONED
// minted-account run — i.e. it carries a mint token (E2E_ACCOUNT_TOKEN, used by
// the CI workflow to mint/reap the account) or an already-minted session JWT
// (E2E_SESSION_JWT). A prod target WITHOUT either is still refused, so a stray /
// mis-configured invocation can never provision-and-leak against prod.

Check warning on line 91 in e2e/cohort.ts

View workflow job for this annotation

GitHub Actions / typos 

"mis" should be "miss" or "mist".

/** The prod api host. A prod target is only allowed for a sanctioned minted run. */
export const PROD_API_HOST = 'api.instanode.dev'

/** True when the resolved api base points at the prod api host. */
export function isProdApiTarget(apiUrl: string): boolean {
if (!apiUrl) return false
try {
return new URL(apiUrl).host.toLowerCase() === PROD_API_HOST
} catch {
// Not a parseable URL — be conservative and substring-match the host so a
// malformed-but-prod-looking value can't slip past as "not prod".
return apiUrl.toLowerCase().includes(PROD_API_HOST)
}
}

/**
* True when this process is a SANCTIONED minted-account run: it holds a mint
* token (the workflow mints/reaps the account out-of-band) or an already-minted
* session JWT. Either proves the run is the cohort-scoped, reaped, skip-guarded
* path rather than a stray prod invocation.
*/
export function isSanctionedMintedRun(): boolean {
return !!(process.env.E2E_ACCOUNT_TOKEN || process.env.E2E_SESSION_JWT)
}

/**
* Guard a LIVE spec's resolved api target. Throws (failing the spec loudly,
* never silently passing) when E2E_API_URL points at prod WITHOUT a sanctioned
* minted-account run. Staging targets and sanctioned prod runs pass through.
* Specs call this once at module load (before any provision) via topGuard().
*/
export function assertSafeApiTarget(apiUrl: string): void {
if (isProdApiTarget(apiUrl) && !isSanctionedMintedRun()) {
throw new Error(
`Refusing to run LIVE E2E against prod (${PROD_API_HOST}) without a sanctioned ` +
`minted-account run. Set E2E_ACCOUNT_TOKEN (CI mints+reaps a cohort account) ` +
`or E2E_SESSION_JWT (a pre-minted cohort session), or point E2E_API_URL at staging. ` +
`This guard exists so a stray run can never provision-and-leak real prod resources.`,
)
}
}

// ── Workflow-minted account (item 2) ─────────────────────────────────────────
// The prod E2E workflow mints an ephemeral cohort account up front
// (POST /internal/e2e/account) and exports its session JWT + identity into the
// env. When E2E_SESSION_JWT is set, the authed legs use THAT account's bearer
// instead of self-minting from E2E_JWT_SECRET — so the authed flow runs against
// prod as a real, skip-guarded cohort team. Anon legs are unaffected.

/** The minted account's identity + bearer, surfaced from the workflow env. */
export interface MintedSession {
/** Bearer token for authed requests (the api session JWT). */
token: string
/** The minted team's id (the workflow reaps the account by this out-of-band). */
teamID: string
/** The minted user's email, when the workflow exported it. */
email: string
/** The minted tier (e.g. 'pro'), when the workflow exported it. */
tier: string
}

/**
* Returns the workflow-minted session when E2E_SESSION_JWT is set, else null.
* Authed legs prefer this over self-minting so a prod run uses a real cohort
* account. E2E_TEAM_ID / E2E_ACCOUNT_EMAIL / E2E_ACCOUNT_TIER are the companion
* fields the workflow exports from the mint response.
*/
export function mintedSession(): MintedSession | null {
const token = process.env.E2E_SESSION_JWT
if (!token) return null
return {
token,
teamID: process.env.E2E_TEAM_ID ?? '',
email: process.env.E2E_ACCOUNT_EMAIL ?? '',
tier: process.env.E2E_ACCOUNT_TIER ?? '',
}
}
6 changes: 5 additions & 1 deletion e2e/live-anon-provision.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@

import { expect, test, type APIRequestContext } from '@playwright/test'

import { cohortName, COHORT_MARKER } from './cohort'
import { cohortName, COHORT_MARKER, assertSafeApiTarget } from './cohort'
import {
recordEntity,
loadLedger,
Expand Down Expand Up @@ -157,6 +157,10 @@ test.describe('LIVE — every anonymous provision flow → backend-assert → re
'E2E_LIVE=1 but E2E_API_URL/AGENT_API_URL is unset — no backend to target.',
)

// Prod-target safety (item 3): refuse an un-sanctioned prod target; allow it
// only for a minted-account run (E2E_ACCOUNT_TOKEN/E2E_SESSION_JWT present).
if (LIVE && API_URL) assertSafeApiTarget(API_URL)

// Backstop reaper (rule 24): even if a per-service test throws before its
// inline reap, afterAll reaps every still-ledgered entity. The standalone
// reap-cohort.ts re-runs this same path in CI teardown if the whole process
Expand Down
Loading
Loading