One Hono/TypeScript API for Node/Docker and Cloudflare Workers. The dashboard and generated SDK use that same public API; MCP discovers its OpenAPI contract. PostgreSQL is the source of truth: queued mail rows are the dispatch queue, a Postgres outbox drives campaign orchestration and webhooks, and private R2/S3 stores attachments. On Cloudflare a Durable Object per account/region hosts the long-lived dispatcher and a queue carries verified provider feedback; no Redis, D1, passwords, teams, or production admin bypass.
For Hosted Ohio receiving bootstrap, readiness, SNS dead-letter replay, quota increases and the exact production rollout checks, see Receiving operations. See receiving measurements for the SQL and storage budgets.
Send and schedule need only the current revision. The request freezes the audience and content into one snapshot, reserves the recipients and moves the campaign to preparing; the 202 response means that background workflow was accepted, not that validation or SES delivery has finished. GET /v1/campaigns/{id}/audience counts the saved audience (matched, eligible, suppressed, unsubscribed) before a send is confirmed; it is a live read and freezes nothing. In MCP Code Mode, previewCampaignAudience exposes the same count; sendCampaign and scheduleCampaign take the current revision.
A campaign has one state, written only by src/states.ts: draft → preparing (every message validated in the background; delivery.prepared is the cursor) → scheduled (waiting for scheduledAt) or sending (recipients released into the dispatch queue; delivery.expanded) → done, set by the dispatcher once the queue drains for the campaigns it moved rows for (one statement per draining batch, campaign rows only; the minute sweep finishes any it missed). paused is entered by the user or the health check and resumes to wherever the cursors say. failed keeps delivery.error: if nothing was released yet it can be edited (back to draft) or sent again from a fresh snapshot; once recipients were released it can only be resumed (continuing from delivery.expanded on the same snapshot, so nobody receives the message twice) or canceled, and it keeps its reservation until then. canceled is terminal. One job row per campaign (campaign.run:<id>) does all of the work and waits in place for the schedule, the dispatch buffer, pacing or the SES quota, so a campaign can never be left sending with nothing queued or preparing after its job exhausted its retries: the last retry or a definitive error leaves failed.
Only referenced personalization properties are copied into immutable recipient rows; the rendered base email is shared. Email rows are materialized progressively rather than all existing immediately; future schedules reserve capacity without filling the active dispatch buffer. delivery reports the cursors, pacing, pause and hold reasons; its canceled count is unmaterialized cancellations, separate from canceled email rows. Final personalized audit snapshots are saved at dispatch. Never manually replay an uncertain SES send.
| Bound | Current implementation |
|---|---|
| Matching contacts per campaign | 2,000,000, before consent/suppression filtering |
| Concurrent campaign runs | Bounded by worker/runner job concurrency; other campaigns remain durable queue work |
| Referenced personalization snapshot | 1 GiB per campaign, 1 MiB per recipient |
| Preparation/release chunk | Up to 20,000 / 2,000 recipients respectively; preparation is also bounded to 4 MiB and ten seconds per step |
| Active campaign dispatch buffer | Ten seconds of the cached regional send rate per campaign (400 floor, 20,000 cap) and twenty seconds per workspace (1,000 floor, 50,000 cap). On MX, each shared workspace plans against 25% of the pool rate; BYO uses its full account rate. The daily quota and dispatcher gate remain account-wide. |
| Outstanding campaign intents | 2,000,000 per originating credential; 4,000,000 per workspace |
| Orchestration jobs per process/invocation | JOB_CONCURRENCY: default 2, maximum 8 for Node and 6 for Cloudflare (campaign runs, webhooks, maintenance; never mail) |
| Dispatcher shards | DISPATCH_SHARDS Durable Objects per account/region on Cloudflare (configured to 4; one per BYO account/region), each pacing 1/N of the regional quota with up to six provider calls in flight. Sending up to six emails wakes one shard per target region; larger bursts wake all shards. The Node runner dispatches in-process. |
These are application bounds, not a guaranteed SES send rate. Mail is dispatched by long-lived batch dispatchers (src/dispatcher.ts): one statement leases up to twenty due rows, one transaction elects the batch (origin authorization, consent locks, campaign cancellation, unsubscribe tokens, attempt events), provider calls run six-wide behind an in-memory pacing gate, and one statement records every outcome. sending_region_limits is the shared one-minute quota cache; only a shard that finds it stale calls GetAccount. No SES request holds a database transaction, and a row is sent at most once because the claim flips it queued → attempting under its dispatch version. Direct sends retain their separate pending limits. Node runners hold one in-process gate each, so run one runner per installation or split the quota explicitly.
Apply migrations through 019_batch_dispatcher.sql before deploying this backend; pre-migration email.dispatch job rows complete as no-ops once the dispatcher owns queued rows. For Docker, stop old runners and upgrade API/runner images together; the new runner continues a previous release's campaign.prepare/expand/finish rows as runs of the same campaign, but old runners do not recognize campaign.run. Do not roll back to an older backend while campaigns are active.
MX regional sending (operator rollout): Apply migrations through 071, then deploy the compatible API, runners and dispatchers with BANE_REGIONAL_SENDING=false (the default). The backfill copies the current connection, domain ID and region, not retained mail. A metadata lock timeout means retry the migration, not disable the timeout. Do not enable the flag until old instances have drained and the regional IAM/SES canaries pass. The flag applies only to MULTI_TENANT=true: From's closest claimed domain selects the sending region, each batch item resolves independently, and a conflicting explicit region rejects the send. It does not publish a new stack, provision Virginia, move a domain, or change receiving DNS. Disable the flag to stop new regional admissions; retain the schema and compatible dispatchers so already-pinned work and late feedback keep their original bindings. Workspaces with managed bindings cannot fall back to singleton sending.
Disabling a region retires its binding generation. The minute recovery sweep cancels up to 1,000 still-queued envelopes per pass with SENDER_BINDING_CHANGED, including scheduled mail. It leaves attempts alone; resuming a region never reauthorizes old work. Draft editing, history filters and campaign controls remain available.
Hosted trusted sending (operator only, migration 081): BANE_TRUSTED_POOL=false by default. Requires both MULTI_TENANT=true and BANE_REGIONAL_SENDING=true. Ohio remains the default and the only receiving pool. Shadow scores do not route traffic. From api/, node --import tsx scripts/set-domain-pool.ts "$WID" DOMAIN trusted --reason "why" records initial placement and queues hosted.trusted. probation selects Ohio; "$WID" --prepare-trusted --reason "why" prepares just the workspace tenant without selecting any domain. Set the same three flags in the script environment. Check jobs.last_error, the regional states and the operator audit before enrollment. A set/topic or tenant with ambiguous ownership requires manual review, not automatic repair. Do not use a tier change to move an existing binding or bypass a hold. A live move needs separate target-identity staging, DNS consent, canary and drain.
Dedicated Hosted accounts: Use the three offline-rendered IAM policies in the matching account. Prepare Ohio receiving first. Verify signed simulator delivery and tenant isolation in each enabled pool before admitting customer traffic. There is no identity-adoption manifest or shared-account release checker for these accounts.
Platform domain (operator only, migration 107): PUBLIC_URL's host and its subdomains are refused to every workspace except BANE_PLATFORM_DOMAIN_WORKSPACE (Hosted with regional sending only, unset by default), and that workspace claims nothing there until an operator has adopted the root. Even then it never claims, receives on or sends from the platform's own hosts: dev., e2e./e2e-*., in., domainconnect., imap., tracker. and everything below them (reservedPlatformName in src/domain-claims.ts; add new platform hosts there before they go live). A release fixture that holds its own e2e* claim keeps sending from it. The sender policy lives in regional_sender, which admission, campaign reviews, mailbox replies and dispatch all read with the running configuration: removing or changing the variable stops the workspace sending as the platform domain at once, including mail already queued. Receiving then ends with the usual 72-hour proof grace; delete the domain to stop it at once.
The root's SES identity was made by hand and has no claim tag, so only an operator adopts it. From api/, with a short-lived DB role and a signed-in operator role in the platform account, run node --import tsx scripts/adopt-platform-domain.ts "$WID" (dry run). Review its DKIM tokens, MAIL FROM and default set, then run it again with --apply --fingerprint <the dry run's fingerprint> --reason "why"; a different live identity is refused, and so is one with sending authorization policies. It pins that fingerprint, associates the identity with the workspace's Ohio tenant (its only AWS write), records platform-domain-intent and platform-domain-adopted in hosted_operator_audit, and chooses the root as the receiving host with a store catch-all: unknown recipients are kept as unrouted mail and never become addresses. The validation mailboxes a certificate authority may use (admin@, administrator@, hostmaster@, postmaster@, webmaster@, root@) are never created automatically under the platform domain, even with a create_address catch-all; only an inbox created on purpose receives them. SES's own DKIM check stands in for the _bane TXT. Nightmail never creates, changes or deletes that identity, and the runtime policy can't; deleting the domain only releases the claim. The fingerprint is checked daily and on Check now: a changed identity then stops sending and receiving together, and only another dry run and --apply restore it. To undo an adoption, including an aborted one, run the script with --revoke (dry run), then --revoke --apply --reason "why": it releases the claim, removes the identity from the workspace's tenant and records platform-domain-revoked. It works even after the variable was changed.
Hosted bulk DMARC (opt-in): BANE_REQUIRE_DMARC_FOR_BULK=false by default. Enable only after checking senders' domain readiness and allowing time for DNS repairs. When enabled, admitting a marketing email or launching/scheduling a campaign requires a checked exact From domain with an effective DMARC policy (including inherited sp or p) and aligned DKIM or custom MAIL FROM. Unknown, stale or invalid observations fail with BULK_DMARC_REQUIRED; transactional mail and BYO are unaffected. This is an admission check, not a recall of already-queued mail. MX verification and periodic DNS upkeep populate the cached observation; Check now requests a fresh check. A recommended p=none TXT is for an empty configuration only. Nightmail does not ingest aggregate reports; add rua only if you operate a reporting destination. Inventory legitimate senders before tightening the policy.
Upgrading from the live/test split: release 052_prepare_drop_environment.sql with the backend that stops using environment columns. Migration 052 permanently discards test data and test-only OAuth approvals, but keeps the columns, defaults and conflict targets needed by the previous single-mode backend (2f350fa). Release 053_drop_environment.sql separately, only after all API instances, runners and dispatchers use the new backend and old invocations have drained. After 053, do not roll back to a backend that names those columns. Fresh installations can apply both migrations together.
Use Node.js 24 and Docker Compose for the local quickstart. All commands below run from api/ unless stated otherwise.
bun install
cp .env.example .envFill .env locally; never commit it or paste credentials into the dashboard. Generate each secret independently with:
node -e 'console.log(require("node:crypto").randomBytes(32).toString("hex"))'| Setting | What to enter |
|---|---|
BANE_ROOT_SECRET |
At least 32 random bytes, such as the 64-character hex output above. This is the installation root secret for authentication and domain-separated webhook encryption. |
BANE_PLATFORM_AWS_ACCOUNT |
12-digit platform AWS account ID. Required for the API, runners and operator scripts when MULTI_TENANT=true and any of BANE_REGIONAL_SENDING, BANE_TRUSTED_POOL or BANE_BYO_V2 is true. |
BANE_NOTIFY_FROM |
Hosted only, optional. The From address for owner notices about sender standing (warn, restrict, pause, quarantine), on the platform domain of BANE_PLATFORM_DOMAIN_WORKSPACE; the notices are sent through that workspace. Unset: notices show only in the dashboard and the account.standing.changed webhook. |
BANE_ALLOW_SES_SANDBOX |
Development installations only (default false). Lets Hosted pools become ready while the AWS account is in the SES sandbox, so sends reach only the mailbox simulator and verified addresses. Never set it in production. |
DISPATCH_RATE_FACTOR |
Optional pacing target as a multiple of the SES MaxSendRate (default 1; production uses 1.1, i.e. 22/s at a 20/s quota). Slots are spaced evenly at the target and the target follows quota changes automatically; any throttled response brakes back to the raw quota for ten seconds, so a modest overshoot cannot spiral into retries. |
DISPATCH_TARGET_RATE |
Optional absolute SES sending target in recipients per second; replaces the quota-derived rate (the brake still applies). Leave unset unless the SES quota is known to be stale. |
DISPATCH_DAILY_HEADROOM_PERCENT |
Percent of the SES daily quota and of each Hosted workspace’s effective daily cap kept for transactional recipients (default 10; 0 disables the reserve). Campaigns wait automatically for capacity. This does not split the per-second rate. |
DISPATCH_SHARDS |
Cloudflare only: number of shards for self-hosted sending or the MX shared pool (configured to 4). Each paces 1/N of the regional target and holds up to six provider calls in flight. MX BYO accounts use one shard. |
JOB_CONCURRENCY |
Optional active orchestration jobs (campaign runs, webhooks, maintenance) per process/invocation. Defaults to 2; capped at 8 for Docker/Node and 6 for Cloudflare. Mail dispatch is separate: the runner's in-process dispatcher, or DISPATCH_SHARDS Durable Objects on Cloudflare. |
ACCOUNT_ISSUER, ACCOUNT_CLIENT_ID, ACCOUNT_CLIENT_PRIVATE_KEY |
HTTPS OIDC issuer (defaults to https://account.anoma.ly), confidential client ID, and server-only P-256 private JWK with kid. |
ACCOUNT_ID_TOKEN_ALG |
Issuer's ID token algorithm: ES256 (default) or RS256. Client assertions use ES256. |
ACCOUNT_REQUIRED_ACR |
Exact acr required in the signed ID token for each new dashboard session and in every MCP access token. Hosted installations must set urn:anomaly:acr:google (startup refuses otherwise). Self-hosted: set it empty explicitly for an issuer without a method requirement. |
AUTH_ALLOWED_SUBJECTS |
Self-hosted access: comma-separated exact issuer subjects (sub), never addresses. Listed owners and admins may use the dashboard and MCP. Find a subject with scripts/list-signin-attempts.ts after the person's first, refused sign-in. AUTH_ALLOWED_EMAILS and AUTH_ALLOWED_DOMAINS were removed; startup refuses them. |
DEFAULT_WORKSPACE_OWNERS |
Self-hosted: allowed subjects with no memberships become owners of default at their first sign-in if listed here. Comma-separated exact subjects, empty for none, or * (the default) for every allowed subject. |
AUTH_STAFF_GOOGLE_DOMAINS |
Hosted: Google Workspace domains whose accounts may sign up and create workspaces, read from the sign-in's provider identity (hd, see src/account-policy.ts), never from an address. Hosted dev and production commit it empty (staff sign-up off) until the account service's provider-identity claim name is confirmed. |
PUBLIC_URL |
Canonical external origin, without path, query, fragment, or credentials. HTTPS is required except for local sign-in/development at localhost and 127.0.0.1. Docker default: http://127.0.0.1:8793; AWS provisioning requires a public HTTPS origin. |
DEFAULT_SES_REGION |
Initial region, default us-east-1. Migration seeds the database once and also imports already-used regions to avoid stranding existing work. Subsequent region/default changes use the API, not this environment value. |
AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN |
Explicit server-only IAM credentials for SES discovery, provisioning, and live sending; the session token is optional for temporary credentials. No automatic role lookup or token renewal. |
REPUTATION_ALERT_SLACK_URL |
Optional Slack incoming webhook (https://hooks.slack.com/...) for hourly SES reputation alerts. Posts when an enabled region's bounce or complaint rate crosses 80% of the warning threshold, the warning threshold (5% bounce, 0.1% complaint), 80% of the at-risk threshold, or the at-risk threshold (10%, 0.5%), and when it falls back. Escalations send immediately; other changes wait six hours after the previous message, a level is not repeated within 24 hours, and dropping a level needs a 10% margin. Requires live SES and cloudwatch:GetMetricData. Store it as a secret (wrangler secret put on Cloudflare). |
TYPESAFE_API_KEY |
Required for Hosted live SES when BANE_HOSTED_SCREEN=true; missing configuration fails startup. Optional for explicit screen: true elsewhere. Use a TypeSafe AI key or an OpenCode Console key with OpenCode's JEV_ENDPOINT. Store it as a secret (wrangler secret put on Cloudflare). |
JEV_ENDPOINT |
Optional Jev endpoint (default https://api.typesafe.ai/v1/systemone). OpenCode's free tier is https://opencode.ai/inference/systemone/v1/systemone with JEV_MODEL=jev-1.13-free. |
JEV_MODEL |
Optional Jev model for the content screen (default jev-latest). |
JEV_SCREEN_MODE |
reject (default) rejects screened live transactional mail at or above the threshold before it reaches SES: status rejected, error code CONTENT_REJECTED_SPAM, and a reject event with providerCalled: false and the score. flag is advisory for optional checks only. Mandatory Hosted probation/marketing checks always reject above threshold and defer on outage; they never send unscreened. Probation mail is screened and delivered as one copy: its style sheet rules are inlined and its HTML reduced to the markup and inline styles a client shows as text before the classifier reads it, and that reduced copy is what SES sends (see "Probation HTML" under Sending and operational boundaries). Trusted and BYO senders keep their markup; their screen reads the text in document order, inline runs as one word. HTML too large or too deeply nested to read within budget is rejected with CONTENT_TOO_LARGE_TO_SCREEN. |
JEV_SCREEN_THRESHOLD |
Optional spam probability, 0–1, at which the screen acts (default 0.7). |
JEV_MESSAGE_LIMIT_BYTES |
Optional screened-text limit per Hosted message in UTF-8 bytes (default 24576, the limit the screen always had). The screen itself is unchanged: it reads the HTML's text with its alt and title text, the text part and raw MIME, and up to 60 link hosts. Admission measures what it will read and refuses a Hosted message over the limit with 422 CONTENT_TOO_LARGE_TO_SCREEN and error.details[0] { screenedBytes, limitBytes } (or { linkHosts, linkHostLimit }) instead of rejecting it at dispatch; the dispatcher's check remains as a backstop. Bring-your-own AWS mail with screen: true is screened at dispatch as before. Each classifier request carries about 7 KB of new text, so 24 KiB is at most 4 requests per message. |
JEV_CAMPAIGN_LIMIT_BYTES |
Optional screened-text limit for a Hosted campaign's own copy and each of its copies, and for campaign tests (default 49152, at most 8 requests). A campaign is screened copy by copy, each copy exactly as a single message is, but each distinct copy once (src/sending.ts campaignVerdicts): a verdict is keyed by the copy as it is delivered (src/screen.ts copyKey: sender, reply-to, subject, the HTML, text and raw parts, headers and attachments, the classifier and model): the customer's copy byte for byte, built with the slots Nightmail adds per recipient (the unsubscribe footer's link and the List-Unsubscribe headers) generated for a placeholder link rather than found and replaced in the delivered copy (src/sending.ts screenedAs; the recipient's own link must be one this installation generated, generatedUnsubscribeUrl), and is stored per campaign in sending_campaign_screens, so identical copies share it and a relaunch reuses it. Preparation screens the campaign's own copy first (a positive verdict fails the campaign before release). Each release first screens the next recipients' copies as they will be delivered (up to 2,000 ahead, a page of 500 per pass) and releases no one past the last copy with a verdict; each screened recipient counts toward reputation then, released or not; refused copies are rejected at dispatch with CONTENT_REJECTED_SPAM, and a refused share past the screen's restrict line (5% and 10) quarantines the campaign once. The dispatcher keys each copy from the same customer copy it delivers and sends only under a passing verdict for that key, screening a copy without one (for example after a tier change) or waiting. Requests, failed ones included, are charged to the review's budget before they are made (screen_budget, (recipients + 10) × 8; CONTENT_SCREEN_BUDGET_EXCEEDED past it); an interrupted screen resumes at its next request, and outages back off from 30 seconds to 15 minutes. A campaign launched before this release is screened per recipient as before. |
POSTGRES_PASSWORD, DATABASE_URL |
Generate a password and substitute the same value into the local database URL. Compose supplies its internal database hostname automatically. |
S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY |
Local MinIO credentials; use a nonempty access ID and a strong generated secret. The local bucket is created separately below. |
MINIO_IMAGE |
The local object store image, pinned by digest (name@sha256:…). MinIO no longer publishes anonymous community images, so build or mirror one you trust; Compose refuses to start without it. |
Keep ENABLE_LIVE_SES=false initially. ADMIN_API_KEY, public WORKSPACE_ID, and a separate ENCRYPTION_KEY are not fresh-install settings. There is no seeded production administrator: access comes only from an approved OIDC identity. See upgrade restrictions before reusing an older database.
Register a confidential web client with your OIDC issuer. Register ${PUBLIC_URL}/api/auth/callback/account as its exact callback and ${PUBLIC_URL}/ as its post-logout URI. Upload the public P-256 JWK and select private_key_jwt (ES256). Supply the matching private JWK through your server secret manager. Use the same issuer and client settings for the API and job runner. A self-hosted issuer must support discovery, PKCE S256, signed ID tokens and the openid email profile scopes. Nightmail uses PAR when advertised.
Identity is (issuer, subject); an email address is display and contact data only, because an issuer can assert a verified address that the sign-in method never vouched for. A new subject is admitted only by: the sign-in's provider identity (hosted staff, AUTH_STAFF_GOOGLE_DOMAINS), a pending invitation link it followed (the invitation also has to name its verified address, and acceptance binds the membership to the subject), AUTH_ALLOWED_SUBJECTS (self-hosted), or an operator binding the subject to a user. Nightmail records every refused, verified sign-in by subject (nightmail_signin_attempts, kept 30 days); list them with node --import tsx scripts/list-signin-attempts.ts, confirm with the person out of band, then add the subject to AUTH_ALLOWED_SUBJECTS or link it to an existing user (INSERT INTO nightmail_oidc_identities(issuer,subject,user_id)). Nobody is linked to an existing user by address; the migration only copies existing (issuer, subject) links. A missing or unverified address never blocks an admitted subject's sessions, MCP access or queued sends; only an invitation, whose target is an address, needs the verified address it names.
Nightmail stores a hash of the browser's random session token and checks the session expiry and workspace membership in PostgreSQL. Signing out revokes the Nightmail session.
On sign-in Nightmail refreshes the address and verification flag from the signed ID token or subject-checked UserInfo response. An address change invalidates prior sessions. A missing or unverified address cannot accept invitations. When moving a test installation from pairwise to public subjects, review the user's identity and current mailbox first. With sign-ins fenced, remove the obsolete nightmail_oidc_identities binding and approve the user's exact address in bane_legacy_claims for the expected issuer. The next verified sign-in binds the public subject to the original Nightmail user ID.
The image bundles the dashboard with the API on one origin, http://127.0.0.1:8793. The API and background worker are separate services using the same image and database. Stop any host API using port 8793 first.
-
Start the local infrastructure after completing
.env:docker compose -p bane-api-local up -d postgres storage
Postgres is exposed on
127.0.0.1:55432; MinIO on127.0.0.1:59000. Wait until both services are ready before provisioning the bucket. -
Create the private local bucket once on a fresh installation:
node --env-file=.env --input-type=module - <<'JS' import { S3Client, CreateBucketCommand } from '@aws-sdk/client-s3'; const e = process.env; const client = new S3Client({ endpoint: e.S3_ENDPOINT, region: e.S3_REGION, forcePathStyle: true, credentials: { accessKeyId: e.S3_ACCESS_KEY_ID, secretAccessKey: e.S3_SECRET_ACCESS_KEY } }); await client.send(new CreateBucketCommand({ Bucket: e.S3_BUCKET })); JS
Do not recreate an existing bucket or make it public. Managed storage can instead be provisioned through its provider.
-
Build and apply migrations to the intended database:
docker compose -p bane-api-local build api docker compose -p bane-api-local run --rm api node dist/migrate.js
Migrations are explicit, serialized, and checksum-checked. HTTP startup does not run them automatically.
-
Start both processes:
docker compose -p bane-api-local up -d api worker
Open
http://127.0.0.1:8793and sign in using your approved account. The image runs as the unprivilegednodeuser and excludes local secrets/test files. -
Create a scoped API key for SDK/MCP calls. Check
GET /v1/me:simulated: truemeans this deployment will not send through SES. Simulation still writes local data and delivers configured webhooks.
For managed infrastructure, run the same image with your actual database/private storage settings rather than the local Compose overrides. API command: node dist/server.js; background worker: node dist/runner.js. Multiple workers use Postgres row locks and leases to claim jobs safely. Put public deployments behind HTTPS, with PUBLIC_URL set to the browser-visible origin.
Use the same local Postgres/MinIO and bucket as above, but stop Docker's api/worker before running host processes. Keep Node.js 24 active in each terminal.
- Set
PUBLIC_URL=http://127.0.0.1:5173inapi/.env. Configure the OIDC client originhttp://127.0.0.1:5173and redirecthttp://127.0.0.1:5173/api/auth/callback/account. KeepPORT=8793,API_BASE_URL=http://127.0.0.1:8793, and host-local Postgres/S3 URLs. - From
api/, runbun run migrate, thenbun run dev. The API listens on 8793. - From
api/in a second terminal, runbun run worker. - From
app/in a third terminal, runbun install, thenbun run dev. Open http://127.0.0.1:5173; Vite proxies the API/auth requests to 8793, keeping browser authentication on one origin. - Sign in with your real approved account. No browser API URL/secret configuration is required. Fake OIDC credentials do not unlock the dashboard.
Use bun start instead of bun run dev for a non-watching host API. Both API and worker load api/.env when run from api/.
Use ordinary PostgreSQL, including managed providers such as PlanetScale Postgres. For remote Node/migration connections, require sslmode=verify-full and a verifiable certificate; do not disable certificate validation. Only localhost and the local Compose postgres hostname may omit TLS. Use your provider's actual Postgres connection URL, with URL-encoded credentials. For PlanetScale pooled URLs, follow its current pooler restrictions and use a supported direct connection for migrations if the pooler cannot support migration locking/session behavior. Do not substitute a MySQL connection string. See PlanetScale Postgres connections.
Attachments belong in a private S3/R2 bucket. Set the bucket, region, endpoint, and storage credentials for your provider; never put them in client-side environment variables. Local MinIO root credentials are development-only—not a recommended production access policy. Keep database and object-storage backups together. Object keys belong to the installation's data namespace; do not rename/move them casually during upgrades.
On upgrade, dashboard users sign in again and theme/workspace preferences reset once. Existing API keys, scoped OAuth grants, signed webhook secrets, and published domain-ownership proofs continue to work. New proof records use _bane; do not replace an existing _opensend record just to rename it.
Hosted received mail can use private Cloudflare R2 storage: MAIL is distinct from sending uploads. BANE_MAIL_STORE=r2 switches new shared-pool writes (and enables agent inboxes); unset it to resume S3 writes. Agent inbox mail stays in R2 while MAIL is bound. Keep the binding for existing downloads. SES staging (3 days) and quarantine (30 days) remain in S3. Non-inbox raw mail still expires after 90 days; extracted attachments do not expire. BYO and self-hosted mail stays in its configured S3 bucket.
Hosted BANE_* configuration variables accept the corresponding MX_* names as fallbacks. Set the new names for all new deployments.
wrangler.prod.jsonc contains the production bindings for https://bane.mx on the separate bane-production Worker. wrangler.dev.jsonc targets the bane-dev Worker. To self-host in another account, replace account_id, the custom-domain route and PUBLIC_URL, the Hyperdrive ID, and the R2/Queue names with your own resources; create the queues (wrangler queues create bane-work, bane-webhooks, bane-feedback-v2, bane-feedback-v2-dlq and bane-feedback-dlq) before the first deploy. On a fresh installation drop only LEGACY_FEEDBACK_QUEUE_NAME, LEGACY_FEEDBACK_CUTOVER and the original legacy consumer. Keep both archive-only DLQ consumers and their name/binding settings: each is the other's failure destination. The DispatcherShard Durable Object migration is applied by wrangler deploy; DISPATCH_SHARDS sets how many shards share each region's quota. Do not reuse production bindings for staging or preview builds. The dashboard build is served from app/dist as Workers assets on the API's origin.
Use a read/write database role for Hyperdrive and a separate schema-owner credential for direct Node migrations. Configure Hyperdrive origin TLS as verify-full; Cloudflare requires an uploaded CA certificate ID for that mode. The deployed connection uses the validated ISRG Root X1 trust anchor. Node migration connections accept PlanetScale's sslrootcert=system URI parameter without treating it as a filesystem path, while retaining certificate and hostname verification.
Disable Hyperdrive query caching with --caching-disabled; a shorter TTL is not equivalent. Authentication, consent, idempotency, and job state need fresh reads. Hyperdrive still provides pooling. See query caching.
Supply BANE_ROOT_SECRET and ACCOUNT_CLIENT_PRIVATE_KEY using Worker secrets; configure your real ACCOUNT_CLIENT_ID, allowlists, and HTTPS PUBLIC_URL for this deployment. This deployment also stores ACCOUNT_CLIENT_ID and the allowlists as Worker secrets rather than committing those installation values. For AWS discovery, explicit provisioning, or live SES sending, also supply AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and optional AWS_SESSION_TOKEN securely. Use interactive wrangler secret put --config wrangler.prod.jsonc (or the dev config), not secrets in config or command arguments. Pasted temporary credentials are not automatically renewed.
Build app/ with bun install and bun run build before bun run cf:check in api/. cf:check is a deployment dry run, not a deployment or remote-resource check. For local Workers, keep an ignored api/.dev.vars containing only the required Worker settings and secrets, including both OIDC credentials, allowlists, BANE_ROOT_SECRET, and the local PUBLIC_URL. This avoids loading unrelated Docker/storage credentials from .env. Securely set CLOUDFLARE_HYPERDRIVE_LOCAL_CONNECTION_STRING_HYPERDRIVE, then use bun run cf:dev -- --port 8794 --local-upstream 127.0.0.1:8794 --upstream-protocol http and a matching PUBLIC_URL/OIDC callback. The explicit local upstream prevents Wrangler from rewriting same-origin authorization headers to the production custom domain. Local Hyperdrive cannot certify production pooling/caching; see local development.
HTTP, Queue, and scheduled handlers share the outbox. Queue messages are wakeups; committed jobs survive a lost wakeup. Workers consume one wake signal per queue invocation and stop claiming new jobs after a two-second turn, while allowing every in-flight job to finish. Database claims lease only as many jobs as there are free local lanes; lanes refill individually as jobs finish, without waiting for the slowest job in a group. At most one feedback-first reservation is active per invocation, with feedback-only fallback when no send work exists. Invocation-local rotation provides job-class fairness without writing a shared scheduler row on every claim. The job_schedule row remains for admission locking and wake coalescing. A campaign launch asks for an uncoalesced wake so its run starts at once. Cron recovers due work and performs hourly retention. PostgreSQL transactions/leases, not wake messages or process-local state, control dispatch.
Apply migration 018_wake_coalescing.sql before deploying this Worker; singleton wake coalescing requires its nullable job_schedule.wake_not_before column. Singleton notifications share a one-second window per workspace and are delayed one second; larger bursts and queue/scheduled continuations remain ungated.
Inbound rollout: use /v1/inbound for addresses, threads, events and webhooks. For Node/Docker, stop old API and runner processes and apply migrations before starting this version. Migration 050_inbound_event_names.sql preserves event cursors, normalizes lifecycle names and pending deliveries, and removes message.updated from subscriptions. Existing addresses and keys do not change. A merge does not create an SES receiving rule, change MX, verify a forwarding destination, allowlist a webhook hostname, or register a webhook. Configure those separately using the support inbox setup. Before directing customer traffic to it, use an authorized external inbox to check a signed webhook, a reply, and the public From and thread in the recipient’s mail client.
Inbound recovery (migration 084): Apply the receiving IAM changes before cutover. From api/, node --import tsx scripts/reconcile-pool-receiving.ts --apply reapplies the Ohio lifecycle and SNS retry policy without clearing the pool’s ready state; it checks BANE_PLATFORM_AWS_ACCOUNT against STS and the existing binding. The hourly staging sweep lists staging/ for missed notifications older than one hour. Only the object-bound SES trace establishes a recipient; unaffiliated hosts are ignored and hosts being set up can hold mail for 48 hours. Inspect mail_receipt_holds.error, mail_receipts.error, and failed mailbox.recoverStaging jobs. Ambiguous objects go to private quarantine/ for 30 days when permitted. Published BYO v4 stacks need supplementary s3:PutObject on quarantine/?* and s3:AbortMultipartUpload on workspace raw keys; without the latter, the bucket lifecycle cleans abandoned uploads.
Select the matching hosted configuration explicitly. There is no self-hosted deployment workflow in this repository. Schema migrations run with the migration role, not as a side effect of Worker startup.
The XML-builder alias in Wrangler selects the AWS SDK's non-browser parser: Workers do not provide DOMParser. Outbound certificate, confirmation, and webhook requests use manual redirect handling and reject non-success responses without following redirects.
From api/ on Node 24, run bun install then bun run e2e with BANE_E2E_API_KEY in the environment. With a local env file, run node --env-file="$HOME/.config/bane/e2e.env" --import tsx scripts/e2e.ts. BANE_E2E_URL defaults to https://bane.mx. The full suite uses the dedicated unrestricted read/send/manage workspace key; it checks the Hosted connection, region and exact test domain ID before writing. Use separate E2E credentials for each hosted environment.
Each run sends to run-tagged SES simulator addresses and its own two temporary MX addresses. A second one-recipient campaign is scheduled 24 hours ahead, paused, paced, resumed and canceled before dispatch. The paused outbound webhook sends only a synthetic probe to https://httpbin.org/status/204; the inbound webhook sends only thread-state metadata from a run-owned address there. Neither subscribes to customer mail. The API records HTTP attempts, not request headers: this does not independently verify receiver-side HMAC. Cleanup targets only returned IDs. MX retains immutable mail, idempotency, consent/suppression and one-row import records; terminal campaigns are archived, not deleted.
e2e-report.json contains the commit, timings, failures with request IDs, and every OpenAPI operation either attempted, explicitly skipped with a reason, or not reached. Failed steps, cleanup errors and unaccounted operations fail the check. CI uploads the report even on failure and lists each step in the Actions summary. Domain-restricted denials on required flows are reported as failures, not skips. Budget: about four minutes including cleanup.
Dev fixture capacity. Every dev deploy runs scripts/dev-e2e-fixture.ts before the live E2E. On dev only, and only for the workspace named E2E that owns the fixture domain with both fixture claims verified, it sets a finite operator override of 100,000 recipients per rolling 24 hours (90,000 for marketing with the default reserve). Even one run a minute at the script's 25-admission ceiling stays under it. The first change writes one hosted_operator_audit row (set-dev-e2e-send-cap); later deploys change nothing. It never lowers a larger override and never touches an exempted workspace. Simulator recipients still consume capacity, and minute limits, pending limits, reputation holds and provider quotas still apply. The E2E key cannot grant this; the deploy does it with the stage's database credentials. It replaced a one-time 500/day setup that ran out on Nov 12.
POST /v1/emails/preview (previewEmail) previews HTML without sending mail, saving a draft or fetching remote assets. Use a read credential and supply exactly one of html or emailId, a nonempty list of distinct clients, and theme (light or dark). A stored outbound email must be visible to the caller under the same workspace, domain and content-redaction rules as getEmailContent; text-only or raw-only mail returns 422 EMAIL_HTML_UNAVAILABLE. The optional device selects the viewport width in CSS pixels: mobile is 390 and desktop is 800. By default, iOS, Android and Samsung Email use mobile; other clients use desktop.
The response contains previews in the requested order. Each has client, html, and compatibility issues. These are approximations, not screenshots: classic Outlook returns issues only (html: ""), and new Outlook for Windows uses the Outlook web model without verified Windows behavior. The input limit is 256 KiB of UTF-8 HTML; document complexity and total output HTML (4 MiB) are also bounded. A limit failure returns 413 rather than partial results. Treat preview HTML as untrusted: call iframe.setAttribute('sandbox', '') (or use <iframe sandbox="">) before assigning iframe.srcdoc. Never insert the HTML into the parent page. Its embedded Content Security Policy blocks remote assets; do not remove it or grant sandbox permissions.
Set the iframe's layout width, not just the width of its visible crop, to the requested device preset. Pasted markup and retained HTML use the same pure functions (emulate, lint in src/email-preview/index.ts). They reuse Nightmail's bounded parse5 tree builder: HTML5 table repair without a browser DOM, script execution, network requests or a new dependency. Links are inert. Remote and CID images show their blocked/alt state, and fonts use local fallbacks. Color remapping and mobile fitting are approximations; standalone HTML cannot identify repeated passages in a conversation. Don't treat these previews as client screenshots or use the transform to sanitize HTML for another purpose.
An issue's feature is a Can I email slug: error means reported unsupported, warning means partial or unmeasured, not that the layout necessarily breaks. lint considers all profiles; a preview's issues describe its selected profile. Features not detected or not covered by the data have no guarantee. To refresh the vendored MIT-licensed snapshot, follow scripts/refresh-email-support.rb (pinned upstream checkout and content digest, no runtime fetch). The version and support notes used for each cell are in src/email-preview/vendor/caniemail/evidence.json.
/openapi.jsoncomes from the actual Zod route schemas; stable operation IDs define generated SDK methods.bun run sdkexports the contract, generates../sdk/src, and compiles it. Do not edit generated code. See SDK usage.- The dashboard uses the same public API as integrations; OIDC establishes an administrator session, not a separate dashboard business API. API keys are hashed at rest and scoped to a workspace. New keys use
nm_<64 hex digits>(address and inbox keysnm_mbx_…); existingbane_,bane_mbx_,os_andos_live_keys remain valid. Retired test keys (os_test_orbane_test_) are rejected. Requests explicitly selecting the obsolete test environment fail with400 ENVIRONMENT_REMOVEDrather than sending real mail. A sending-domain restriction authorizes that domain and its subdomains, matching SES identity inheritance; a subdomain restriction does not authorize its parent or siblings. Give integrations only necessary grants; sending-domain restrictions are not a general data-isolation boundary. - Hosted MCP is served at
/mcpon this API's origin. MCP clients sign in at the account issuer (ACCOUNT_ISSUER), which issues them their own scoped access tokens for this resource. The default catalog groups operations by resource and action;/mcp?codemode=false(what OpenCode adds to a remote URL) lists individual operations with input and output schemas; no local process, pasted API key, or server-side code-mode wrapper is required. Each tool call acts in the workspace it names (workspace, else the connection's default) with the token's scopes intersected with the caller's role there, checked on every call. Writes require the corresponding OAuth scope andconfirm: true. - Collections return
{ data, nextCursor }. HTTP202means queued, not delivered. Read stored message/event state to find the outcome. - Domain-restricted keys can edit and preview untargeted drafts whose saved From address is on an allowed domain or subdomain. Once a campaign has an audience it is no longer visible to them. Selecting an audience, testing or launching a campaign, and reading delivery statistics require an unrestricted credential. Campaign deliveries are omitted from domain-key mail reads. Contacts, lists, segments, imports and AudienceQL also require unrestricted access (
403 UNRESTRICTED_KEY_REQUIRED). - This includes campaign test deliveries. Historical sends without recorded test provenance are visible only to unrestricted readers. A domain key may read or reuse an attachment it uploaded or one linked to direct mail it can read. Apply migration
150_attachment_uploader.sqlbefore deploying; no historical content is backfilled or deleted. - Mail sent from a receiving address or inbox (replies, new conversations, forwards and mail-client submissions) is conversation mail: a reply can quote the message it answers.
GET /v1/emails, its detail routes and Logs show it only to credentials that can read that mailbox's mail: dashboard sessions and unrestricted manage credentials. Other readers get404for it; their ordinary sends are unaffected. An upload that has gone out only in conversation mail follows the same rule:GET /v1/attachments/{id}, its content and attaching it to new mail answer404for those readers, unless they uploaded it or it is also attached to mail, a campaign or a review they can read. - Webhook URLs are write-only. API and MCP responses show a masked suffix, not a usable URL. Omit
urlon a settings update to keep the endpoint, or send a full HTTPS URL to replace it. Signing secrets use their existing reveal and rotation operations. - Use
Idempotency-Keyfor retriable writes: exact repeats return the recorded result; conflicting payloads returnIDEMPOTENCY_CONFLICT. Without a key, requests do not create permanent idempotency records.
Simulated mail always uses the full dispatch path and the real SES SDK with an in-process transport instead of AWS networking. Live mail never selects this transport. Defaults: SIMULATED_SES_LATENCY_MS=100, SIMULATED_SES_RATE=1000, and SIMULATED_SES_DELIVERY_DELAY_MS=250. Override the latency when modeling a different observed provider call; zero latency measures application overhead only.
Simulated dispatch verifies attachment bytes, constructs MIME, paces against the cached regional quota, claims consent/authorization, and stores acceptance atomically. It then emits two synthetic Send/Delivery callbacks that substitute for incoming SNS requests: on Cloudflare they travel the same feedback queue as verified live callbacks; on Node and in the acceptance fixture they are durable operation.simulatedFeedback jobs. Both paths run the same routine-ingestion statement, creating receipts when they arrive. Statuses progress through accepted/sent/delivered, but all records remain simulated: true and provider IDs start with sim_; they are not evidence of actual delivery. The public SNS endpoint still requires real signature/account verification. AWS-stored templates are explicitly rejected on simulated deployments; template discovery/rendering and AWS-side validation are not emulated. Simulated feedback uses the same receipt, status, and fan-out logic, but does not certify SNS signature verification or real mailbox delivery.
api/scripts/benchmark-sending.ts requires a dedicated local bane_perf_* database, all migrations, explicit ENABLE_LIVE_SES=false, and no AWS credentials. Run via node --import tsx scripts/benchmark-sending.ts with that isolated configuration. BENCHMARK_RECIPIENTS is capped at 1,000; BENCHMARK_REPLICAS, BENCHMARK_LANES, and BENCHMARK_CONNECTIONS control local load. Optional BENCHMARK_REPORT saves JSON metrics. Fixtures remain in the isolated database for inspection. Do not run it concurrently with an independent job runner. It measures the shared Node/database pipeline, not Cloudflare Queue startup or Hyperdrive latency; short high-rate runs do not establish sustained capacity.
Verified SNS callbacks never touch the sending jobs table. On Cloudflare the /v1/events/ses handler checks the SNS signature and account binding, tags the callback with an installation HMAC, enqueues it on FEEDBACK_QUEUE (bane-<stage>-feedback-v2) and returns 202. The tagged consumer requires a valid tag on every delivery. Failed tags, terminal ingest failures and exhausted retries go to operation_feedback_failures with reason FEEDBACK_QUEUE_RECOVERY, complete platform provenance, and lossless structured-clone payload encoding. Both native feedback DLQs have archive-only consumers and redrive storage failures to the other actively consumed DLQ; neither invokes ingestion. Only an unsigned message on the original legacy queue whose Cloudflare Message.timestamp is strictly before LEGACY_FEEDBACK_CUTOVER may enter the historical drain. Body timestamps, absent/malformed/future cutovers and replayed wrappers never qualify. node --import tsx scripts/replay-feedback-quarantine.ts lists recovery records; --apply <ids> rechecks the original installation tag or historical-recovery MAC and leaves unsigned/forged rows untouched. An archive signature proves the record was preserved, not that an untrusted body is valid feedback.
Queue cutover: apply migrations through 204 and create both v2 queues first. Fence old/foreign legacy producers, pause the old consumer, and drain in-flight old Worker invocations. Record the fixed UTC cutover in the stage config (for example 2026-11-12T18:30:00.000Z; do not use the current time on each deployment). Then deploy the tagged producer, both normal consumers and both archive-only DLQ consumers, and resume SNS delivery. The checked-in empty value quarantines all unsigned items until the operator records the cutoff. An age cutoff does not authenticate anything an attacker already injected before it: leave it empty and recover from original SNS evidence if that historical backlog is not trusted. Once the old queue has drained, remove its original consumer, name and cutoff; retain both DLQ archive consumers until their failure destinations have been replaced by another actively drained queue. Do not reopen or advance the cutoff for replay. Stage verification keys before switching signing roots, and keep PREVIOUS_BETTER_AUTH_SECRET until queues, DLQs, spill storage and recovery rows have drained all old-key items.
Historical recovery: before acknowledging an eligible unsigned legacy item, the consumer signs a domain-separated recovery envelope covering the full body, original queue name, platform message ID, enqueue timestamp and approved cutoff. It stores that envelope as FEEDBACK_LEGACY_RECOVERY, then sends it to the tagged queue. A transient ingest failure and later DLQ redrive preserve that signed provenance even though the new queue timestamp changes. If re-enqueue fails, the archive remains available to scripts/replay-feedback-quarantine.ts --apply legacy:<digest>. Successful ingestion removes it. The old shared 10,000-row rejection cap is gone; an untrusted backlog cannot consume a trusted recovery allowance. During a PostgreSQL outage, authenticated records spill to the private attachments bucket under feedback-recovery/v1/; the minute cron verifies each MAC and content digest, writes the complete record/chunks to PostgreSQL, then deletes the spill object. This also handles envelopes too large for a queue. Unresolved table rows and spill objects have no automatic expiry. Alert on FEEDBACK_RECOVERY_SPILLED, FEEDBACK_RECOVERY_DRAIN_FAILED, corrupt spill records, and DLQ backlog; repair storage before the platform's queue-retention deadline. Never manufacture an old timestamp for raw legacy DLQ data or treat an archive signature as provider authentication.
The quarantine listing returns up to 500 rows and a next cursor; pass --after <next> to continue. The untagged:, legacy: and queue: IDs all work with --apply, but only authenticated underlying feedback is ingested.
Routine Delivery (and any remaining Send) callbacks are persisted by one set-based statement per batch: receipt, matched event, monotonic status (the highest-ranked new event wins when several share a batch), public event and matching webhook deliveries. Bounce, complaint, engagement and subscription callbacks are processed inline with their receipt, and messages are acknowledged or retried individually. If a callback arrives before provider acceptance is recorded, its receipt and a deferred recovery job commit together. The configuration set no longer subscribes to SEND events—acceptance is recorded from the SendEmail response—so re-run provisioning after upgrading to update the event destination. The admission rate limiter does not apply to this endpoint. On Node the handler ingests routine callbacks inline and queues non-routine ones as jobs. With no matching subscribers, no webhook job is created.
MX offers the Nightmail mail template only when the domain’s discovered provider advertises support. GET /v1/domains/{id}/dns-setup is owner/admin dashboard only. It reads the domain once, then checks up to five DNS labels with a three-second total network budget. Each isolate caches up to 256 public results for five minutes; timeouts, rate limits and provider outages are cached for 30 seconds. Manual DNS stays available. Configure both DOMAIN_CONNECT_PRIVATE_KEY and DOMAIN_CONNECT_KEY_ID or neither: without them, provider information is returned without an apply URL. Incomplete settings or a non-RSA-2048 PEM fail configuration loading.
-
Outside the repository, generate an RSA-2048 PEM (
umask 077; openssl genrsa -out domain-connect.pem 2048). Set the full PEM including its original newlines as theDOMAIN_CONNECT_PRIVATE_KEYWorker secret, and setDOMAIN_CONNECT_KEY_ID=_dck1. Do not use the private PEM as DNS data. -
Publish two separate TXT records at the same owner,
_dck1.domainconnect.bane.mx.(relative name_dck1.domainconnectin thebane.mxzone). Take the base64 of the DER SubjectPublicKeyInfo, not the PEM text, PKCS#1 public key or a certificate. With 200-character chunks the values arep=1,a=RS256,d=<first 200 base64 characters>andp=2,a=RS256,d=<remaining characters>. No whitespace or line breaks inside a value. Compare the production public key in the Pulumi program with these values before enabling publication (one record per output line):openssl pkey -in domain-connect.pem -pubout -outform DER | openssl base64 -A | fold -w 200 | awk '{print "p=" NR ",a=RS256,d=" $0}'
-
Submit the template to Domain-Connect/Templates and onboard it with each DNS provider. Cloudflare: domain-connect@cloudflare.com, with the template link, full key TXT name, SVG logo and DNS-only proxy setting for DKIM CNAMEs. Vercel: domainconnect@vercel.com, with the template link and a short in-product demo video. IONOS: domain_connect_admin@domain.ionos.com, with the template link and signing-key hostname.
The synchronous protocol uses groupId (comma-separated; unselected groups stay in place). Nightmail sorts and signs the encoded query, excluding key and sig, then appends those two parameters in that order. A ten-minute signed return token is also in redirect_uri because Cloudflare ignores state. The callback checks the OIDC session, the original workspace’s current role and domain ownership before restoring that workspace and redirecting to ${PUBLIC_URL}/domains/<id>. A successful return or replay only queues the same throttled DNS/SES verification; neither grants ownership. For rotation, publish the new selector’s TXT records before switching the key pair and leave the old records through the outstanding flows’ lifetime and DNS TTL.
Hosted installations take payment for the prepaid balance through Stripe: Checkout top-ups, a saved card, invoice-backed off-session charges for saved-card top-ups and auto-recharge, and operator refunds. Stripe is only the payment rail; Nightmail keeps the balance and ledger and uses no Stripe Subscriptions, Billing or Meters. Use a Stripe account of its own for Nightmail: a shared account works (events without Nightmail's bane_attempt or bane_refund metadata are ignored) but mixes tax registrations and Radar rules. Self-hosted installations answer the billing routes with 404; a Hosted one without both keys answers the money routes with 503 BILLING_NOT_CONFIGURED.
- Key. Create a restricted key (
rk_…): write access to Customers, Checkout Sessions, Invoices (with Invoice Items), Credit Notes, Refunds and Payment Methods; read access to Payment Intents, Setup Intents, Disputes, Events and Radar early fraud warnings. Set it as theSTRIPE_SECRET_KEYWorker secret (wrangler secret put STRIPE_SECRET_KEY --config wrangler.prod.jsonc). A test-mode key (rk_test_…) only accepts test-mode events, and a live key only live ones. - Webhook. Add the endpoint
${PUBLIC_URL}/v1/billing/stripewith API version2026-10-28.endive(STRIPE_API_VERSIONinsrc/billing/stripe.ts; event payloads must match it) and these events:checkout.session.completed,checkout.session.async_payment_succeeded,checkout.session.async_payment_failed,checkout.session.expired,invoice.paid,invoice.payment_failed,payment_intent.succeeded,payment_intent.payment_failed,payment_intent.canceled,refund.created,refund.updated,charge.refunded,charge.dispute.created,charge.dispute.closed,radar.early_fraud_warning.created,payment_method.detached. Set its signing secret as theSTRIPE_WEBHOOK_SECRETWorker secret. - Tax.
STRIPE_AUTOMATIC_TAX(a plain variable, defaultfalse) turns on Stripe Tax for Checkout and the off-session invoices. Set it totrueonly once Stripe Tax registrations exist, and set the default product tax code (SaaS, business use) in Stripe. No code change is needed either way; prices are shown before tax. - Stripe dashboard. Turn on Email customers about successful payments: receipts come from Stripe. Processing fees are absorbed.
STRIPE_API_BASE exists for tests only (a fake Stripe); leave it unset.
How the money moves: Nightmail writes a payment attempt, commits, then calls Stripe with an idempotency key derived from the attempt (never inside a database transaction). Credit is applied only when Stripe reports the payment (the webhook, the inline off-session charge or reconciliation) and only once per attempt. Every minute the Worker queues due auto-recharges (enforcement on only); every five minutes it reconciles attempts left open for five minutes against Stripe, re-running unanswered charges with the same keys for 22 hours, then looking the invoice up by metadata. A declined or 3DS-requiring auto-recharge turns auto-recharge off and emails the owners once; there is no automatic retry. Alert on BILLING_AMOUNT_MISMATCH, BILLING_DISPUTE_OPENED, BILLING_EARLY_FRAUD_WARNING and BILLING_LATE_PAYMENT (errors) and BILLING_WEBHOOK_FAILED, BILLING_RECHARGE_FAILED and BILLING_RECONCILE_FAILED (warnings).
With ENABLE_LIVE_SES=false or missing AWS credentials, all keys use the simulator. SES discovery and provisioning return 503 SES_NOT_CONFIGURED. A campaign records its launch mode. If the worker mode changes before its run finishes, the run stops and the campaign fails with SES_MODE_CHANGED. Use a separate deployment for simulation: workspace data, consent and webhooks are real.
- Supply the explicit server-side AWS credentials above for a least-privilege IAM principal, never root or administrator keys. Set
DEFAULT_SES_REGIONbefore the first migration; region selection is persisted in PostgreSQL afterward. STS verifies the AWS account; provisioning records trusted account/topic identifiers in the database.SES_REGIONS,AWS_ACCOUNT_ID,SNS_TOPIC_ARNS, andSES_*_CONFIGURATION_SETare not normal installation inputs. - Set
ENABLE_LIVE_SES=true, read regional discovery, then explicitly request provisioning through the public API below. Provisioning creates two installation-scoped configuration sets (transactional/marketing), one SNS topic, their event destinations, and a signed HTTPS feedback subscription. Names are generated automatically. The feedback callback must be public HTTPS before provisioning. For local Docker, keepPUBLIC_URL=http://127.0.0.1:8793for OIDC login and setSES_FEEDBACK_URL=https://YOUR-TUNNEL/v1/events/ses. Route that path to the local API; OIDC’s localhost callback does not change. Recipient unsubscribe links still usePUBLIC_URL, so a local-only dashboard URL is not suitable for real marketing campaigns. For a fully public installation, omit the override and use a public HTTPSPUBLIC_URLwith its matching OIDC callback. - Verify regional SES identities and publish the DKIM records returned by SES, never guessed records. Review SES sandbox/production access and quotas separately; successful setup is not approval to send. See SES verified identities and configuration sets.
- Verify public HTTPS, private buckets, verified database TLS, disabled Hyperdrive caching where used, proxy/log-export redaction, and least-privilege database/IAM/SNS policies. Webhook endpoints must be public HTTPS DNS names: credentials, IP literals, internal TLDs and this installation's own hosts (
PUBLIC_URL,SES_FEEDBACK_URL) are rejected, and Node/Docker delivery resolves each host itself and refuses private, loopback, link-local, CGNAT and ULA answers at connect time (Cloudflare Workers cannot reach those ranges). - With explicitly authorized credentials and recipients, test a real OIDC consent/callback, a live SES send and received message, authentic SNS feedback, and public signed webhook delivery. Only then enable intended live workloads. Local mocks and dry runs cannot certify these integrations; no Cloudflare deployment or real OIDC/SES verification is implied here.
These are authenticated public API operations, not separate dashboard-only endpoints. Region access requires an unrestricted principal (no sending-domain restriction); AWS reads require read access, and configuration/provisioning require manage access.
| Operation | Behavior |
|---|---|
GET /v1/regions |
read access: database catalog of default/enabled regions, cached discovery status, and provision job status/errors. Never calls AWS. |
PUT /v1/regions/{region} |
Body { "enabled": true } and/or { "makeDefault": true }. Disabling the default, a region with queued/in-flight/uncertain mail or scheduled/sending campaigns, or active provisioning is blocked. Disabling deletes no AWS resources and preserves trusted feedback for historical mail. |
GET /v1/regions/{region}/discovery |
Enable the region first. Automatically discovers on the first uncached read, including for the default region; caches for 15 minutes. ?refresh=true forces AWS reads. Credential, PUBLIC_URL, or SES_FEEDBACK_URL changes invalidate the cache. AWS operations are read-only; reports and discovered DOMAIN identity names are saved in the local catalog. |
POST /v1/regions/{region}/provision |
Body { "confirm": true }. Returns 202 { jobId, status }, reusing an active job. The durable worker performs setup with a 90-second attempt budget and normal bounded retries. Read the catalog for job completion and refresh discovery afterward; a pending SNS confirmation is not ready, even if the job completed. |
Setup manages only resources tagged with this installation's bane:installation-id and bane:purpose=ses-feedback; it never automatically adopts manually created resources. Name/ownership collisions and missing permissions produce coded blockers, not permission escalation. SNS publishing is scoped to the verified account and the two configuration-set ARNs. The subscription uses signed SNS JSON with RawMessageDelivery=false; only the existing signature-verified callback confirms subscriptions for trusted topics. There is no public SubscribeURL trust override. See SNS delivery verification.
The managed destinations enable SEND, DELIVERY, BOUNCE, COMPLAINT, REJECT, RENDERING_FAILURE, DELIVERY_DELAY, OPEN, and CLICK. Open/click tracking still follows each message’s explicit tracking override; tracking: false disables it. SUBSCRIPTION is not enabled because Nightmail owns consent. Setup does not change DNS, obtain SES production/sandbox approval, mutate IAM, delete unrelated resources, add SQS/Lambda, or request mailbox access. Unrelated event destinations/subscriptions remain unchanged.
Use iam-policy.json for the dedicated Nightmail IAM user. Replace every YOUR_AWS_ACCOUNT_ID with your 12-digit AWS account ID, create a customer-managed policy in IAM, and attach it to that user. It covers discovery, provisioning, identity verification, custom MAIL FROM configuration, stored-template reads/rendering, and sending; it grants no IAM administration or resource deletion. The only storage access is to Nightmail's own bane-*-mail-* mailbox buckets.
Generate a personalized, gitignored copy with bun run iam-policy:personal -- YOUR_12_DIGIT_ACCOUNT_ID. This writes iam-policy-personal.json beside the canonical template for pasting into IAM.
Per-message open/click tracking overrides require ses:ApplyTrackingConfigurationOverrides in addition to ses:SendEmail. A live SES denial confirmed this permission on September 10, 2026, although AWS’s published service authorization reference did not list it. Keep the separate tracking statement in the generated policy. The subscription statement permits HTTPS callbacks at /v1/events/ses; you can replace its host wildcard with your exact public callback URL. Validate sending with an authorized recipient after applying the policy—read-only discovery cannot prove send authorization.
Review resource scopes and tag conditions for your account and generated regional names. The current commands in src/ses-setup.ts require these groups; this is an action inventory, not a complete verified deployment policy or the separate permissions needed for sending/identity management:
- Identity: STS
GetCallerIdentityverifies the caller/account. - SES reads (
ses:):GetAccount,ListEmailIdentities,GetConfigurationSet,GetConfigurationSetEventDestinations. - Reputation (
cloudwatch:):GetMetricDatareads the account-levelAWS/SESReputation.BounceRateandReputation.ComplaintRatemetrics. It does not support resource scoping. Without it, discovery still succeeds and the dashboard shows reputation rates as unavailable. - SNS reads (
sns:):GetTopicAttributes,ListTagsForResource,ListSubscriptionsByTopic,GetSubscriptionAttributes. - SES provisioning (
ses:):CreateConfigurationSet,CreateConfigurationSetEventDestination,UpdateConfigurationSetEventDestination, plusTagResourcepermission for creation tags. - SNS provisioning (
sns:):CreateTopic,SetTopicAttributes,Subscribe,SetSubscriptionAttributes, plusTagResourcepermission for creation tags. - Inbound (
ses:,s3:): enabling receiving on a domain (src/mailbox-setup.ts) uses SES receipt-rule actionsDescribeActiveReceiptRuleSet,DescribeReceiptRuleSet,CreateReceiptRule,UpdateReceiptRuleandDeleteReceiptRule, which do not support resource scoping. It joins the account's active receipt rule set and never creates or activates one: activation changes how the whole account receives mail, so when none is active setup stops withRECEIPT_RULE_SET_REQUIREDand the exact twoaws sescommands for an administrator to run with their own credentials. It creates anbane-*-mail-<region>bucket (CreateBucket,ListBucket,PutBucketOwnershipControls,PutBucketTagging,PutBucketPublicAccessBlock,PutBucketPolicy,PutLifecycleConfiguration) and reads/writes raw messages and attachments in it (GetObject,PutObject,DeleteObject). It reuses the SNS statements above for itsbane-*-inboundtopic and the/v1/events/sessubscription.
When the Docker job worker starts with AWS credentials, it queues read-only discovery for enabled regions with missing or stale reports. Repeated starts reuse pending discovery jobs. This never queues provisioning. The dashboard shows discovery progress and polls for its results.
Registered SNS feedback trust is bound to the provisioned AWS account and topic, not to short-lived API credentials: rotating access keys or session tokens does not drop feedback. Discovery observations are refreshed for changed credentials. Do not switch AWS accounts on an existing provisioned installation; use a separate installation instead.
Use the route schemas in src/ses-regions.ts for the exact API contract. The historical AWS reference provides background; its broader manual-configuration guidance is not an exact policy for this setup flow.
202 means queued, not delivered. The default email.rejected webhook now also reports local terminal outcomes: read data.status (suppressed, canceled, acceptance_unknown) and data.code. Do not resend acceptance_unknown: SES may have accepted it. Admission allows 15 final headers including threading and the two marketing unsubscribe headers; attachment MIME types and content IDs are limited to 78 ASCII characters.
email.bounced includes the SES bounce type. Only Permanent counts as a hard bounce; SES validation drops remain separate. Other bounces are recorded as non_permanent_bounce and cannot replace a delivered status. Only explicit not-spam complaint feedback (ignoring case and surrounding whitespace) is recorded as complaint_feedback without suppression or a complaint alert. These rules apply to new feedback; they do not clear prior suppressions.
Monitor SES_FEEDBACK_QUARANTINED: migration 087 retains isolated, deterministically invalid callbacks in operation_feedback_failures for 30 days. Its payload is private JSON text. Database outages and persistent lock conflicts return the message to the queue instead. A job whose final running lease expires remains inspectable as failed with JOB_LEASE_EXHAUSTED; the associated campaign or webhook delivery fails with it. A campaign cancellation takes effect immediately; its email records and webhook deliveries settle in pages of 1,000.
| Capability | Current boundary |
|---|---|
| Single / batch sends | Durable per-message jobs, at most 50 recipients/message and 100 messages/submission. Plain ASCII addresses (punycode domains supported), with an optional validated fromName for the sender display name; no arbitrary message headers. Each result has a message ID, not a separate batch-status resource. SES sends are individual, not SendBulkEmail. |
| Probation content | While a Hosted sender is on the probation tier, each message is delivered as the copy the content screen read, so what the recipient can read is what was screened. Style sheet rules (<style>, in the head or body) are first inlined onto the elements they select, by specificity and source order with !important honored, so class-styled fonts, colors, buttons and spacing survive; at-rules (@media, @supports, @font-face, @import) and pseudo-elements are not applied, and selectors beyond type, class, id, attribute, structural pseudo-classes (:first-child, :nth-child, …), :not() and the four combinators never match. The copy then keeps text, links, images (src, alt, size), tables and their presentational attributes (width, align, bgcolor, padding, borders) and the inline styles a client shows as text: fonts and sizes down to 8 px, colors, backgrounds (color only), spacing, borders, alignment, display as block, inline or inline-block. An element the light-scheme rendering does not display (display:none from its own style or an inlined rule, the hidden attribute) is left out with its content, so a dark-mode image swap resolves to the light image and a mobile-only block to its desktop state. Dropped: the sheets themselves, <link>, scripts, comments including Outlook conditional comments (VML buttons fall back to their HTML button), <noscript>, <template>, SVG, forms and controls, class/id/title, background images, list markers that spell anything (start, type, value, letter and roman list-style-type) and list items without text, everything that could draw boxes in another order than the document's (dir, direction, unicode-bidi, float, position, flex, grid, writing-mode), and the inline properties that hide or move text (visibility, opacity, overflow, clip, transform, filter, mix-blend-mode, negative margins and spacing, text-indent, mso-*, values using var(), calc() or url()). Text set in its background color is recolored, unknown elements show their text, and the bidi embedding, override and isolate controls that reorder what a client draws are removed from every delivered field: HTML, alt, subject, text part, preview text, display name and attachment names (which also lose every other format character). One hidden block stays: a preheader, when the first text of the message is a div or p its own style hides, up to 300 characters; campaign previewText is added back as Nightmail's own hidden preheader. Pictures carry words the screen cannot read, so images count only by their alt text and by the size a client draws them at (the larger of the width/height attributes and styles, with min-width/min-height dropped; an image without a declared size counts as full width) against the words beside them, counting only characters that draw (Hangul fillers and other default-ignorable code points do not): a 200 × 200 logo needs no words and every further 600 × 300 px needs 25, otherwise the message is rejected with CONTENT_IMAGE_ONLY. Attachments are delivered only when their normalized extension is a standard one (images, PDF, office documents, archives, audio, video, plain text) and the declared type (application/octet-stream meaning unspecified, delivered as the extension's type) and the first bytes agree with it (ATTACHMENT_CONTENT_MISMATCH otherwise); SVG, HTML, XML and mail archives, by type, name or content, and inline attachments that are not PNG, JPEG, GIF, WebP or BMP are rejected with ATTACHMENT_TYPE_UNSCREENABLE. PDFs and other click-to-open attachments are delivered as uploaded, since they open on a click and receiving systems scan them. The stored snapshot is the submitted message. Trusted-tier senders keep their message as submitted. |
| Campaigns | Versioned drafts, previews, a live audience count, scheduling, pause/resume/pacing and cancellation before dispatch. Draft html is block HTML (GET /v1/campaign-content-guide); the server validates it on save and renders the styled email plus a plain-text part at test/send, so the dashboard composer, SDK and MCP share one content document. Send and schedule freeze up to 2,000,000 matching contacts and validate every message in durable, bounded steps; poll the campaign's state. CSV imports are capped at 50,000 rows and 5 MiB per import. Larger input fails rather than truncates. Revocation cannot recall accepted or in-flight mail. |
| Templates | SES Get/TestRender/Get snapshots; callers must HTML-escape template data. The campaign editor escapes its own simple substitutions. Local test rendering is not proof of a real SES render. |
| Attachments | JSON/base64 upload plus raw-byte POST /v1/attachments/upload for the SDK/CLI, and authenticated /v1/attachments/{id}/content download with private object IDs and ownership/reference checks. One shared validator for every upload path (/v1/attachments, template assets, inbound replies): any file type Amazon SES delivers (its banned executable/script extensions are refused), base64 with or without line breaks or padding, and no MIME-type or file-signature second-guessing. At most 8 MiB decoded attachments, 16 MiB estimated encoded message size, and 512 KiB per direct body part. No arbitrary URL fetching. SES-rendered raw templates cannot add structured attachments. |
| Inbound | Messages up to 6 MiB and 30,000 lines are parsed for text/HTML bodies (capped at 1 MB each), up to 100 attachments, and threading headers. Larger or line-heavy messages up to 40 MiB are stored as downloadable raw mail (status: unparsed, bodyTruncated: true). Large messages use a bounded header read and server-side S3 copy; private headers are scrubbed before publication. Uninspectable oversized header blocks fail closed. A message read with includeHtml=true returns the HTML as stored and sets htmlTooComplex when it is past the parsing budget (20,000 nodes, 256 levels deep): show text instead, as the dashboard does. Catch-all auto-creation is limited to 50 addresses per domain in a rolling hour, 500 per domain and 2,000 per workspace; excess recipients are stored as unrouted. |
| Retries / rate | In-memory pacing per dispatcher shard from the cached regional quota. Rate throttling backs off 2s, 4s, 8s, 16s, 32s. An account pause brakes the shard for five minutes; mail held by that pause expires after 24 hours (SES_ACCOUNT_PAUSE_EXPIRED). Ambiguous acceptance is never automatically replayed. Outbound email webhooks retry transient failures up to ten times including the first attempt, over about 42 hours. Other orchestration jobs retain their six-attempt policy. |
| Webhooks | Ten event types, seven operational defaults, signed deliveries, encrypted per-endpoint secrets and custom request headers, pause/resume, and manual retries. Endpoint tests and manual retries share the inbound redelivery budget of 60 manual posts per workspace per hour (WEBHOOK_REDELIVERY_LIMIT). Any public HTTPS endpoint; IP literals, internal names, the installation's own hosts and (on Node) hosts resolving to non-global addresses are rejected. Paused local fixtures do not prove public delivery. A webhook that an MCP approval set up, changed or read the signing secret of follows that approval. While the approval is not active, its deliveries fail unsent (APPROVAL_INACTIVE) and can be retried by hand later. When the approval is removed, the webhook is paused and nothing is posted to it, endpoint tests included, until its URL is sent again (WEBHOOK_URL_REQUIRED otherwise); it then gets a new signing secret. |
| Unsubscribe | Immediate hosted GET and provider one-click POST; persistent opaque capabilities opt out of all installation marketing. Repeated clicks are idempotent. Link scanners can trigger footer opt-outs intentionally; HEAD returns 405 without changing consent. No preference-center UI. |
| Domain readiness | SES-returned DKIM zone is authoritative. Missing DNS is explicit; lists are capped at ten with paced control-plane reads. region is optional on creation (the workspace connection's region on MX, the default region when self-hosted). MX verification never gives up: paced checks back off to hourly for about three days, then every six hours, then daily; FAILED only means the SES identity is unavailable to this claim. Domain detail can configure a custom MAIL FROM subdomain with SES default-value fallback and then shows its MX/SPF records. Nightmail neither creates AWS connections nor replaces DNS records. |
| MX states | One explicit state per entity, written only by src/states.ts: domain claims carry sending_state (pending, verified, lost, failed) and receiving_state (off, pending, active, lost) plus the routing host; lost is a two-hour grace after proof goes missing. Connections carry state (awaiting_role, provisioning, ready, failed) and tenant (pending, ready, failed); Hosted is ready instantly, a background job creates the SES tenant, and until it is ready every send path (API, mailbox replies, campaigns) answers TENANT_NOT_READY and domain verification waits. Campaigns carry state (draft, preparing, scheduled, sending, paused, done, failed, canceled) with their delivery cursors as columns; see Large campaigns above. |
To authenticate an email or inbound webhook, include "headers": [{"name":"Authorization","value":"Bearer <endpoint-token>"}] in POST /v1/webhooks or POST /v1/inbound/webhooks. On PATCH, omitting headers preserves them only if the URL stays the same. Changing the URL clears old headers unless you supply replacements. Send [] to remove all headers. Current headers accompany every attempt, including retries, redelivery and email webhook tests. The signing protocol is unchanged.
Header names must be unique HTTP tokens (case-insensitive, 64 characters maximum); values must be HTTP field bytes (tab, visible ASCII or Latin-1 above 0x7F), at most 2,048 characters. Up to 10 headers and 8 KiB total (UTF-8 names + values). Host, Content-Length, Content-Type, Content-Encoding, Transfer-Encoding, Connection, Keep-Alive, TE, Trailer, Upgrade, HTTP2-Settings, Expect, Proxy-*, User-Agent, Webhook-Id, Webhook-Timestamp, Webhook-Signature and Nightmail-Test (also legacy Bane-Test and OpenSend-Test) are reserved. Cookie is also reserved so a server-to-server callback cannot forward a browser session cookie; use endpoint-specific credentials instead.
Migration 070_webhook_headers.sql adds nullable encrypted header storage; existing endpoints send no extra headers. Header values are write-only. Managers see {name,hint,updatedAt} (a masked suffix, fully masked for short values); unrestricted read API keys see {name}. Address-scoped inbound keys cannot inspect workspace endpoints. No custom headers are stored in delivery histories.
Errors use { error: { code, message, requestId, retryable, field? } } and an x-request-id header. Search JSON logs by that ID; dispatch carries it through the outbox. Logs include operation/job/attempt/error codes, not request bodies, credentials, or unsubscribe tokens. Unexpected stack traces remain server-side. API-key last-use timestamps update at most once per minute.
Detailed email/event/webhook logs expire after 30 days. Active jobs, campaign drafts, engagement, consent/suppression, idempotency results, and valid unsubscribe links have separate lifecycles. Unreferenced old attachment metadata is removed transactionally; object deletion is queued durably. Failed deletion jobs retain object keys for recovery. Contact deletion is not a promise to purge all audit/safety records, imported copies, or campaign revisions.
Hosted daily limits follow the sender score (how sending limits work): 100 recipients per rolling 24 hours while calibrating (until 100 real recipients were accepted and 24 hours passed), then 200 at a score of 65 up to 10,000 at 90, moving at most +5 or −10 a day. The ten-minute standing job (src/standing.ts, elected from the minute cron and the Node runner; the hourly maintenance runs it too) stores each workspace's score, limit and ladder step in sender_standing, and admission reads them in its existing statement. Default daily and minute budgets are shared by workspaces created by the same approved account, even after an ownership transfer. Explicit operator overrides remain per workspace and may expire. HOSTED_SEND_CAP_FIRST_WEEK is now only the limit of a workspace the job has not seen yet; the second-week and mature caps are retired. Live SES mailbox-simulator recipients consume capacity but never count as evidence. Suppressed mail and local simulated sends consume no daily quota. Marketing may use the limit minus max(100, 10%) of it (overrides keep the plain DISPATCH_DAILY_HEADROOM_PERCENT reserve). Campaigns wait for capacity.
Inbound defaults per workspace are 5,000 messages and 2 GiB per hour with a backlog of 5,000. Operators can set workspace-specific values in mail_receive_limits. MIME with excessive headers (64 KiB or 256 header lines), deep nesting, or an exhausted cooperative one-second parser budget remains available as raw mail. Inbound event long polls reserve one request-budget unit per requested second in addition to ordinary admission.
The reputation ladder runs with the standing job: notice, warn (limit frozen, owners emailed), restrict (marketing held with 409 MARKETING_HELD, campaigns pause, transactional mail flows) and pause (409 HOSTED_SENDING_PAUSED on admission and dispatch). Each line needs a count and a rate over the last 24 hours of hard bounces and complaints per destination against 48 hours of accepted volume, or 7 days against 8; the thresholds are in sending-limits.md. Bounces alone hold marketing; they pause only when transactional mail itself bounces 5% and 50. Content-screen verdicts are counted when the screen judges a message, the dispatcher's or, for a campaign recipient's copy, its release's (hosted_screen_hourly, real recipients only, each recipient once), so spam admitted before a resume still counts after it. A pause queues hosted.tenantStatus, which sets the workspace's SES tenants to DISABLED (UpdateReputationEntityCustomerManagedStatus) as a backstop. Queued mail of a workspace paused for 72 hours is canceled (ACCOUNT_PAUSED). Every change of standing writes standing_events, publishes account.standing.changed, and at warn and above emails owners and admins from BANE_NOTIFY_FROM (an address on the domain of BANE_PLATFORM_DOMAIN_WORKSPACE, sent through that workspace; unset, the notice is dashboard- and webhook-only). Inspect STANDING_CHANGED logs, connection.standing on GET /v1/me and standing on GET /v1/sender-score.
What counts as marketing is decided by Nightmail, not only by the caller's kind: a message labelled transactional with 10+ real recipients, list headers (List-Id, other List-*, Precedence: bulk/list/junk) or 10+ identical copies in one request is stamped snapshot.bulk and treated as marketing for holds, the transactional reserve, the dispatch fence and reputation attribution (hosted-sending.ts bulkFlags). It keeps its stream and content. Internal: not in the public docs.
An exempt (first-party) workspace (set-workspace-exemption.ts, operator-only and audited) is never held or paused by the ladder: its standing is computed as usual but stops at warn, so it keeps sending transactional and marketing mail. Crossing a restrict or pause line warns: the owner email and account.standing.changed at warn, and a reputation.alert to the operators (HOSTED_EXEMPT_OVER_LINE, REPUTATION_ALERT_SLACK_URL, once per line a day). Only an operator pauses it, with a reason (the operator API, or scripts/pause-hosted-sending.ts). AWS's own tenant pauses still apply, and alert the operators too.
Nightmail is the source of truth for the tenant status. Every pause and resume bumps provider_connections.tenant_generation, and hosted_tenant_status records what was applied to each regional tenant at which generation. The job applies what Nightmail wants now, region by region, and runs again if a transition happened meanwhile. A disable is journaled (DISABLING) before AWS is called and ENABLED is recorded only after AWS confirms, so an outcome lost between AWS and the database counts as possibly disabled and a resume enables it again; every standing run queues any workspace still out of line (tenantStatusDrift), including a tenant that became ready after the pause. The runtime IAM policy must allow ses:UpdateReputationEntityCustomerManagedStatus on the workspace tenants; until it does, the job logs SES_TENANT_STATUS_DENIED and records nothing (the database hold applies either way).
After reviewing the cause, an operator resumes through the operator API (below) or with DATABASE_URL set runs node --import tsx scripts/resume-hosted-sending.ts WORKSPACE_ID --reason "why" from api/. Queued mail can dispatch again, and only events after the resume count toward another pause. A resume keeps the score and limit and records a strike. For 48 hours the limit is min(current limit, 50% of the limit before the resume), operator overrides included; it is stored as an absolute cap, so a second resume in the window cannot raise it (nextCap/nextCapAt on GET /v1/me say when the rest returns). --restore 100 (restore: 100 in the API) opts out and is audited. A pause whose SES tenants were disabled stays held, with a "resuming" reason, until the tenant job has enabled them again. A marketing hold placed by the hourly check before the standing shipped (provider_connections.marketing_held_at) becomes a restrict on the next standing run. Content-screen rejections count against the daily limit like sends; other rejections are refunded.
Operators (Nightmail staff). Platform roles (operator, operator:read) live in platform_operators, never in a workspace: grant one with node --import tsx scripts/set-operator.ts EMAIL operator --reason "why" (off removes it; the user must have signed in once). Operators use the /operator area of the dashboard, backed by /v1/operator/* (hidden from the public OpenAPI document and SDK): the review queue, a workspace page, the evidence timeline, pause, resume (50% or 100%), marketing hold, campaign release, limit overrides with an expiry (an override ends a resume's reduced limit), and the audit log. Each change needs a reason (10+ characters), takes an optional ticketRef, Idempotency-Key and If-Match (the standing version), and writes hosted_operator_audit (operator, via, reason, ticket, request ID, before and after) in the same transaction. Identity is the signed-in dashboard user, so Anomaly sign-in (OIDC) replaces Google sign-in here without an API change.
AWS tenant pauses (EventBridge). Nightmail mirrors AWS's own tenant pauses: an SES Sending Status Disabled event from AWS (or from the strict reputation policy) delivered to the pool's feedback SNS topic pauses the workspace with SES_TENANT_PAUSED; only an operator resumes it, after AWS reinstates the tenant. Nightmail's own switch ("Status manually updated.") and every enable are timeline entries only. Infrastructure follow-up: an EventBridge rule on the default bus in each pool region ({"source":["aws.ses"],"detail-type":["Sending Status Disabled","Sending Status Enabled"]}) targeting that region's Nightmail feedback topic, whose topic policy must allow events.amazonaws.com to publish.
Every operator script that changes a workspace (pause-hosted-sending, resume-hosted-sending, set-hosted-limits, set-workspace-exemption, set-domain-pool, adopt-platform-domain, adopt-identities and rehome-hosted-account's --apply, --restore and --finalize) requires --reason (at least 10 characters), refuses before any change without one, and writes it to hosted_operator_audit with the change. Dry runs and plans need none.
Before moving an existing production sender onto Hosted, compare its peak rolling-day volume with the workspace's effective cap and queue limit. Apply the workspace override below before cutover if the signup tier is too low; do not raise the shared defaults for unrelated tenants.
To change a workspace’s Hosted limits, run node --import tsx scripts/set-hosted-limits.ts WORKSPACE_ID --send-cap 120000 --pending-cap 25000 --reason "Approved migration volume" with the same operator access after migration 062. The first cap is real recipients per rolling 24 hours (1–10,000,000); at the default 10% headroom this example reserves 12,000 recipients for transactional mail. The second cap is outstanding direct emails per stream (transactional and marketing are counted separately; 1–4,000,000). Use default to clear either override. For ordinary workspaces this restores the age tier or the 10,000-message floor, respectively. HOSTED_LIMITS_CHANGED prints the old and new overrides (null means default); the audit row keeps them with the reason. Overrides survive reconnects and do not apply to BYO.
After migration 082 and the matching API and runner rollout, use a short-lived operator DB role and run node --import tsx scripts/set-workspace-exemption.ts WORKSPACE_ID on --reason "Approved sending volume" from api/. Use off with a reason to remove it. For a Hosted shared connection only, on removes the default daily recipient cap, signup ramp and workspace marketing reserve, and raises the default direct pending limit to 200,000 per stream. Explicit numeric overrides still win; default restores the applicable exempt or ordinary limit. Usage continues to accrue while exempt: off applies the ordinary policy to the existing rolling window. The command records the reason and old/new state in hosted_operator_audit and logs HOSTED_EXEMPTION_CHANGED. It does not change SES account quota or headroom, reputation holds, suppression, placement, or any other limits. Campaign bounce/complaint auto-pause is mandatory for all Hosted shared senders.
Lift a bounce suppression (operator only). A hard bounce or validation drop suppresses the address for the workspace with no public way back, which locks a person out of sign-in mail even after their mailbox exists. With a short-lived operator DB role and AWS credentials for the Hosted pool account, run node --import tsx scripts/lift-suppression.ts WORKSPACE_ID ADDRESS --reason "..." from api/. Before changing anything it checks the credentials' account in every Hosted region and reads the address (as given and lowercased; SES matches case exactly) on the workspace's SES tenant lists and on each account-level list. It removes the workspace's own tenant bounce entries. The account-level list is shared by every workspace sending from that account (the probation pool suppresses at account scope), so a bounce entry there stops the command until you rerun it with --account-level; a complaint entry on any list, or a complaint in Nightmail, is never lifted. hosted_operator_audit gets a started row before the first SES change and a lifted, failed (with the removals already made) or superseded row after; rerun after a failure to finish. The removals and the clear run under the contact's row lock, which feedback ingest needs to record a bounce or complaint for the address; Nightmail's suppression is checked again under it and each SES entry is read again just before it is removed, so a complaint or newer bounce seen at any point stops the lift (superseded) and is kept. It does not change reputation counters, and a still-invalid address bounces and is suppressed again.
Drain mail writers before applying 062: it backfills recent mail and feedback while holding write locks. Hosted windows use UTC hour buckets (the current hour and the preceding 23 or 47); an admission expires at its hour start plus 24 hours, up to 59 minutes earlier than an exact rolling cutoff. Suppression and definitive pre-acceptance failure refund the original hour. Reputation resumes still exclude everything through the exact review timestamp.
Re-home a legacy Hosted Ohio installation (operator only, migrations through 092). Keep production on the pre-rename release until the switch. Never deploy the renamed release against the old account: it resolves different pool names. The re-home script never calls AWS or DNS. Take a restore-tested encrypted backup and an object manifest. Verify destination IAM, SES production access, quota, active receipt-rule set and the HTTPS subscription endpoint. Save receiving hosts, custom MAIL FROM settings and account/tenant suppression lists.
MCP clients authorised against https://opensend.mx/mcp must reconnect to https://bane.mx/mcp.
-
Block new admissions on both origins; hold incoming SMTP upstream, disable the old catch-all and drain old sends, campaigns, receipts and archives. Keep signed outbound feedback and unsubscribe working. After draining, pause every connection, retaining previous holds. Fence old HTTP writers, cron, queues, runners and deployments; wait for outstanding work to exit. The advisory lock is not a Worker maintenance gate. Do not switch to simulated sending.
-
With the pre-rename release fenced, run
bun run migratefrom the newapi/checkout using a short-lived operatorDATABASE_URL, then plan the re-home. Running migration 092 before the switch is safe here only because production has no stored MCP grants: the pre-rename Worker cannot readbane:*scopes. With live grants, run 092 together with the renamed release. Supply every workspace ID, including unconnected sign-ins anddefaultif present. Review connected IDs, unconnected workspaces, receiving hosts, counts, database identity and digest. All running jobs and pending reputation alerts must drain. Pending outbound feedback/webhook/publication jobs may accumulate while consumers are stopped. Apply only the reviewed digest, with writers still fenced:cutover=(node --import tsx scripts/rehome-hosted-account.ts --run-id "$RUN" --region us-east-2 --old-account "$OLD" --new-account "$NEW" --workspaces "$WIDS" --old-bucket "opensend-mx-$OLD-mail-us-east-2" --new-bucket "bane-$NEW-mail-us-east-2") "${cutover[@]}" --plan "${cutover[@]}" --apply --expect-plan-sha256 "$REVIEWED_HASH" --reason "Account re-home $RUN" # Alternative: rollback BEFORE starting new-account work, with writers still fenced. "${cutover[@]}" --restore "$RUN" --reason "Re-home rollback $RUN"
-
At the switch, with sending paused and drained, deploy the renamed release for the first time together with the new runtime key pair,
BANE_PLATFORM_AWS_ACCOUNT=009882681002and the production bindings fromwrangler.prod.jsonc. Keep the existing database and encryption secrets. Do not allow old binaries to run against reset identities. Verifier jobs target the new account. Tenant jobs are deliberately not queued under the old deployment: their retry budget can expire before the new pool exists. With the new runtime environment, trigger the same pool-then-tenants reconciliation as minute cron fromapi/, then start new job consumers:node --import tsx --input-type=module <<'RECONCILE' import { Client } from 'pg'; import { drizzle } from 'drizzle-orm/node-postgres'; import { postgresConnection } from './src/adapters/node.js'; import { loadConfig } from './src/config.js'; import { reconcileSharedConnections } from './src/provider-connections.js'; const config = loadConfig(process.env); const client = new Client(postgresConnection(process.env.DATABASE_URL)); const unused = async () => { throw Error('Reconciliation must not use application storage'); }; await client.connect(); try { await reconcileSharedConnections({config, db: drizzle(client), storage: {get: unused, put: unused, delete: unused}}); } finally { await client.end(); } RECONCILE
No application
S3_*variables or R2 binding are needed. Run a second pass after the live Worker confirms the SNS subscriptions at/v1/events/ses; a successful command exit alone does not mean the pool is ready. Immediate restore requires the expected post-apply state and original inputs; restore the pre-rename release and old account configuration together, while fenced. After reconciliation or new mail, use a separately drained reverse migration instead. -
Verify both pool topics, the catch-all, new bucket and tagged probation tenants. Copy and verify the manifest's workspace raw/attachment objects at the same keys (preserve metadata, tags and original expiration deadlines) before downloads. Keep the old bucket. Publish returned DKIM CNAMEs, retain claim TXT and old selectors, restore MAIL FROM and suppression lists, and verify domains again after pool receiving is ready.
-
Check prior receiving hosts and mailbox IDs, pre-cutover downloads, and controlled inbound/outbound/feedback canaries. Release only this run's uniquely named hold:
UPDATE provider_connections SET paused_at=NULL,pause_reason=NULL WHERE kind='shared' AND pause_reason=:hold. Do not useresume-hosted-sending.ts: it changesresumed_at, resetting the reputation review window. Keep the old outbound SNS route directly served. After the rollback window, run"${cutover[@]}" --finalize "$RUN" --reason "Re-home finalized $RUN"to delete that run's before-images; the hash and timestamps remain and automatic restore becomes impossible.
Non-manage readers receive app unsubscribe capabilities redacted from content and nested event/webhook payloads; raw MIME is withheld. This does not sanitize arbitrary third-party password-reset links—content-read grants belong only with trusted integrations. Campaign substitutions reject executable, unquoted, comment, and foreign-markup contexts and validate completed URLs. SES templates retain their caller-escaping contract.
Postgres request/resource budgets bound major amplification paths. Authenticated credentials default to 1,200 requests/minute. Outstanding (queued or attempting) direct email is capped per workspace at the account's SES 24-hour quota, between 10,000 and 200,000 (shared-pool workspaces default to 10,000); there is no per-key cap. Stored attachments are limited to 1 GiB per workspace and expanded batch content to 128 MiB. These are implementation limits, not plan entitlements or full isolation from shared infrastructure costs.
Coarse admission runs before DB access: 6,000 requests/minute per IPv4 or IPv6 /64 per Node process or Cloudflare location (approximate). SNS has a separate Cloudflare lane at the same limit, and its signature is verified before looking up the topic. An isolate/process reserves up to 40 public credential attempts per peer per minute before dispatch; successful authentication refunds the reservation. Public auth entry points have their own 40/minute lane. Node does not trust forwarded-IP headers. Use correctly configured proxy/WAF controls for distributed abuse. Cloudflare invocation URL logs/traces are disabled; review every external log sink independently.
Every account has one append-only trail, audit_events (migration 260): who acted (a member, an API key, an agent (its MCP sign-in or an agent token), an operator, a support grant, Nightmail itself or Stripe), in which workspace and for which user, the action (api_key.create, member.role_change, domain.delete, support.view…), the target, a before/after summary, the reason and ticket, the request ID, and the client's network (/24 or /48) and user-agent family. Summaries never hold secrets (key values, hashes, passwords, webhook secrets or URLs) or message content.
- Immutable in SQL: triggers refuse
UPDATE,TRUNCATEand anyDELETEbeforeretain_untilfor every role, the superuser included. A role that owns the table (or holds DDL on it) can still disable the triggers, so the API must connect as a non-owner role withSELECT,INSERTandDELETE(the purge) only; check it withSELECT privilege_type FROM information_schema.role_table_grants WHERE table_name='audit_events' AND grantee=current_user. The hourly maintenance purge deletes expired rows in pages. - Retention: 1 year by default (it covers card dispute windows and an annual review), 7 years for billing and payment events, 90 days for the per-call log of a support grant.
- Writers:
recordAudit(tx, c, event)inside the change's own transaction, orauditInsert(...)composed into the change's single statement (no extra round trip). Send admission never writes here; sending activity is in the sending logs. Operator scripts, the operator API, hosted pauses and limit overrides reach the trail through SQL triggers (migration 262). - Readers: owners and admins see their workspace's customer-visible rows in Settings → Activity (
GET /v1/activity). Operators see everything throughGET /v1/operator/workspaces/{id}/auditandGET /v1/operator/users/{id}/audit(filter bysince,until,action,actorType,actorId,requestId; cursor pages;include=enforcement,billingjoins the standing events and billing ledger where those tables exist). Each operator read is itself written to the trail.
Nightmail staff with the operator role leave design and bug notes on the dashboard itself (hosted only), and agents fix them.
-
Leaving notes: turn on UI notes in the account menu or with
⇧N, then ⌘-click (Ctrl-click off a Mac) any element, or long-press it on a phone. Each note records the page and route pattern, viewport, theme, device, build, a locator (CSS path, test IDs, the nearestdata-component, text), the element's box and computed styles, and a screenshot crop with the element outlined. Addresses are masked in all of it, the screenshot included. Open notes show as numbered pins on the page;/operator/noteslists them all. -
Storage:
ui_notes(migration 304), global to the deployment. Screenshots are private objects in the attachments bucket (ui-notes/<note id>), served only through signed links that expire after five minutes (GET /v1/ui-notes/screenshots/<token>). Every change is an internal audit row (ui_note.*,ui_note_key.*). -
Agents: an operator makes a UI notes key for themself on
/operator/notes→ Agent keys (up to 90 days). It can only list and read notes and set their status, PR link and resolution, and only while that person still has the operator role. Every other operator route still refuses any bearer token. Use it withscripts/ui-notes.ts:BANE_URL=https://bane.mx BANE_UI_NOTES_KEY=nm_notes_… node --import tsx scripts/ui-notes.ts list --status open --json --screenshots ./ui-notes node --import tsx scripts/ui-notes.ts get <id> --screenshot ./note.webp node --import tsx scripts/ui-notes.ts set-status <id> in_progress node --import tsx scripts/ui-notes.ts set-status <id> fixed --pr https://github.com/anomalyco/nightmail/pull/<n>
Internal: these routes serve Nightmail's operators and its support platform, never customers. /v1/operator/* answers only an operator and /v1/support/* only a support grant; every customer credential (any dashboard role, API keys, MCP grants, agent tokens, inbox keys, app passwords) gets the same 403 with no data on every path there, real or not. Neither appears as an MCP tool, and /v1/support/* is marked x-internal in /openapi.json and left out of /docs, /docs.md and /llms.txt. It stays in the SDK for the support platform.
Configuration (hosted only):
| Variable | What |
|---|---|
BANE_SUPPORT_URL |
Where "Contact support" sends a member (https). The handoff rides in the fragment: #handoff=<token>. |
BANE_SUPPORT_SIGNING_KEY |
Ed25519 private JWK (kty, crv, d, x) that signs handoffs. Generate one with node -e "console.log(JSON.stringify(require('crypto').generateKeyPairSync('ed25519').privateKey.export({format:'jwk'})))". A secret. Public keys: GET /v1/public/support-keys. |
CF_OBSERVABILITY_TOKEN |
Cloudflare API token that can read Workers Observability on this account (see below). A secret. Without it (and on Node), GET /v1/support/requests answers available: false, reason: not_configured. |
CF_OBSERVABILITY_ACCOUNT_ID, CF_OBSERVABILITY_SCRIPT |
The account and this Worker's script name whose request lines support reads (set in wrangler.{dev,prod}.jsonc). |
The support platform's access: a service binding, no token. The support Worker (support/, bane-support / bane-support-dev) calls the SupportPlatform entrypoint this Worker exports (src/worker.ts, logic in src/support-platform.ts) over a Cloudflare service binding in the same account: { "binding": "BANE", "service": "bane-production", "entrypoint": "SupportPlatform" }. Only a Worker whose deployed config declares that binding can call it, so there is nothing to leak or rotate; removing the binding or the entrypoint revokes it. Its methods are the whole surface: verifyHandoff(token) (Nightmail checks its own signature, audience, issuer and expiry, then the stored row: used or not, the person's message), verifyEmail, mintGrant, revokeGrant, verifyStaff(token) (below) and read(path, grant) for GET /v1/support/*. They run through the normal routes (validation, audit rows, budgets) as the operator service:bane-support, on their own admission lane, and only on /v1/operator/support/{verify-email, grants, grants/{id}/revoke, handoffs/{id}, staff/verify}: every other operator route refuses it, and no bearer token is an operator. (BANE_SUPPORT_PLATFORM_TOKEN is retired: delete the secret from any Worker that still has it.)
Staff sign-in (the support Worker's internal ticket pages). The page sends a Nightmail operator to GET /v1/operator/support/staff-sign-in?return=<support origin>/staff/callback (an operator's own dashboard session only; any other return is 422 RETURN_URL_INVALID). Nightmail writes a support.staff_sign_in audit row and redirects (302) to return?staff=<token>: an EdDSA JWS with the support signing key (aud: bane-support-staff, sub the operator's user ID, email, role, a random jti, 2 minutes). The support Worker calls verifyStaff(token) (POST /v1/operator/support/staff/verify, the binding only): { ok: true, staff: { userId, email, role }, jti, expiresAt } while the subject is still an operator, else { ok: false, code: STAFF_INVALID | STAFF_EXPIRED | NOT_OPERATOR }. Nightmail keeps no record of used tokens: the support Worker accepts each jti once, until expiresAt.
The flow. The agent never chooses or asserts who the customer is; the support platform's server does, then hands the agent a grant bound to that one user.
- In-app: "Contact support" calls
POST /v1/contact-support(dashboard session, any role) with the page, recent request IDs, an optional message and an off-by-defaultshareContent. Nightmail stores a handoff (sho_…) that opens one grant within 30 minutes, and returns the support URL with an EdDSA JWS (10 minutes,aud: bane-support, claims: workspace, user, role, page, request IDs,shareContent). The platform verifies it with the public keys, or reads it back withGET /v1/operator/support/handoffs/{id}(which also shows whether it was used). The platform should treat the JWSjtias single-use too. - By email: Nightmail does not see the email: the support platform is responsible for verifying the sender (a DMARC pass aligned with the From domain, never a display name or a forwarded copy, or a magic link the person opened). It then calls
verifyEmail(POST /v1/operator/support/verify-email) with the address and that evidence:dmarcneeds the verified message'smessageId, and amagic_linkconfirmation must be from the last day and not in the future. Both are recorded. Nightmail answers with the user and their workspaces only for a verified email of a signed-in user. If there are several workspaces, the platform asks which one. - Mint:
mintGrant(POST /v1/operator/support/grants) withuserId,ticketRef,reason,verification(handoffwith itshandoffId,dmarcormagic_link), optionalworkspaceIdsandttlMinutes(≤ 120). It returns anm_support_…token once. A handoff grant covers the handoff's workspace unlessworkspaceIdssays otherwise, and using the handoff marks it used: a second mint gets409 SUPPORT_HANDOFF_USED. One active grant per ticket, at most five per user. - Use: put the token in the agent's environment only. Every
/v1/support/*route derives the customer from the grant;workspaceIdcan only narrow it to one of the grant's workspaces (the ones the user still belongs to). Grants are read-only (GET), limited to 120 requests a minute (a deliverability or request read below counts as 5), and expire within 2 hours.POST /v1/operator/support/grants/{id}/revokeends one at once.
What a grant reads (compact JSON, cursor pages where lists are long):
| Route | Returns |
|---|---|
GET /v1/support/context |
Start here: user, workspaces and roles, sending and limits, domains, failures in the last 24 hours, open enforcement, the last 20 audit entries, the handoff and request IDs to drill into |
GET /v1/support/workspaces |
Workspaces with role, sending connection and resource counts |
GET /v1/support/domains |
Domains with DKIM, MAIL FROM, DMARC and receiving state |
GET /v1/support/emails, /emails/{id} |
Sent or received mail: addresses, subject, status, error codes, delivery events; the body only with the customer's opt-in |
GET /v1/support/events |
Delivery events (bounce, complaint, reject…) |
GET /v1/support/webhook-deliveries |
Webhook deliveries with attempts, last status and error; never payloads or URLs |
GET /v1/support/standing |
Pauses and reasons, limit overrides, sender standing, the last 24 hours by status, 60 days of daily score, limit and step, and whether a limit request (requestedLimit) would be approved automatically |
GET /v1/support/billing |
Balance and ledger of workspaces the user owns or administers (when billing is enabled) |
GET /v1/support/audit |
The customer-visible audit trail of workspaces the user owns or administers |
GET /v1/support/requests |
The customer's own failed requests (status=error ≥ 400, 5xx or 4xx; since/until within the last 3 days, default 24 hours; optional route, code) from this Worker's request lines in Workers Logs: up to 50 groups by route template, method, status, error code and principal type, with counts, first and last seen and 5 request IDs. Counts as 5, cached 60 seconds |
GET /v1/support/requests/{requestId} |
One request: its request line (route, status, code, duration) from Workers Logs, the audit entries, campaign launches and routing versions it wrote. Counts as 5 |
Request logs (src/support-logs.ts). Both request reads ask the Workers Observability telemetry API (POST /accounts/{account}/workers/observability/telemetry/query, events view, 5 seconds, at most 1,000 lines) for this Worker's event: request lines whose workspace is one of the grant's, then drop any returned line outside those workspaces and keep only route template, method, status, code, duration, principal type, workspace, time and request ID: never messages, addresses, user agents, URLs or bodies. The support grant's own reads and operator routes are left out. A Cloudflare error answers available: false, reason: unavailable (logged as SUPPORT_LOGS_UNAVAILABLE with the HTTP status). The token: an account-owned API token per account with the Workers role Metadata Read-Only on this one Worker (bane-production or bane-dev), which replaced the legacy Account → Workers Observability → Read that the nightly report's token uses for the same query. The API reference lists Workers Observability Write for this endpoint: check a new token with one query before relying on it.
Deliverability reads (src/support-deliverability.ts, aggregates in src/deliverability.ts). Each reads one workspace (workspaceId, or the grant's only one), counts as 5 requests, runs in one read-only transaction whose statements stop after 3 seconds (422 SUPPORT_READ_TOO_LARGE: narrow the read), and is cached 60 seconds per workspace, route and parameters (computedAt). Windows are at most 30 days, hourly at most 72 hours; at most 20 groups plus other and 50 rows. Recipients appear only as domains and providers, diagnostics are redacted (<address>, <ip>), and content only with the customer's opt-in. Each returns findings[]: heuristics with a severity, a code, plain words and the numbers behind them.
| Route | Returns |
|---|---|
GET /v1/support/deliverability/metrics |
Per day or hour, optionally per campaign, From domain, recipient provider or stream: accepted, delivered, bounces, complaints, opens (and human opens), clicks, unsubscribes, screen refusals, each rate with numerator and denominator |
GET /v1/support/deliverability/bounces |
Bounces, deferrals and complaints by provider, type, subtype, enhanced status and redacted diagnostic, with the campaigns behind them; the first-send bounce rate |
GET /v1/support/domains/{id}/health |
The stored verification, one live DNS read (SPF, DMARC, BIMI, the MAIL FROM's MX and SPF; 5 labels, 3 seconds, cached 5 minutes), the alignment verdict, 30 days of volume and warm-up age, the marketing share and bulk-looking transactional mail |
GET /v1/support/screen |
The Hosted content screen over 7 days: hourly counts and the last 50 refusals with subject, code and spam score |
GET /v1/support/campaigns, /campaigns/{id} |
Recent campaigns with audience, sender, subject, pause reason and lifetime statistics; one campaign adds hourly pacing and, with the opt-in, its content |
GET /v1/support/emails/{id}/lint, /campaigns/{id}/lint |
Metadata checks always (parts, List-Unsubscribe, alignment, subject style); content checks with the opt-in (the screen's image rule, links, shorteners, hidden text, unresolved tokens, staging links) |
GET /v1/support/audience/health |
Per list and overall: consent, suppressions, engagement recency and stale share, role, free-mail and mistyped addresses, providers; recent imports and their bounces |
Redaction. Never key values, hashes, webhook secrets, endpoint URLs (only their last four characters), app passwords or webhook payloads. Email bodies only when the grant came from an in-app handoff whose shareContent was ticked, and only in the workspace the customer pressed "Contact support" in (grant.contentWorkspaceId); an emailed request never shares content. Every call is written to audit_events (actor: the grant, with the operator or service that minted it; 90 days), and each workspace's Activity shows "Nightmail support viewed this workspace" once per grant, with whose request and which ticket let support in.
Server-side mint (the support Worker, never the agent):
// wrangler: "services": [{ "binding": "BANE", "service": "bane-production", "entrypoint": "SupportPlatform" }]
const checked = await env.BANE.verifyHandoff(handoffToken); // { ok, handoff: { id, userId, workspace, shareContent, usedAt, … } }
if (!checked.ok) return refuse(checked.code);
const grant = await env.BANE.mintGrant({ userId: checked.handoff.userId, ticketRef: conversationId, reason: 'Answering the customer’s support chat', verification: 'handoff', handoffId: checked.handoff.id, ttlMinutes: 120 });
// grant.status 201 → grant.body.token (nm_support_…): keep it server-side; tools add it to each read:
const context = await env.BANE.read('/v1/support/context', grant.body.token);An OpenCode custom tool the agent calls (.opencode/tool/get_user_info.ts; the grant comes from the environment, so the agent cannot widen it):
import { tool } from '@opencode-ai/plugin';
export default tool({
description: 'Who the customer is and what is going on in their Nightmail account: workspaces, sending and limits, domains, failures in the last 24 hours, recent changes and request IDs to look up.',
args: { workspaceId: tool.schema.string().optional().describe('Narrow to one of the customer’s workspaces.') },
async execute({ workspaceId }) {
const url = new URL('https://bane.mx/v1/support/context');
if (workspaceId) url.searchParams.set('workspaceId', workspaceId);
const res = await fetch(url, { headers: { authorization: `Bearer ${process.env.BANE_SUPPORT_GRANT}` } });
return await res.text();
},
});Hosted installations (MULTI_TENANT=true) bill a prepaid USD balance per workspace; self-hosted installations have no billing. Money is integer micro-cents (µ¢: 1¢ = 1,000,000 µ¢). Rate version 1: $0.20 per 1,000 emails sent (per recipient) or received (per message), $0.10 per 1,000 Omen evaluations, $0.25 per GB-month of received mail older than 90 days. Each UTC month includes 3,000 sent and 3,000 received emails, counted separately (sends beyond 3,000 are charged even while received mail is still free, and the other way round), and 1 GB of that storage. IMAP/SMTP access is free, with a fair-use download limit of 5 GB a day and 30 GB a month per inbox (override per inbox with inbox_state.egress_daily_bytes / egress_monthly_bytes). BYO workspaces are metered at $0.
billing_config.mode is a database row, so switching needs no deploy and rolls back instantly:
| Mode | Meters | Settlement | 402 gate, campaign waits, notices, auto-recharge |
|---|---|---|---|
enforce (the default since migration 300) |
record | hourly; usage is taken from promotional credit, then paid credit | live |
shadow (kill-switch) |
record | hourly; usage rows post $0 with the would-be cost in detail.costUc |
would-be denials log BILLING_SHADOW_DENY; notices are not sent |
off (kill-switch) |
record | none; usage metered while off is settled at $0 when billing comes back on | none |
Usage is priced under the mode it was metered in. scripts/set-billing-enforcement.ts settles every workspace under the old mode before it switches (except to off, so the kill-switch never waits on the settler), and a workspace's free flag starting or ending settles that workspace first.
- Metering adds no round trip: statement triggers on
sending_emails(sends, pre-SES refunds into the current hour) andmail_messages(receives,verdicts.omenCalls, stored bytes per day). Sandbox mail (region='sandbox', fromPOST /v1/inboxes/{id}/simulate) is never metered. Storage counts mail kept past 90 days: the raw size oflifecycle='kept'mail (default mail's raw MIME expires at 90 days; its extracted attachments aren't counted). Admission stays one statement; ingest stays two (three with files). - The hourly job (
billing/cron.ts) settles closed hours per workspace (billing_settle) and once a UTC day runs the storage day, the unpaid scan, the drift check (BILLING_DRIFT) and the BYO fair-use report.BILLING_SETTLE_LAGlogs an error when unsettled usage is older than 3 hours. - With an empty balance under
enforce: the API returns402 BALANCE_EXHAUSTED, campaigns wait withwaiting_reason='balance'until a credit, SMTP submission gets451 4.7.1, and Omens return theironErrorvalue without calling Jev. Received mail is accepted (and billed) for 7 days, then held underheld/in the pool bucket whenbilling_config.receive_holdis on; a credit that leaves paid plus promotional credit at $0 or more releases it, and the hourly job releases any held mail of a workspace that is no longer held. A debt older than 60 days is written off, but the hold stays until a credit. billing_accounts.mode:prepaid(default),invoiced(charged into a negative balance, never gated or held) orexempt(settled at $0).BANE_PLATFORM_DOMAIN_WORKSPACEis never gated. Migration 249 made every workspace that existed at launch (all Anomaly's)exempt; workspaces created later areprepaid.- Received mail that SES's spam scan flagged (verdict FAIL or GRAY, which ingest junks) is free: no receive unit, no free-tier use, no stored bytes. Omen evaluations on it are still metered.
- Prepaid credit expires 12 months after purchase. Each top-up is a credit lot (
billing_credit_lots); paid usage, refunds and disputes spend the lot that expires first (promotional credit is still spent before paid credit), and refunds of usage go back to the credit spent last. The hourly settler sends onecredit_expiringnotice per top-up 30 days before its remaining credit expires, then, once the expiry hour has closed and the workspace is settled, posts the remainder as oneexpireledger row (expire:<workspace>:<lot>). Unused credit is otherwise not refundable: operator refunds record a basis (a billing error, an account Anomaly closes, or a legal requirement). billing_config.storage_billing_fromprices storage at $0 before that date (the 90 days' notice).bill_simulatedmeters installation-simulated sends (dev and E2E only).- Operator money routes (credits, adjustments, write-off, refunds, close-refund) require an
Idempotency-Key: a retry returns the first answer and moves no money twice. - Turning on
receive_holdneedss3:PutObject,s3:GetObjectands3:DeleteObjectonarn:aws:s3:::<pool bucket>/held/*for the Worker's pool role, and no lifecycle rule onheld/.
A normal bun run migrate needs only DATABASE_URL and the intended DEFAULT_SES_REGION on first setup. Without rotation inputs it does not re-encrypt webhooks. If BANE_ROOT_SECRET is supplied, it first decrypts one stored webhook signing secret read-only; KEY_ROTATION_REQUIRED means the sample could not be decrypted and nothing was written. No stored webhook is required, and secretless CI skips this probe. Only explicit rotation requires the new root and previous key as described below.
MIGRATION_LEDGER_AMBIGUOUS means both opensend_migrations and nightmail_migrations exist. Stop and compare their migration names and checksums with the checkout. Only after confirming that nightmail_migrations is authoritative and that opensend_migrations is empty or contains only duplicate entries, drop the stray opensend_migrations and rerun. If they disagree, reconcile the histories first; do not replay migrations.
BANE_ROOT_SECRET is the root for authentication and HKDF-derived, domain-separated webhook encryption keys. Webhook ciphertext carries a version/key identifier. Back up the old secret securely before rotation; losing a needed decryption key can make stored signing secrets unrecoverable.
- Generate a new root secret. Set the old root as
PREVIOUS_BANE_ROOT_SECRETand the new root asBANE_ROOT_SECRETin every API/worker environment. Restart the services with the same configuration. - Run
bun run migrateagainst the intended database (Docker:docker compose -p bane-api-local run --rm api node dist/migrate.js). Migration re-encrypts stored webhook secrets with the new root. Resolve any failure before retiring old key material. - Keep the previous root until migration is complete, all services use the new configuration, and on Cloudflare the feedback queues and their dead-letter queue hold no item tagged under the old root (items are tagged at enqueue and verified under either root at consumption). Then remove
PREVIOUS_BANE_ROOT_SECRETand restart all services. Dashboard sessions are opaque database records and are revoked independently.
For an older ENCRYPTION_KEY installation, retain that old key only as upgrade-time decryption input while configuring a new BANE_ROOT_SECRET. Do not set both legacy ENCRYPTION_KEY and PREVIOUS_BANE_ROOT_SECRET. Run migration to re-encrypt secrets, verify completion, then remove ENCRYPTION_KEY and restart. Do not copy obsolete admin-key/bootstrap settings into a new deployment. Old credentials do not create a OIDC administrator.
Stop before migration if the old installation used a custom WORKSPACE_ID or contains non-default namespace data. The new installation uses a fixed default namespace, with no public workspace setting. It needs an explicit manual namespace plan covering database rows, queued work, and stored object keys; simply removing the old variable can make existing data inaccessible. Do not silently rename data or discard storage. Resolve any older in-progress encryption-key transition before this upgrade as well.
The only acceptance-test file is api/api.acceptance.test.ts. Idempotency and access control are also model-checked by api/tear against the same kind of isolated database. bun run test uses the local API/job worker and a paired local database; its OIDC identities are synthetic and its OAuth transport is mocked. It does not prove normal OIDC login with a real client. Do not run it against production or unrelated data: synthetic contacts/campaigns and audit records are created.
Use an isolated database with synthetic identities, never real user data, for local acceptance. For that dedicated local test installation, the suite needs the server's matching BANE_ROOT_SECRET, API_BASE_URL, and paired DATABASE_URL. Allow the fixture's subject acceptance-operator (AUTH_TEST_SUBJECT) in AUTH_ALLOWED_SUBJECTS. Reserve that subject and operator@example.com for the fixture; neither may collide with an existing user. Restart the API/worker after changing configuration. These are test-only identities/permissions, not production onboarding, and fake OAuth credentials alone are not an alternate login path.
The dashboard is desktop-first. At a 390-pixel viewport its controls remain reachable, but the persistent sidebar leaves a cramped main pane; small-screen layout polish remains a follow-up, not a verified mobile-ready claim.
Useful checks, from their respective packages:
api/:bun run check,bun run test,bun run sdk, andbun run cf:check(after the dashboard build).app/:bun run build.mcp/:bun run checkandbun run build; exercise the stdio client against the local API with a scoped key on a simulated deployment.
Live acceptance is opt-in with LIVE_SES_TEST=1, SES_TEST_RECIPIENT, and SES_TEST_FROM for explicitly authorized real sending. The commands above describe checks, not a claim that they have passed on your installation. Docker/Workers builds, local simulation, OIDC mocks, and source review do not replace the external release gates.
POST /v1/inboxes creates an expiring inbox with a random address on one of the workspace’s active receiving hosts. It needs a workspace administrator or an API key that creates inboxes: POST /v1/api-keys with access: {"type":"addresses","addressIds":[],"mail":["create","read"]}, which then reads only the inboxes it created. The default lifetime and grace period are 24 hours each. A clientId replays a create without revealing its key again. Each workspace can create 60 inboxes per minute and keep 1,000 retained, counting grace and pending erasure. Keys that create inboxes and MCP must omit localPart; generated names cannot override an existing wildcard or auto-create catch-all. API keys for one inbox stop working when it is deleted. Reply sending is opt-in, limited to authenticated authors in the same inbox and thread and to 20 messages a day.
GET /v1/inboxes/{id}/messages?wait=30 returns the oldest message after after, starting with the first delivery even if it arrived before the call; pass the returned cursor to advance in commit order. Junk, unverified and held mail never wake a waiter; list them with include=held (without wait). Mail past 200 messages or 25 MiB (or a lower configured cap), and mail during the grace period, is held: stored, readable and announced by message.held, never dropped. inbox.created, inbox.deleted, message.held and inbox.quota_exceeded are additive event types; consumers should ignore types they don’t know. The hourly sweep erases due inboxes and their unreferenced messages and objects, and releases real addresses to normal domain routing. An admin can release an old reservation with DELETE /v1/inbound/retired/{address}. Erasure does not shorten database backup retention or remove copies another address still holds.
Without a receiving domain, create type: "sandbox" and POST { from, subject, text, html? } to /v1/inboxes/{id}/simulate. Its generated .invalid host is permanently disabled for external mail and has no SES resources. Simulated mail uses the same store and cursor as received mail, is marked simulated: true and has no sender authentication. The limits are 50 per inbox and 1,000 per workspace per UTC day; sandbox inboxes never send.
Storage. On hosted Nightmail, agent inboxes are available once BANE_MAIL_STORE=r2 (see the rollout steps), and shared-pool inbox mail always goes to R2, so erasing it needs no S3 delete grant. Mail kept in S3 (self-hosted installs and BYO connections) is erased only if the storage role has s3:DeleteObject on workspaces/*/raw/* and workspaces/*/att/*; the BYO template does not grant it. A denied delete leaves a failed mail.deleteObjects job holding its keys, and raw MIME still expires through the bucket’s 90-day rule.
For upgrades, supply the former BETTER_AUTH_SECRET value as BANE_ROOT_SECRET to preserve encrypted webhooks and agent-token signatures. Hosted CI reads the existing AWS secret under its historical name.