fix(runtime): classify exhausted Codex edge rejection

[liugddx]

Summary

Fixes #3959

Preserve the distinction between Codex JSON authorization failures and transient HTML edge rejections after the bounded retry budget is exhausted. The transport emits a dedicated internal sentinel only when all three replayable edge retries have actually been consumed and the terminal response is still the same HTML edge rejection. Parseable JSON wins over a misleading Content-Type, non-replayable bodies are never stamped as exhausted edge failures, and provider classification trusts only the internally constructed error provenance.

The exhausted sentinel is ProviderUnavailable for presentation but non-retryable at the outer model loop, because the transport has already spent its complete 2/10/30-second retry budget.

Verification

• built @maka/core, @maka/storage, @maka/mcp, @maka/computer-use, and @maka/runtime
• node --test "packages/runtime/dist/__tests__/subscription-model-fetch.test.js" "packages/runtime/dist/__tests__/provider-error-classification.test.js" (36 passed)
• Biome format/check passed for all changed files

Regression coverage includes the original 2/10/30 second HTML edge retry budget, JSON auth with an incorrect HTML content type, non-replayable HTML responses, three HTML retries followed by a JSON auth failure, spoofed sentinel codes, JSON 403 server_error remaining Auth, and the trusted exhausted-edge diagnostic projection.

AI use

Select exactly one:

• No generative tool made a substantive contribution
• Generative tooling made a substantive contribution

Tool(s) and scope: Codex authored the implementation, tests, and review follow-up; the human contributor reviewed the changes and owns the submission.

Checklist

• Tests cover the change and fail without it
• Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

• Yes — exhausted transient Codex edge rejections are presented as provider unavailable without restarting their consumed transport retry budget, while genuine JSON authorization failures remain Auth.
• No

[Astro-Han]

I reviewed this PR at exact head e65e7c0259bbf0e2ede7f2b943e76c58f9dd8047 (base 6762085e, 4 files +58−8, 1 commit).

Spec: NO-GO — 1×P1 + 2×P2

• P1 — the 403 override is broader than the trusted Codex sentinel. Moving the whole PROVIDER_UNAVAILABLE_PROVIDER_CODES set before 401/403 at packages/runtime/src/provider-error-classification.ts:741-744 also changes an existing JSON 403 {error:{code:'server_error'}} from Auth/non-retryable to ProviderUnavailable/retryable, contrary to Codex HTML 403 retry exhaustion is misclassified as authentication failure #3959’s requirement that JSON 401/403 remain auth/account failures. Any provider payload can also spoof openai_codex_edge_rejection; the classification does not require trusted OpenAiCodexEdgeRejectionError provenance. Narrow the 403 override to the internally created sentinel/provenance.
• P2 — header-only HTML detection can erase a real JSON auth code. At packages/runtime/src/subscription-model-fetch.ts:272-275, four 403s with body {"error":{"code":"account_not_authorized"}} but Content-Type: text/html are retried, then overwritten with the edge sentinel. Parseable provider JSON should win over a misleading content type; add the transport→diagnostic regression.
• P2 — “exhausted” is stamped even when zero edge retries occurred. The retry branch at :205-220 requires replayability, but the terminal stamp requires neither replayability nor edgeRetry === 3; a non-replayable body’s first HTML 403 becomes retryable ProviderUnavailable without consuming the 2/10/30s budget.

Positives: the transport test proves the fourth attempt and exact 2/10/30 delays; existing abort tests cover both signal sources; HTTP 403/request ID and bounded/redacted diagnostics remain preserved.

Standards: 1×P1 + 1×P2 + 1 judgment P3

• P1 — mandatory AI-use declaration is absent, required by CONTRIBUTING.md:30-34 / PR template. Complete it; if substantive AI authored the change, affected commits also need the required Generated-by: trailer.
• P2 — required PR template was replaced. The custom body omits the Checklist and behavior-change declaration required by CONTRIBUTING.md:79-83.
• Judgment P3 — exhaustedEdgeRejection may be true after zero retries on a non-replayable body (subscription-model-fetch.ts:339-350); rename to the actual fact or gate it on real exhaustion.

No architecture-boundary breach or actionable duplication was found.

Other checks: git diff --check passes; exact-head test run 33027481811 and windows_recovery run 33027481814 are terminal SUCCESS; expected path-filtered checks are green. Post-seal the PR is OPEN / non-draft / MERGEABLE / BLOCKED / REVIEW_REQUIRED, head unchanged, 0 reviews/comments.

What I did not check: full local suite beyond the checks noted.

Gate: exact head has P1/P2 findings and therefore cannot be approved or merged until the 403 override, HTML detection, exhaustion stamping, and template/AI disclosure issues are fixed.

---

Automated review notice: This comment was posted by an automated review agent operated by Astro-Han. It is not an independent human review and does not replace one.

[liugddx]

Thanks for the exact-head review. I addressed all in-scope P1/P2 findings in head �9e95c812: (1) the exhausted-edge sentinel is now trusted only from the internally created OpenAiCodexEdgeRejectionError/Maka model-failure provenance, so arbitrary provider server_error or spoofed 403 payloads remain Auth; (2) parseable JSON bodies win even when Content-Type is incorrectly ext/html; (3) the sentinel is stamped only after all three replayable edge retries are consumed, while non-replayable requests retain the normal HTTP error. Regression coverage now includes each boundary. I also restored the required PR template sections, declared substantive AI use, and added the Generated-by: Codex trailer. CI was green before this push; the new head is ready for re-review. There are no inline threads to resolve.

[liugddx]

The updated head is �9e95c812; all review findings are covered by the new transport/classification tests and the required PR template/AI disclosure is present. No inline review threads exist on this PR; only the automated summary comment required a follow-up.

[Astro-Han]

Adds a dedicated internal sentinel for the case where Codex's edge returns HTML 403 across the whole bounded retry budget, and teaches the classifier to read it as ProviderUnavailable instead of Auth, so an edge/WAF incident stops telling users to re-authenticate (#3959).

The problem is real and the direction is right — a 403 from a WAF is not an authorization answer, and classifying it as one sends users to fix a credential that was never broken. The retry budget, the retry-after handling, and the provenance idea are all sound instincts. Three things need to change before I can approve, and two of them are in the trust boundary this PR introduces.

P1 — the sentinel is stamped from the retry counter rather than from what actually failed (subscription-model-fetch.ts:217). Once three edge retries are consumed, any subsequent failure on that request is stamped as an exhausted edge rejection, and the real provider code is discarded. An expired token on attempt 4 gets reported as a provider outage and retried. Details and the one-conjunct fix are inline.

P1 — the provenance check is bypassable two independent ways, both introduced in this diff: the PROVIDER_UNAVAILABLE_PROVIDER_CODES entry at :40 grants the classification to exactly the payloads the trust check rejected, and the trust function's own model_failure clause at :796 accepts a forgeable code that normalizeProviderFailure copies straight out of the provider's payload. Both inline. I want to be clear that this is not a hypothetical-attacker objection: the practical case is a user-configured OpenAI-compatible proxy, and the consequence is bounded. It matters because a guard that is defeated in the same diff that adds it will be relied on as sound by the next change.

P2 — the exhausted signal is handed to a second retry authority, which restarts the budget it was meant to end. ProviderUnavailable with no retry-after returns {retryable: true} from providerRetryMetadata (:248-258), and ai-sdk-backend.ts:2583 retries up to MAX_PROVIDER_ATTEMPTS_PER_STEP = 10. Each outer attempt re-enters the inner 2/10/30 s edge budget. A persistent edge block therefore turns a 45-second terminal failure into roughly forty requests against an actively-rejecting WAF over several minutes before the turn ends. It is abortable and it does not corrupt anything, so this is not a blocker — but the transport already owns the retry budget for this exact condition, and the sentinel's whole meaning is "that budget is spent". Consider returning retryable: false, or carrying a retryAfterMs that reflects what has already been spent.

Three P3s, none blocking:

• durableProviderErrorClass (:457-463) now passes ProviderUnavailable through for every classification, not just this one. Needed for the Codex case, but it also changes the durable diagnostic for structured server_error payloads whose status lives in a nested source. Telemetry class only, no runtime behavior.
• The cross-module contract is four bare string literals plus an error name in two more places, none imported from a shared export. Worth a single exported constant. Note that the classification does not depend on the provider's message text, which is good — but it does depend on the body being HTML-shaped (subscription-model-fetch.ts:268-277). If Codex's edge ever emits a JSON body with an HTML content type, the JSON.parse pre-check returns false, no retry happens at all, and #3959 is unfixed for that shape. I could not verify what the edge actually emits; that claim rests on the issue's analysis.
• The PROVIDER_UNAVAILABLE_PROVIDER_CODES check moved below the /^5\d\d$/ check (:757-766). Both branches return the same class, so it is a no-op reorder adding diff noise to a carefully ordered function. Drop it.

Ungraded note on the body: it says the sentinel is emitted "only after all three replayable edge retries have actually been consumed", which is true, but the implication that the final failure is an edge rejection is not — that is the P1 above.

Also worth coordinating with #3812, which touches the same classifier. They merge cleanly and do not miscompose today, but after both land providerRetryMetadata answers "retryable?" from three independent authorities: the classified class, a raw HTTP status range, and an isRetryable flag walked from the cause chain. Once #3812 exists, this PR could arguably be implemented entirely in the transport by setting that flag on its own error, with no classifier change at all. Worth picking one seam before both land.

AI use: Claude Code (Opus) produced the initial findings; I then independently re-verified each graded one against the PR head b9e95c8 — reading the throw site and openAiCodexHttpError, the classifier's branch ordering, isTrustedCodexEdgeRejection, and the normalizeProviderFailure → providerFailureSummary chain that populates the envelope's code. The second bypass at :796 is from that re-verification, not from the initial pass. No scenario was reproduced at runtime. The reviewer of record reviewed and accepted this.

简体中文

为 "Codex 边缘在整个有界重试预算内持续返回 HTML 403" 这一情形引入专用内部哨兵，并让分类器把它读作 ProviderUnavailable 而非 Auth，使边缘/WAF 事故不再提示用户重新认证（#3959）。

问题是真的，方向也对——WAF 返回的 403 不是一个授权答复，把它当作授权答复会让用户去修一个从未损坏的凭据。重试预算、retry-after 处理和"出处校验"的思路都是好的直觉。Approve 前有三处需要改，其中两处就在本 PR 引入的信任边界上。

P1 —— 哨兵按重试计数器打戳，而非按真正失败的东西打戳（subscription-model-fetch.ts:217）。三次边缘重试用尽后，该请求上任何后续失败都会被打成"边缘拒绝已耗尽"，真实的 provider code 被丢弃。第 4 次尝试上过期的令牌会被报成服务商故障并继续重试。细节与只需一个合取项的修法见行内。

P1 —— 出处校验有两条独立的绕过路径，且都由本 diff 引入：:40 的 PROVIDER_UNAVAILABLE_PROVIDER_CODES 条目恰好把分类授予了信任检查已经拒绝的那些载荷；而信任函数自身 :796 的 model_failure 分支接受一个可伪造的 code，该 code 由 normalizeProviderFailure 直接从服务商载荷里抄过来。两条均见行内。需要说明：这不是"假想攻击者"式的反对——现实情形是用户自配的 OpenAI 兼容代理，后果也是有界的。它重要是因为：一个在引入它的同一 diff 里就被击穿的守卫，会被下一次改动当作可靠前提来依赖。

P2 —— 把"预算已耗尽"的信号交给了第二个重试权威，而后者重启了它本要终结的预算。 无 retry-after 的 ProviderUnavailable 在 providerRetryMetadata（:248-258）返回 {retryable: true}，而 ai-sdk-backend.ts:2583 最多重试 MAX_PROVIDER_ATTEMPTS_PER_STEP = 10 次，每次外层尝试又重新进入内层 2/10/30 秒的边缘预算。因此持续性的边缘封锁会把一次 45 秒的终态失败，变成数分钟内对一个正在主动拒绝的 WAF 发起约四十次请求。它可被中止、不破坏任何状态，所以不阻塞——但传输层本就为这一条件持有重试预算，而哨兵的全部含义正是"该预算已经花完"。可考虑返回 retryable: false，或携带一个反映已花费预算的 retryAfterMs。

三条 P3，均不阻塞：

• durableProviderErrorClass（:457-463）现在对所有 ProviderUnavailable 分类放行，而不只是本情形。Codex 这条路径需要它，但它同时改变了那些状态藏在嵌套源里的结构化 server_error 载荷的持久诊断分类。仅影响遥测类别，不影响运行时行为。
• 跨模块契约是四个裸字符串字面量，外加两处错误 name，都没有从共享导出引入。值得抽一个导出常量。要肯定的是：分类不依赖服务商的可读消息文本；但它依赖响应体呈 HTML 形状（subscription-model-fetch.ts:268-277）。若 Codex 边缘某天以 HTML content-type 返回 JSON 体，JSON.parse 预检返回 false，将完全不再重试，#3959 对该形状仍未修复。我无法核实边缘实际输出什么，这一点依赖 issue 的分析。
• PROVIDER_UNAVAILABLE_PROVIDER_CODES 检查被移到了 /^5\d\d$/ 检查之后（:757-766）。两个分支返回同一类别，因此是空操作重排，给一个精心排序的函数增加了 diff 噪声。建议撤掉。

关于正文的未定级说明：文中称哨兵"仅在三次可重放边缘重试全部真正耗尽后"才发出，这是对的；但由此暗示最终失败就是一次边缘拒绝，则不成立——即上面那条 P1。

另外建议与 #3812 协调，它改动同一个分类器。两者能干净合并，今天也不会错误组合；但两个都落地之后，providerRetryMetadata 将从三个独立权威回答"是否可重试"：分类出的类别、原始 HTTP 状态区间，以及从 cause 链上走出来的 isRetryable 标志。一旦 #3812 存在，本 PR 其实可以完全在传输层实现——在自己构造的错误上设置该标志，完全不动分类器。建议在两者落地前先选定一个接缝。

[Astro-Han]

This is a synthesis of the independent bounded freshness review by @Sol-404ARE at exact head b9e95c812708c994e52f146479fea14c84a45fc0 (base 9d62ee9a, actual merge-base 6762085e, 4 files +160−8) versus the sealed baseline e65e7c0259bbf0e2ede7f2b943e76c58f9dd8047. I verified the diff and the exact-head CI myself; the file:line findings below are from Sol's sealed review.

What I checked myself:

• Read gh pr diff 3960 for b9e95c81 (4 files/+160/−8, rebase from 6762085e to 9d62ee9a but actual merge-base unchanged) and confirmed the repair delta is 4 files +108/−6, not a pure rebase/tree-equivalent.
• Checked exact-head CI: test run 33036087975 SUCCESS, windows_recovery run 33036087985 SUCCESS, OPEN/MERGEABLE/BLOCKED/REVIEW_REQUIRED.

Findings from Sol's review (file:line anchored):

Standards — GO (0 hard violations)

• Hard Standards GO: the previously missing AI-use declaration and Generated-by: Codex trailer are now present, and the PR template fields are completed. Title/branch/runtime boundary pass.
• Judgment-only P3 Mysterious Name — packages/runtime/src/subscription-model-fetch.ts:217,341-353: edgeRetry === edgeRetryDelaysMs.length is named and passed as exhaustedEdgeRejection, but it only proves the first three attempts were edge retries, not that the terminal response is still an edge rejection. After three HTML 403s, a fourth JSON 401/403, 429, or 5xx would still be named and stamped as OpenAiCodexEdgeRejectionError. The code should retain and check the final response's edge fact before naming/stamping.

Old Standards closures: AI-use, template, title/branch, and runtime boundary are now closed.

Spec — NO-GO: 1×P1

• P1 — terminal sentinel still mislabels a later real JSON auth response as an edge outage. At subscription-model-fetch.ts:205-217, after three transient HTML 403s edgeRetry===3; if attempt 4 returns HTTP 403 {error:{code:'account_not_authorized'}}, line 217 still passes true to openAiCodexHttpError, which overwrites the original JSON code with openai_codex_edge_rejection and names it OpenAiCodexEdgeRejectionError. Downstream it becomes ProviderUnavailable/retryable, directly violating issue Codex HTML 403 retry exhaustion is misclassified as authentication failure #3959 “JSON 401/403 remain auth/account”. The fix should require both budget exhausted and isTransientOpenAiCodexEdgeRejection(response, detail) on the terminal response; a guard test is missing: three HTML 403s (retry-after 0), fourth JSON 403 should yield a normal OpenAiCodexHttpError/Auth and preserve the original provider code.

Old Spec closures: JSON 403 server_error precedence, spoofed AI_APICallError sentinel, mislabelled text/html parseable JSON, and non-replayable first HTML 403 are now closed. The 2/10/30 delays, abort handling, and HTTP/request-ID diagnostics remain correct.

What I did not judge:

• Full local runtime suite beyond the focused checks (the isolated worktree had no installed deps, so no local suite was claimed).

Gate: the rebase itself is not pure tree-equivalent, and one P1 (plus one judgment P3 naming) remains; the PR cannot be considered merge-ready until the terminal sentinel also checks the final response's edge fact and the naming is corrected.

---

Automated review notice: This comment was posted by an automated review agent operated by Astro-Han. It is not an independent human review and does not replace one.

[liugddx]

Addressed the latest exact-head review in af00398c0: the terminal sentinel now requires the final response to remain an edge rejection; the generic provider-code and model-failure trust bypasses are removed; the exhausted transport budget is not restarted by the outer model loop; and a three-HTML-then-JSON-auth regression preserves the real provider code. Local focused result: 36 passed, runtime build and Biome clean. I replied to and resolved all three inline threads. Ready for a fresh exact-head review after CI.

[liugddx]

Rebased the review-complete repair onto current upstream/main@4b745fc56; new exact head is f0e751a24. All three inline threads are replied to and resolved. Fresh test / windows_recovery runs are in progress; a new exact-head review is still required.

[Astro-Han]

This is a synthesis of the independent bounded freshness review by @Sol-404ARE at exact head f0e751a2434073fd1b9977b8f00d05d094776e36 (base 4b745fc5, actual merge-base 6762085e, 4 files +207−10) versus the sealed baseline b9e95c812708c994e52f146479fea14c84a45fc0. I verified the diff and the exact-head CI myself; the file:line findings below are from Sol's sealed review.

What I checked myself:

• Read gh pr diff for f0e751a2 (the 4-file repair delta +207/−10 on top of b9e95c81; full PR diff includes a rebase to 4b745fc5 with many workflow/docs changes that are not part of the repair) and confirmed the repair adds the final-response edge check and two regression tests.
• Checked exact-head CI: test run 33042145703 SUCCESS, windows_recovery run 33042145686 SUCCESS, OPEN/MERGEABLE/BLOCKED/REVIEW_REQUIRED.

Findings from Sol's review (file:line anchored):

Spec — NO-GO: 1×P1

• P1 — the repair flips retryable to false, directly violating issue Codex HTML 403 retry exhaustion is misclassified as authentication failure #3959's acceptance that ProviderUnavailable be retryable. At packages/runtime/src/provider-error-classification.ts:227-230, the trusted sentinel openai_codex_edge_rejection is now classified as { retryable:false }, and provider-error-classification.test.ts:147-152 locks that false. This is not just diagnostic: normalizeModelFailure copies the bit, settleModelStepOutcome turns false into a terminal failure, and ai-sdk-backend.ts:2581-2584 only runs the bounded outer provider retry when true. The shortest repro is the existing test's OpenAiCodexEdgeRejectionError: it is still ProviderUnavailable but now retryable:false, so a transient outage after the full 2/10/30 transport budget terminates immediately. The PR body's “inner 2/10/30 already exhausted, avoid outer retry budget” is a defensible new policy, but without an updated and approved spec/acceptance it remains a clear spec regression. Fix by either restoring retryable:true or updating and getting approval for the spec/acceptance evidence.

Old closures: the previous P1 (final stamp must also check current response's edge fact) is now closed — the new code requires both budget exhausted and isTransientOpenAiCodexEdgeRejection on the terminal response, with a 3×HTML→final JSON auth regression that preserves OpenAiCodexHttpError/account_not_authorized. JSON precedence, non-replayable guard, HTTP 403 diagnostics, trusted provenance, and ProviderUnavailable presentation remain correct.

Standards — GO (0 hard violations)

• No hard Standards violations; the previous P3 naming issue is now closed (exhaustedEdgeRejection now requires both retry budget exhausted and final response still satisfies isTransientOpenAiCodexEdgeRejection, proven by the new test).
• No new documented-standard breach or meaningful Fowler smell; the commit retains Generated-by: Codex.

Other checks: git diff --check and 4-file Biome pass; exact-head test and windows_recovery are green.

What I did not judge:

• Full local runtime suite beyond the focused checks (the isolated worktree had no installed deps, so no local suite was claimed).

Gate: the rebase is not pure tree-equivalent and one P1 retryable regression remains; the PR cannot be considered merge-ready until retryable is restored to true or the spec/acceptance is updated and approved.

---

Automated review notice: This comment was posted by an automated review agent operated by Astro-Han. It is not an independent human review and does not replace one.

[Astro-Han]

All three graded findings are closed, each at the seam rather than at the symptom.

P1 (sentinel stamped from the retry counter) — subscription-model-fetch.ts:220 now requires edgeRetry === edgeRetryDelaysMs.length && isTransientOpenAiCodexEdgeRejection(response, detail), so an expired token on attempt 4 is no longer reported as an edge outage. That was the one that mattered most: it made the fix reproduce the shape of the bug it exists to fix, in the other direction.

P1 (two bypasses of the provenance check) — both gone. The PROVIDER_UNAVAILABLE_PROVIDER_CODES entry is removed, and isTrustedCodexEdgeRejection is now only the cause-chain walk for an Error named OpenAiCodexEdgeRejectionError. The forgeable model_failure clause that accepted a provider-supplied code is deleted, so the guard is no longer defeated inside the diff that adds it.

P2 (the exhausted signal handed to a second retry authority) — providerRetryMetadata short-circuits with retryable: false ahead of the generic ProviderUnavailable branch, and the comment states why. A persistent edge block now ends at roughly 45 seconds instead of fanning out to about forty requests against an actively-rejecting WAF.

P3 (durableProviderErrorClass) — tightened further than I suggested: ProviderUnavailable passes through only when isTrustedCodexEdgeRejection(facts), so structured server_error payloads keep their previous durable class.

One P3 remains, non-blocking: the cross-module contract is still string literals. OPENAI_CODEX_EDGE_REJECTION_CODE exists at provider-error-classification.ts:38 but isn't exported, and subscription-model-fetch.ts:352 plus the OpenAiCodexEdgeRejectionError name are still spelled out by hand in both modules — two producers and two consumers agreeing by spelling.

Still worth coordinating with #3812 before both land (it's currently red), since afterwards providerRetryMetadata answers "retryable?" from three independent authorities.

---

AI-assisted review: I verified each fix against the current head myself — the sentinel conjunct, the removal of both bypasses, the short-circuit ordering in providerRetryMetadata, and the tightened durableProviderErrorClass condition. No tests run. AI review is not independent human review.

简体中文

三条定级发现全部关闭,且都修在接缝上。哨兵现在要求"重试预算耗尽且本次失败确为瞬态边缘拒绝",不再因为计数器耗尽就把第 4 次尝试上的过期令牌误报成边缘故障——那一条最要紧,它让修复以相反方向复现了它要修的 bug 的形状。两条绕过路径均已删除,信任检查只剩 cause chain 上的真实出处。providerRetryMetadata 在通用分支之前短路返回 retryable: false,持续边缘封锁在约 45 秒终止,而非扩散成约四十次请求。durableProviderErrorClass 收得比我建议的更紧。

剩一条 P3 不阻塞:跨模块契约仍靠字面量拼写一致,OPENAI_CODEX_EDGE_REJECTION_CODE 未导出。