Skip to content

feat: add default codex_harness.cjs with retry logic for Codex engine#30035

Merged
pelikhan merged 5 commits intomainfrom
copilot/add-default-agent-harness
May 4, 2026
Merged

feat: add default codex_harness.cjs with retry logic for Codex engine#30035
pelikhan merged 5 commits intomainfrom
copilot/add-default-agent-harness

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 4, 2026

Summary

Adds a default agent harness for the OpenAI Codex engine, following the established pattern of copilot_harness.cjs and claude_harness.cjs.

Changes

New files

  • actions/setup/js/codex_harness.cjs — Wraps codex exec with retry logic for transient OpenAI API errors (rate limits, server errors). Reads the prompt via --prompt-file (a harness-only flag) and passes content as the last positional arg to codex exec. Since Codex has no --continue for session resumption, all retries are fresh runs with exponential backoff (max 3 retries, 5s→60s).
  • actions/setup/js/codex_harness.test.cjs — 23 unit tests covering prompt file resolution, error pattern detection, and retry policy.

Modified files

  • pkg/workflow/codex_engine.go — Added GetHarnessScriptName() returning "codex_harness.cjs" (implements HarnessProvider interface). Updated GetExecutionSteps() to use the harness via nodeRuntimeResolutionCommand, passing --prompt-file instead of the previous shell $INSTRUCTION variable.
  • pkg/workflow/codex_engine_test.go — Three new tests: TestCodexEngineGetHarnessScriptName, TestCodexEngineExecutionUsesHarness, TestCodexEngineExecutionCustomHarness.
  • pkg/workflow/engine_agent_import_test.go, pkg/workflow/engine_args_test.go — Updated existing tests to expect --prompt-file instead of $INSTRUCTION.
  • Lock files — Recompiled codex workflows now use the harness invocation.

Retry policy

  • Retries on partial execution (any output produced) or well-known transient errors (rate limit, server error)
  • No --continue support — all retries are fresh runs
  • Exponential backoff: 5s → 10s → 20s (capped at 60s)
  • Max 3 retry attempts

Changeset

  • Type: patch
  • Description: Added the default Codex engine harness with retry handling for transient execution failures.

Warning

Firewall blocked 2 domains

The following domains were blocked by the firewall during workflow execution:

  • ab.chatgpt.com
  • chatgpt.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"
    - "chatgpt.com"

See Network Configuration for more information.

Generated by Changeset Generator for issue #30035 ·

Copilot AI and others added 2 commits May 4, 2026 00:08
Copilot AI requested a review from pelikhan May 4, 2026 00:14
@pelikhan pelikhan marked this pull request as ready for review May 4, 2026 00:21
Copilot AI review requested due to automatic review settings May 4, 2026 00:21
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a default Node.js harness for the Codex engine to provide retry behavior on transient OpenAI failures, and updates Codex execution to route prompts via --prompt-file instead of $INSTRUCTION.

Changes:

  • Added codex_harness.cjs and unit tests to wrap codex exec with retry logic and prompt-file handling.
  • Updated Codex engine execution to invoke the harness via Node runtime resolution and pass --prompt-file.
  • Updated Go/unit tests and recompiled workflow lock files to reflect the new harness-based invocation.
Show a summary per file
File Description
actions/setup/js/codex_harness.cjs New Codex CLI harness implementing prompt-file resolution + retry/backoff.
actions/setup/js/codex_harness.test.cjs Unit tests for prompt-file resolution, error detection, and retry policy.
pkg/workflow/codex_engine.go Uses the harness (default or overridden) and passes --prompt-file to it.
pkg/workflow/codex_engine_test.go Adds tests asserting harness usage, default name, and override behavior.
pkg/workflow/engine_agent_import_test.go Updates expectations from $INSTRUCTION to --prompt-file for Codex steps.
pkg/workflow/engine_args_test.go Updates args-position assertions to key off --prompt-file.
.github/workflows/schema-feature-coverage.lock.yml Lockfile reflects Codex harness invocation + --prompt-file.
.github/workflows/issue-arborist.lock.yml Lockfile reflects Codex harness invocation + --prompt-file.
.github/workflows/grumpy-reviewer.lock.yml Lockfile reflects Codex harness invocation + --prompt-file.
.github/workflows/duplicate-code-detector.lock.yml Lockfile reflects Codex harness invocation + --prompt-file.
.github/workflows/codex-github-remote-mcp-test.lock.yml Lockfile reflects Codex harness invocation + --prompt-file.
.github/workflows/changeset.lock.yml Lockfile reflects Codex harness invocation + --prompt-file.
.github/workflows/ai-moderator.lock.yml Lockfile reflects Codex harness invocation + --prompt-file.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 13/13 changed files
  • Comments generated: 2

Comment thread actions/setup/js/codex_harness.cjs Outdated
Comment on lines +14 to +16
* - Rate-limit errors (HTTP 429 / "rate_limit_exceeded") and server errors (HTTP 429,
* 500, 503) are well-known transient failure modes and are logged explicitly, but
* any partial-execution failure is retried — not just those specific errors.
Comment thread pkg/workflow/codex_engine.go Outdated
Comment on lines +223 to +224
// prompt via --prompt-file. The else branch is retained as a fallback in case the harness
// is explicitly disabled in the future (e.g. engine.harness overrides that clear the name).
…retry

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

Commit pushed: 14e497a

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

🏗️ Design Decision Gate — ADR Required

This PR makes significant changes to core business logic (144 new lines in pkg/) but does not have a linked Architecture Decision Record (ADR).

AI has analyzed the PR diff and generated a draft ADR to help you get started:

📄 Draft ADR: docs/adr/30035-codex-engine-node-harness-with-retry.md

What to do next

  1. Review the draft ADR committed to your branch — it was generated from the PR diff
  2. Complete the missing sections — add context the AI could not infer, refine the decision rationale, and list real alternatives you considered
  3. Commit the finalized ADR to docs/adr/ on your branch
  4. Reference the ADR in this PR body by adding a line such as:

    ADR: ADR-30035: Codex Engine Node.js Harness with Retry

Once an ADR is linked in the PR body, this gate will re-run and verify the implementation matches the decision.

Decision Summary Inferred from Diff

The draft ADR captures the following decision:

  • Decision: Introduce codex_harness.cjs — a Node.js wrapper following the established claude_harness.cjs / copilot_harness.cjs pattern — to add retry logic (up to 3 attempts, exponential backoff 5 s to 60 s) for transient OpenAI API failures, and to switch prompt delivery from shell-variable injection to file-based via --prompt-file.
  • Key constraint: The Codex CLI has no --continue session-resumption, so all retries restart from scratch.
  • Interface: CodexEngine implements HarnessProvider returning "codex_harness.cjs".

Why ADRs Matter

ADRs create a searchable, permanent record of why the codebase looks the way it does. Future contributors (and your future self) will thank you.


📋 Michael Nygard ADR Format Reference

An ADR must contain these four sections to be considered complete:

  • Context — What is the problem? What forces are at play?
  • Decision — What did you decide? Why?
  • Alternatives Considered — What else could have been done?
  • Consequences — What are the trade-offs (positive and negative)?

All ADRs are stored in docs/adr/ as Markdown files numbered by PR number (e.g., 30035-codex-engine-node-harness-with-retry.md for PR #30035).

🔒 This PR cannot merge until an ADR is linked in the PR body.

References: §25295059484

🏗️ ADR gate enforced by Design Decision Gate 🏗️ · ● 213.3K ·

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

test comment

🧪 Test quality analysis by Test Quality Sentinel · ● 2M ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 76/100. Test quality is acceptable — 23% of new tests are implementation tests (threshold: 30%). See the comment above for detailed analysis and suggestions to improve retry-logic test coverage.

@pelikhan
Copy link
Copy Markdown
Collaborator

pelikhan commented May 4, 2026

@copilot review all comments

Copy link
Copy Markdown
Contributor Author

Copilot AI commented May 4, 2026

@copilot review all comments

Fixed both review comments in b3b6b00:

  • codex_harness.cjs line 14: removed HTTP 429 from the server-error description (it belongs only under the rate-limit bullet)
  • codex_engine.go lines 223-224: reworded to describe the else branch as a defensive fallback, without implying a config-based disable mechanism

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

💫 TO BE CONTINUED... Smoke Claude failed! Our hero faces unexpected challenges...

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

📰 BREAKING: Smoke Copilot is now investigating this pull request. Sources say the story is developing...

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

🚀 Smoke Pi MISSION COMPLETE! Pi delivered. 🥧

@github-actions github-actions Bot removed the smoke label May 4, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

⚠️ Smoke Gemini failed. Gemini encountered unexpected challenges...

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

Agent Container Tool Check

Tool Status Version
bash 5.2.21
sh available
git 2.53.0
jq 1.7
yq 4.52.5
curl 8.5.0
gh 2.89.0
node 22.22.2
python3 3.10.16 (PyPy 7.3.19)
go 1.24.13
java 10.0.201
dotnet NOT FOUND

Result: 11/12 tools available ⚠️

Overall Status: FAILdotnet (.NET runtime) is not available in this container.

🔧 Tool validation by Agent Container Smoke Test · ● 130.4K ·

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

Codex smoke: FAIL
PRs: #30030 fix: single-quote GH_AW_OTLP_ENDPOINTS to prevent YAML sequence parsing; #30028 feat: query /reflect before and after running the agent in harnesses
✅ GitHub MCP, Serena, file write, bash, build, comment memory, cache memory
❌ Playwright command compatibility, web-fetch unavailable
Run: https://github.com/github/gh-aw/actions/runs/25295890971

Warning

Firewall blocked 8 domains

The following domains were blocked by the firewall during workflow execution:

  • ab.chatgpt.com
  • accounts.google.com
  • android.clients.google.com
  • chatgpt.com
  • clients2.google.com
  • contentautofill.googleapis.com
  • safebrowsingohttpgateway.googleapis.com
  • www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"
    - "accounts.google.com"
    - "android.clients.google.com"
    - "chatgpt.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex ·

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

Comment Memory

Silent checks complete
Green builds mark the night
Smoke clears into dawn

Note

This comment is managed by comment memory.

It stores persistent context for this thread in the code block at the top of this comment.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

Warning

Firewall blocked 8 domains

The following domains were blocked by the firewall during workflow execution:

  • ab.chatgpt.com
  • accounts.google.com
  • android.clients.google.com
  • chatgpt.com
  • clients2.google.com
  • contentautofill.googleapis.com
  • safebrowsingohttpgateway.googleapis.com
  • www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"
    - "accounts.google.com"
    - "android.clients.google.com"
    - "chatgpt.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex ·

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

UGH! Smoke test caveman was here! Me test things. Things work! Cave fire still burn! 🔥

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

  • accounts.google.com
  • android.clients.google.com
  • clients2.google.com
  • contentautofill.googleapis.com
  • safebrowsingohttpgateway.googleapis.com
  • www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot · ● 1.3M ·

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

Comment Memory

Code runs and breaks
Smoke rises from the test logs
Green means we survived

Note

This comment is managed by comment memory.

It stores persistent context for this thread in the code block at the top of this comment.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

  • accounts.google.com
  • android.clients.google.com
  • clients2.google.com
  • contentautofill.googleapis.com
  • safebrowsingohttpgateway.googleapis.com
  • www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot · ● 1.3M ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ugh. Me review code. Me like retry logic and exponential backoff. Code look good to cave dweller!

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

  • accounts.google.com
  • android.clients.google.com
  • clients2.google.com
  • contentautofill.googleapis.com
  • safebrowsingohttpgateway.googleapis.com
  • www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

📰 BREAKING: Report filed by Smoke Copilot · ● 1.3M

* - If the process produced no output (failed to start / auth error before any work), the
* driver does not retry because there is nothing to resume.
* - Retries use exponential backoff: 5s → 10s → 20s (capped at 60s).
* - Maximum 3 retry attempts after the initial run.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Me like retry logic! Three retry max good number. But maybe log retry count so cave dwellers can see how many times it tried?

} = require("./awf_reflect.cjs");

// Maximum number of retry attempts after the initial run
const MAX_RETRIES = 3;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAX_RETRIES = 3 is good! Cave math: 1 + 3 = 4 total tries. Exponential backoff protect from rate limits. Me approve!

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 4, 2026

Commit pushed: 3d92e2d

Generated by Changeset Generator

@pelikhan pelikhan merged commit 3a4fe48 into main May 4, 2026
@pelikhan pelikhan deleted the copilot/add-default-agent-harness branch May 4, 2026 01:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants