Skip to content

Mark Claude and Codex engines as experimental#4368

Merged
pelikhan merged 2 commits into
mainfrom
copilot/make-engines-experimental
Nov 20, 2025
Merged

Mark Claude and Codex engines as experimental#4368
pelikhan merged 2 commits into
mainfrom
copilot/make-engines-experimental

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Nov 20, 2025

Claude and Codex engines are marked as experimental to reflect their current support status. Documentation now recommends Copilot as the primary engine.

Changes

Engine Configuration

  • Set experimental: true for Claude engine (Codex was already experimental)
  • Updated test assertions to expect experimental status

Documentation

  • Removed detailed Claude/Codex sections from main engine documentation
  • Added experimental status note with reference to availability
  • Changed all examples from Claude/Codex to Copilot
  • Removed engine-specific configuration fields (e.g., user-agent for Codex)

User-Facing Behavior
When compiling with --verbose:

$ gh aw compile workflow.md --verbose
⚠ Using experimental engine: Claude Code

Engines remain fully functional. Users can still set engine: claude or engine: codex in workflow frontmatter.

Original prompt

Make Claude/Codex agentic engines experimental and remove mentions from the documentation.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

- Set Claude engine experimental flag to true in claude_engine.go
- Update Claude engine test to expect experimental status
- Remove Claude and Codex from main documentation in engines.md
- Add note about experimental engines in engines.md
- Update frontmatter-full.md to mark Claude/Codex as experimental
- Update mcps.md example from codex to copilot
- Generalize firewall error message in errors.md
- Update github-agentic-workflows.instructions.md to mark engines as experimental
- All tests pass successfully

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Make Claude and Codex agentic engines experimental Mark Claude and Codex engines as experimental Nov 20, 2025
Copilot AI requested a review from pelikhan November 20, 2025 02:09
@pelikhan pelikhan marked this pull request as ready for review November 20, 2025 02:23
Copilot AI review requested due to automatic review settings November 20, 2025 02:23
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Nov 20, 2025

✅ Agentic Changeset Generator completed successfully.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR marks Claude and Codex engines as experimental to reflect their current support status and updates all documentation to recommend GitHub Copilot as the primary engine. The Claude engine's experimental flag is changed from false to true (Codex was already experimental). All references to Claude and Codex engines in documentation have been removed or marked as experimental, with examples updated to use Copilot instead.

Reviewed Changes

Copilot reviewed 38 out of 38 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
pkg/workflow/claude_engine.go Sets experimental: true for Claude engine
pkg/workflow/claude_engine_test.go Updates test assertions to expect experimental status
pkg/cli/templates/github-agentic-workflows.instructions.md Updates engine documentation to mark Claude/Codex as experimental and recommends Copilot
.github/instructions/github-agentic-workflows.instructions.md Mirror of templates file with same experimental markings
docs/src/content/docs/reference/frontmatter-full.md Updates frontmatter reference to mark experimental engines and removes engine-specific config fields
docs/src/content/docs/reference/engines.md Adds experimental note and removes detailed Claude/Codex documentation sections
docs/src/content/docs/troubleshooting/errors.md Updates error example to be engine-agnostic
docs/src/content/docs/guides/mcps.md Changes example from Codex to Copilot
.github/workflows/*.lock.yml Updates compiled workflows to reflect experimental status (32 lock files)
Comments suppressed due to low confidence (1)

docs/src/content/docs/reference/engines.md:28

  • This line shows 'gpt-5' as the example model but states it defaults to 'claude-sonnet-4', which is inconsistent. If the Copilot engine defaults to claude-sonnet-4, the comment is correct but the example model name is confusing. Consider using a different example model that's actually available for Copilot, or clarifying that this is demonstrating how to override the default.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +912 to +913
# Optional specific LLM model to use (e.g., 'gpt-5', 'claude-sonnet-4'). Has
# sensible defaults and can typically be omitted.
Copy link

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The model name 'gpt-5' appears to be incorrect. OpenAI's latest models are GPT-4 series (as of January 2025). This should likely be 'gpt-4' or 'gpt-4-turbo'. If 'gpt-5' is intentional for future-proofing, this inconsistency should be documented.

Copilot uses AI. Check for mistakes.
@pelikhan pelikhan merged commit a2baa3b into main Nov 20, 2025
38 checks passed
@pelikhan pelikhan deleted the copilot/make-engines-experimental branch November 20, 2025 02:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants