Implement MCP gateway step generation with multi-agent support

[Copilot]

The MCP gateway step (sandbox.mcp) was not being generated despite configuration in workflows. The step should start a gateway process that proxies MCP servers through a unified HTTP endpoint, following the MCP Gateway Specification.

Changes

Configuration parsing (pkg/workflow/)

• Extended SandboxConfig with MCP *MCPGatewayRuntimeConfig field
• Added extractMCPGatewayConfig() to parse sandbox.mcp from frontmatter
• Handles command/container, port, domain, API key, args, entrypointArgs, env variables
• Fixed port type handling (uint64 from YAML parser)

Gateway shell script (actions/setup/sh/start_mcp_gateway.sh)

• Created reusable shell script containing complete gateway lifecycle logic
• Handles configuration building, process start, health polling, output capture, and error handling
• Accepts configuration via environment variables (MCP_GATEWAY_PORT, MCP_GATEWAY_DOMAIN, MCP_GATEWAY_API_KEY, MCP_GATEWAY_COMMAND/CONTAINER)
• Follows same pattern as other action scripts (e.g., start_safe_inputs_server.sh)
• Implements intelligent engine detection to call appropriate converter

Agent-specific configuration converters (actions/setup/sh/)

• Created three converter scripts to transform gateway output to agent-specific formats:
  • convert_gateway_config_copilot.sh - Copilot CLI format (JSON with type and tools fields)
  • convert_gateway_config_codex.sh - Codex format (TOML with [mcp_servers] sections)
  • convert_gateway_config_claude.sh - Claude format (clean JSON without Copilot-specific fields)
• Gateway outputs standard HTTP format per specification
• Converters add/remove agent-specific fields as required
• Auto-detects agent type via GH_AW_ENGINE environment variable or config file presence
• Extensible design allows adding converters for additional agents

Step generation (pkg/workflow/mcp_servers.go)

• Gateway start integrated into "Setup MCPs" step (not a separate step)
• Added generateMCPGatewayStepInline() triggered by features.mcp-gateway: true
• Exports GH_AW_ENGINE environment variable with engine ID (copilot/codex/claude)
• Exports other environment variables and invokes start_mcp_gateway.sh script
• Implements spec-compliant workflow:
  1. Pipes mcp-config.json via stdin to gateway process
  2. Polls /health endpoint (30 attempts, 1s intervals)
  3. Captures stdout for rewritten configuration
  4. Converts gateway output to agent-specific format (Copilot/Codex/Claude)
  5. Updates agent MCP config with gateway-proxied servers
  6. Collects stderr logs to /tmp/gh-aw/mcp-logs/gateway/
• Supports both direct command and Docker container execution modes
• Fail-fast on errors with diagnostic output

Example configuration:

sandbox:
  agent: false
  mcp:
    container: ghcr.io/githubnext/awmg
    version: latest
    port: 8080
features:
  mcp-gateway: true

The gateway lifecycle (start, health check, output capture, PID storage) is managed by the shell script, with agent-specific configuration conversion handled by dedicated converter scripts for each supported agent (Copilot, Codex, and Claude). The "Setup MCPs" step orchestrates MCP configuration generation, gateway initialization, and format conversion in a single cohesive step.

Original prompt

Update the sandbox.mcp gateway start MCP gateway step code implementation to comply with the specification at https://github.com/githubnext/gh-aw/blob/main/docs/src/content/docs/reference/mcp-gateway.md

Pay special attention to the configuration format, secrets compliance, piping stdout to retrieve update configuration, or errors. Collect logs.

Use /health to wait until the configuration should be ready and written to stdout.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[pelikhan]

@copilot

• move shell script to actions/setup/sh
• generate MCP configuration information in a step that replaces "setup MCPs"

[pelikhan]

@copilot add a step that converts the configuration from the gateway into the configuration expected by the agent. This script is agent-specific.

[Copilot]

@copilot

• move shell script to actions/setup/sh ...

Done in commit 026d19b:

1. Moved shell script to actions/setup/sh: Created start_mcp_gateway.sh containing all gateway lifecycle logic (config building, process start, health checks, output capture)

2. Integrated into Setup MCPs step: Gateway start is now part of the "Setup MCPs" step, not a separate step. The step generates MCP configuration and then starts the gateway if enabled.

The workflow now shows:

- name: Setup MCPs
  run: |
    # MCP config generation...
    
    # Start MCP gateway
    export MCP_GATEWAY_PORT="8080"
    export MCP_GATEWAY_DOMAIN="localhost"
    export MCP_GATEWAY_API_KEY="$(openssl rand -base64 45 | tr -d '/+=')"
    export MCP_GATEWAY_CONTAINER='docker run...'
    
    # Run gateway start script
    bash /tmp/gh-aw/actions/start_mcp_gateway.sh

[Copilot]

@copilot add a step that converts the configuration from the gateway into the configuration expected by the agent. This script is agent-specific.

Done in commit 6110728.

Added convert_gateway_config_copilot.sh that converts the gateway's standard HTTP output format to Copilot CLI's expected format:

Gateway output (standard):

{
  "mcpServers": {
    "server": {
      "type": "http",
      "url": "http://localhost:8080/mcp/server",
      "headers": {"Authorization": "apiKey"}
    }
  }
}

Copilot format (converted):

{
  "mcpServers": {
    "server": {
      "type": "http",
      "url": "http://localhost:8080/mcp/server",
      "headers": {"Authorization": "apiKey"},
      "tools": ["*"]
    }
  }
}

The converter:

• Adds tools: ["*"] field required by Copilot
• Ensures Authorization header uses actual API key value
• Auto-detected based on presence of /home/runner/.copilot directory
• Extensible design allows adding converters for other agents

[pelikhan]

@copilot ensure conversion for codex and Claude

[Copilot]

@copilot ensure conversion for codex and Claude

Done in commit b40ec39.

Added converters for all three major agents:

Codex (convert_gateway_config_codex.sh):

• Converts gateway JSON to TOML format
• Outputs to /tmp/gh-aw/mcp-config/config.toml
• Format:

[history]
persistence = "none"

[mcp_servers.server-name]
url = "http://localhost:8080/mcp/server-name"

[mcp_servers.server-name.headers]
Authorization = "apiKey"

Claude (convert_gateway_config_claude.sh):

• Converts to clean JSON without Copilot fields
• Outputs to /tmp/gh-aw/mcp-config/mcp-servers.json
• Removes type and tools fields (Claude doesn't use them)

Engine detection:

• Exports GH_AW_ENGINE variable (copilot/codex/claude)
• Auto-detects from variable or config file presence
• Fallback to default if detection fails

All three agents now supported with proper format conversion.