Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Dec 14, 2025

Playwright browser in GitHub Actions could not navigate to localhost URLs despite localhost being in the allowed domains. The Playwright MCP server uses two separate flags: --allowed-hosts controls MCP server CORS, while --allowed-origins controls browser navigation. We were only setting the former.

Changes:

  • Added --allowed-origins flag to Playwright MCP configurations for all engines (Claude, Copilot, Codex)
  • Both flags now receive the same domain list: localhost, localhost:*, 127.0.0.1, 127.0.0.1:*
  • Updated rendering functions in pkg/workflow/mcp-config.go and pkg/workflow/mcp_renderer.go
  • Extracted domain string formatting to reduce duplication
  • Updated test assertions to verify both flags are present

Example output:

"playwright": {
  "command": "docker",
  "args": [
    "run", "-i", "--rm", "--init",
    "mcr.microsoft.com/playwright/mcp",
    "--output-dir", "/tmp/gh-aw/mcp-logs/playwright",
    "--allowed-hosts", "localhost;localhost:*;127.0.0.1;127.0.0.1:*",
    "--allowed-origins", "localhost;localhost:*;127.0.0.1;127.0.0.1:*"
  ]
}

This enables documentation testing, local server testing, and other workflows that require Playwright to access localhost in GitHub Actions.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/user
    • Triggering command: /usr/bin/gh gh api user --jq .login 08431cddfb8368fd83d5badbf9bfd GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env tags/v6 GO111MODULE ps GOINSECURE GOMOD GOMODCACHE ps (http block)
    • Triggering command: /usr/bin/gh gh api user --jq .login -json 98934cb7db3cf861b1c3912a44c25fe853f52e101adce49a243 ache/go/1.25.0/x64/pkg/tool/linux_amd64/cgo 98934cb7db3cf861git (http block)
    • Triggering command: /usr/bin/gh gh api user --jq .login /run/containerd/io.containerd.runtime.v2.task/moby/412f2133922a5790ab7dae82709d4a5952fe5abddb2578c8a19a9a4c54e16330/init.pid 412f2133922a5790ab7dae82709d4a5952fe5abddb2578c8a19a9a4c54e16330 605619/b123=> a5952fe5abddb2578c8a19a9a4c54e16330/log.json bnext/gh-aw/cmd/aw -d/gh-aw go env s -d/gh-aw /opt/hostedtoolcache/go/1.25.0/xjson GOINSECURE GOMOD GOMODCACHE 605619/b123/impo/var/run/docker/runtime-runc/moby (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>🔍 Multi-Device Docs Testing Report - Technical Limitation Discovered</issue_title>
<issue_description>## Test Summary

Triggered by: @mnkiefer
Workflow Run: #20001915806
Test Date: 2025-12-07
Documentation Site: Built successfully at /docs/dist


Results Overview

Testing Status: Unable to complete browser-based testing
🔴 Critical Issue Identified: Playwright browser sandbox prevents localhost connections


Critical Issues

🔴 Playwright Network Isolation in GitHub Actions

Issue: The Playwright MCP server running in this GitHub Actions environment operates in a sandboxed browser that blocks connections to localhost and 127.0.0.1.

Impact:

  • Cannot perform visual regression testing
  • Cannot test responsive layouts across devices
  • Cannot validate accessibility with live browser interactions
  • Cannot capture screenshots of actual rendered pages

Technical Details:

Error: page.goto: net::ERR_CONNECTION_REFUSED at (redacted)

The documentation site builds successfully and serves correctly:

  • ✅ Build completed in ~16 seconds
  • ✅ Generated 60 pages with 98 workflow examples
  • ✅ Preview server starts on port 4321
  • ✅ Server responds to curl requests
  • ❌ Playwright browser cannot connect due to sandbox restrictions

Recommendations

Option 1: Deploy to Temporary Hosting

Instead of testing against localhost, deploy the built site to a publicly accessible URL:

  • Use GitHub Pages preview deployment
  • Use Netlify/Vercel deploy previews
  • Use ngrok or similar tunneling service

Option 2: Alternative Testing Approach

Use static analysis tools that don't require browser rendering:

  • HTML validation: Use html-validate or W3C validator
  • Accessibility: Use pa11y-ci or axe-core CLI
  • Link checking: Use linkinator or broken-link-checker
  • Responsive analysis: Parse CSS and check viewport rules

Option 3: Docker-based Testing

Run Playwright in a Docker container with proper network configuration:

- name: Run Playwright tests
  run: |
    docker run --network=host -v $PWD:/work -w /work \
      mcr.microsoft.com/playwright:latest \
      npm run test:e2e

Option 4: Use GitHub Actions Service Containers

Configure the workflow to use service containers that share network access:

jobs:
  test:
    services:
      docs:
        image: nginx:alpine
        ports:
          - 4321:80

What Was Tested

Build Process

  • Documentation builds successfully
  • All 98 lab workflows generated correctly
  • Mermaid diagrams processed (10 in architecture docs)
  • Pagefind search index created
  • Sitemap generated
  • All internal links validated

Server Configuration

  • Preview server starts correctly
  • Serves at /gh-aw/ base path
  • Returns valid HTML
  • Includes proper meta tags and SEO

Next Steps

To enable comprehensive multi-device testing for the documentation site, one of the following approaches should be implemented:

  1. Short-term: Use static analysis tools that don't require browser rendering
  2. Medium-term: Deploy to a preview URL before testing
  3. Long-term: Configure proper Docker/service container setup for full Playwright testing

Would you like me to implement any of these alternative testing approaches?


Build Artifacts

The documentation was successfully built to /home/runner/work/gh-aw/gh-aw/docs/dist with:

  • 60 HTML pages
  • 4,364 indexed words
  • 1 language (English)
  • All internal links validated
  • Sitemap and search index generated

AI generated by Multi-Device Docs Tester

Comments on the Issue (you are @copilot in this section)

@pelikhan /q the playwright default domain set should allow localhost 127.0.0.1

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI changed the title [WIP] Fix Playwright network isolation issue in GitHub Actions Add --allowed-origins flag for Playwright browser navigation to localhost Dec 14, 2025
Copilot AI requested a review from mnkiefer December 14, 2025 20:50
@pelikhan pelikhan marked this pull request as ready for review December 14, 2025 23:04
@pelikhan pelikhan merged commit 017e1fb into main Dec 15, 2025
125 checks passed
@pelikhan pelikhan deleted the copilot/fix-playwright-network-issue branch December 15, 2025 01:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🔍 Multi-Device Docs Testing Report - Technical Limitation Discovered

3 participants