Add new 'test' command for single workflow local execution with act #820

Copilot · 2025-09-14T16:37:45Z

This PR implements a new test command that allows users to run a single agentic workflow locally using Docker and the nektos/act tool, addressing the need for local testing and development of workflows before deploying them to GitHub Actions.

What's Changed

The new test command provides a streamlined local testing solution with the following capabilities:

Manual act installation: Detects if the act tool is available and provides clear installation instructions if needed (no automatic installation)
Workflow compilation: Automatically compiles markdown workflows to YAML before execution with safe-outputs forced to staged mode
Single workflow focus: Tests one workflow at a time for simplified and reliable execution
Safe local testing: Automatically sets safe-outputs to staged mode to prevent real GitHub API calls during local testing
Merge origin/main: Successfully merged latest changes from main branch including new env and github-token features
Automated act installation: Added Makefile task for installing act via GitHub CLI extension

Usage Examples

# Test a single workflow with default settings
gh aw test weekly-research

# Test with custom event type
gh aw test weekly-research --event push

# Install act tool for local testing
make install-act

# Install all development dependencies including act
make deps-dev

Implementation Details

Modular CLI Architecture: All test-related code moved to dedicated pkg/cli/test.go file for better organization
Act Installation Instructions: checkActInstalled() function provides clear instructions for manual installation via GitHub CLI or direct download
Safe Compilation: New CompileWorkflowForTesting() function forces safe-outputs to staged mode using SetForceStaged() compiler method
Simplified Interface: Command accepts exactly one workflow (no multi-workflow support) with minimal flags
Error Handling: Proper error messages and user guidance for common issues
Console Formatting: Uses existing console styling functions for consistent output
Merge Resolution: Successfully resolved merge conflicts between SetForceStaged functionality and new env/github-token features
Makefile Integration: Added install-act task using gh extension install https://github.com/nektos/act and included it as dependency in deps-dev

Testing

Updated unit tests to reflect single-workflow API changes and new manual installation behavior
Validation tests cover the new TestWorkflowLocally() function signature
All core functionality validated with build and compilation tests
Code follows existing patterns and maintains compatibility
All tests pass after merge from origin/main
Agent-finish validation completed successfully
New makefile tasks properly integrated and help documentation updated

The command provides a focused, safe approach to local workflow testing with clear user guidance for act installation, and now includes automated installation during development setup.

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: pelikhan <[email protected]>

pelikhan

@copilot

compile the agentic workflow with safe-outputs set to staged if any

cmd/gh-aw/main.go

pelikhan · 2025-09-14T17:36:17Z

cmd/gh-aw/main.go


+	// Add flags to test command
+	testCmd.Flags().StringP("event", "e", "workflow_dispatch", "Event type to simulate (workflow_dispatch, push, pull_request, etc.)")
+	testCmd.Flags().StringP("platform", "p", "", "Platform mapping for act (e.g., ubuntu-latest=catthehacker/ubuntu:act-latest)")


@copilot remove platform flag

Done! Removed platform flag from test command. The flag is no longer available in the CLI interface. Commit: 81cb6fd

pelikhan · 2025-09-14T17:36:35Z

cmd/gh-aw/main.go

+	// Add flags to test command
+	testCmd.Flags().StringP("event", "e", "workflow_dispatch", "Event type to simulate (workflow_dispatch, push, pull_request, etc.)")
+	testCmd.Flags().StringP("platform", "p", "", "Platform mapping for act (e.g., ubuntu-latest=catthehacker/ubuntu:act-latest)")
+	testCmd.Flags().Bool("dry-run", false, "Dry run - show what would be executed without running")


@copilot remove dry-run flag

Done! Removed dry-run flag from test command. The flag is no longer available in the CLI interface. Commit: 81cb6fd

… force staged mode Co-authored-by: pelikhan <[email protected]>

Copilot · 2025-09-14T17:48:45Z

@copilot

compile the agentic workflow with safe-outputs set to staged if any...

Done! Added SetForceStaged() method to the compiler and modified CompileWorkflowForTesting() to automatically set safe-outputs to staged mode when any safe-outputs configuration exists. This ensures local testing is safe. Commit: 81cb6fd

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

api.contoso.com
- Triggering command: /tmp/go-build1970098009/b001/cli.test -test.testlogfile=/tmp/go-build1970098009/b001/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
- Triggering command: /tmp/go-build2039996565/b278/cli.test -test.testlogfile=/tmp/go-build2039996565/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Co-authored-by: pelikhan <[email protected]>

pelikhan · 2025-09-15T02:53:20Z

pkg/cli/test.go

+	}
+
+	// Install act via GitHub CLI extension
+	fmt.Println(console.FormatProgressMessage("Installing act via GitHub CLI extension..."))


@copilot Ask user to install act extension and fail.

Provide instructions so that the user can copy/paste the command to install act.

Do not install act automatically.

Updated the checkActInstalled function to fail when act is not installed and provide clear copy/paste installation instructions instead of attempting automatic installation. The function now displays both GitHub CLI extension installation command (gh extension install https://github.com/nektos/act) and manual installation URL. Commit: 4391e2c

…tructions Co-authored-by: pelikhan <[email protected]>

Copilot · 2025-09-15T03:02:22Z

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

api.contoso.com
- Triggering command: /tmp/go-build2826123294/b278/cli.test -test.testlogfile=/tmp/go-build2826123294/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
- Triggering command: /tmp/go-build2740780253/b001/cli.test -test.testlogfile=/tmp/go-build2740780253/b001/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
- Triggering command: /tmp/go-build3396516522/b001/cli.test -test.testlogfile=/tmp/go-build3396516522/b001/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

pelikhan · 2025-09-15T05:19:03Z

Untested but the need for a better dev loop is definitely something I would like to address.

pelikhan · 2025-09-15T14:38:06Z

@copilot merge origin/main and fix tests

…ged with env/github-token features

Copilot · 2025-09-15T14:45:34Z

@copilot merge origin/main and fix tests

Completed! Successfully merged origin/main and resolved all merge conflicts. The SetForceStaged functionality for test command now works alongside the new env and github-token features from main. All tests pass and agent-finish validation completed successfully. Commit: e597d9d

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

api.contoso.com
- Triggering command: /tmp/go-build2486396845/b278/cli.test -test.testlogfile=/tmp/go-build2486396845/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

pelikhan · 2025-09-15T15:31:05Z

@copilot run "gh-aw test dev" and report output

Copilot · 2025-09-15T15:33:14Z

@copilot run "gh-aw test dev" and report output

Running gh-aw test dev produced the following output:

✗ act tool is required but not installed

ℹ To install act, please run one of the following commands:

⚡ gh extension install https://github.com/nektos/act

ℹ Or install manually from: https://nektosact.com
error: testing workflow locally: failed to ensure act is available: act tool is not installed

The command correctly detected that the act tool is not installed and provided clear installation instructions as designed. The dev workflow exists at .github/workflows/dev.md and contains a Claude-powered poetry generation workflow for testing purposes.

pelikhan · 2025-09-15T15:53:00Z

@copilot

add a makefile task that installs act

gh extension install https://github.com/nektos/act

add task as dependency to dev-deps

Co-authored-by: pelikhan <[email protected]>

Copilot · 2025-09-15T15:56:59Z

@copilot

add a makefile task that installs act
...

Added install-act makefile task that runs gh extension install https://github.com/nektos/act and included it as a dependency in deps-dev. The help documentation has also been updated. Commit: 6dcaffe

pelikhan · 2025-09-15T15:59:00Z

@colipot run test command for dev

make dev-deps
./gh-aw test dev

…2037

…cks in workflow testing

Initial plan

019cc62

Copilot AI assigned Copilot and pelikhan Sep 14, 2025

Copilot started work on behalf of pelikhan September 14, 2025 16:37 View session

Add new 'test' command for local workflow execution with act

deedc40

Co-authored-by: pelikhan <[email protected]>

Copilot finished work on behalf of pelikhan September 14, 2025 16:52

Copilot AI requested a review from pelikhan September 14, 2025 16:52

pelikhan reviewed Sep 14, 2025

View reviewed changes

Copilot started work on behalf of pelikhan September 14, 2025 17:37 View session

Refactor test command: move to pkg/cli/test.go, single workflow only,…

81cb6fd

… force staged mode Co-authored-by: pelikhan <[email protected]>

Final validation and completion of test command refactoring

318294c

Co-authored-by: pelikhan <[email protected]>

Copilot AI changed the title ~~Add new 'test' command for local workflow execution with act~~ Add new 'test' command for single workflow local execution with act Sep 14, 2025

Copilot finished work on behalf of pelikhan September 14, 2025 17:52

Copilot AI requested a review from pelikhan September 14, 2025 17:52

pelikhan reviewed Sep 15, 2025

View reviewed changes

Copilot started work on behalf of pelikhan September 15, 2025 02:53 View session

Update test command to require manual act installation with clear ins…

4391e2c

…tructions Co-authored-by: pelikhan <[email protected]>

Copilot AI requested a review from pelikhan September 15, 2025 03:02

Copilot finished work on behalf of pelikhan September 15, 2025 03:02

pelikhan marked this pull request as ready for review September 15, 2025 05:18

Copilot started work on behalf of pelikhan September 15, 2025 14:38 View session

Resolve merge conflict in pkg/workflow/compiler.go: merge SetForceSta…

e597d9d

…ged with env/github-token features

Copilot finished work on behalf of pelikhan September 15, 2025 14:46

Copilot started work on behalf of pelikhan September 15, 2025 15:31 View session

Copilot finished work on behalf of pelikhan September 15, 2025 15:34

Copilot started work on behalf of pelikhan September 15, 2025 15:53 View session

Add install-act makefile task and include in deps-dev

6dcaffe

Co-authored-by: pelikhan <[email protected]>

Copilot finished work on behalf of pelikhan September 15, 2025 15:58

pelikhan added 4 commits September 15, 2025 14:21

Merge branch 'main' into copilot/fix-ecf7843b-21e3-4064-ad2c-27d959c4…

27bf550

…2037

Merge branch 'main' into copilot/fix-ecf7843b-21e3-4064-ad2c-27d959c4…

cb71b7e

…2037

🛠️ Fix: Ensure GH_TOKEN is set for installing development dependencies

9c0c03a

🛠️ Refactor: Update act installation command and remove redundant che…

1c4265a

…cks in workflow testing

pelikhan closed this Sep 15, 2025

pelikhan deleted the copilot/fix-ecf7843b-21e3-4064-ad2c-27d959c42037 branch September 17, 2025 15:13

Add new 'test' command for single workflow local execution with act #820

Add new 'test' command for single workflow local execution with act #820

Uh oh!

Conversation

Copilot AI commented Sep 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What's Changed

Usage Examples

Implementation Details

Testing

Uh oh!

pelikhan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

pelikhan Sep 14, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 14, 2025

Choose a reason for hiding this comment

Uh oh!

pelikhan Sep 14, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 14, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI commented Sep 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

pelikhan Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI commented Sep 15, 2025

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

pelikhan commented Sep 15, 2025

Uh oh!

pelikhan commented Sep 15, 2025

Uh oh!

Copilot AI commented Sep 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

pelikhan commented Sep 15, 2025

Uh oh!

Copilot AI commented Sep 15, 2025

Uh oh!

pelikhan commented Sep 15, 2025

Uh oh!

Copilot AI commented Sep 15, 2025

Uh oh!

pelikhan commented Sep 15, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Sep 14, 2025 •

edited

Loading

Copilot AI commented Sep 14, 2025 •

edited

Loading

Copilot AI commented Sep 15, 2025 •

edited

Loading