🎯 Repository Quality Improvement Report - Workflow Markdown Authoring Experience

[github-actions[bot]]

Analysis Date: 2026-01-01
Focus Area: Workflow Markdown Authoring Experience
Strategy Type: Custom
Custom Area: Yes - This focus addresses the core user experience of gh-aw: making it easy, fast, and error-free to write agentic workflows in markdown format.

Executive Summary

With 178 workflow markdown files in .github/workflows/, the Workflow Markdown Authoring Experience is critical to gh-aw's value proposition. Analysis reveals significant opportunities to improve the developer experience through better IDE support, clearer validation feedback, comprehensive starter templates, and integrated authoring tools. Currently, developers face inconsistent engine specifications (16 malformed entries), limited IDE autocomplete support, and minimal guidance for complex configurations like tool definitions and safe-outputs. Improving this experience will directly reduce time-to-first-workflow, lower error rates during compilation, and increase user satisfaction.

Full Analysis Report

Focus Area: Workflow Markdown Authoring Experience

Current State Assessment

The repository contains a mature workflow ecosystem with 178 workflow markdown files demonstrating diverse use cases. However, the authoring experience reveals several friction points that slow down workflow creation and increase compilation errors.

Metrics Collected:

Metric	Value	Status
Total workflow .md files	178	✅
Workflows using tool configurations	134 (75%)	✅
Workflows using safe-outputs	131 (74%)	✅
Workflows using imports	70 (39%)	✅
Engine specification errors	16 malformed	❌
Empty/blank engine fields	15 workflows	⚠️
Validation error handling lines	177 lines	⚠️
JSON schema files for validation	1 (agent-output only)	❌
Starter templates in examples/	~6 basic examples	⚠️
Documentation comments in workflows	3130 lines	✅
Average workflow size	~400 lines	✅
Largest workflows	748 lines (copilot-session-insights)	⚠️

Findings

Strengths

• Rich ecosystem: 178 diverse workflow examples serve as learning resources
• Import system: 70 workflows leverage shared imports for reusability
• Comprehensive tool usage: 75% of workflows configure custom tools
• Safe-outputs adoption: 74% use safe-outputs for GitHub API operations
• Inline documentation: Workflows average 17+ comment lines each
• VSCode Copilot enabled: Basic Copilot support for markdown editing

Areas for Improvement

• 🔴 No JSON Schema for frontmatter validation - IDEs cannot provide autocomplete, validation, or hover documentation for workflow frontmatter YAML
• 🟡 Inconsistent engine specifications - 16 workflows have malformed engine values; 15 have empty engine fields
• 🔴 Limited starter templates - Only 6 basic examples exist; no templates for common patterns (scheduled workflows, issue triage, PR automation, multi-repo operations)
• 🟡 Complex tool configuration syntax - Tool definitions lack IDE support and clear examples for advanced scenarios
• 🟡 Validation error messages - 177 error handling lines suggest complexity, but error messages may lack context for fixing issues
• 🔴 No workflow linting command - No gh aw lint or gh aw validate to check workflows before committing
• 🟡 Missing authoring guide - No comprehensive "Writing Your First Workflow" tutorial in documentation
• 🔴 No workflow scaffolding tool - No gh aw init command to generate workflow boilerplate

Detailed Analysis

1. IDE Support Gap

Current State: Only .vscode/settings.json exists with basic Copilot enablement. No JSON schemas for YAML frontmatter validation.

Impact:

• Developers must manually reference documentation for frontmatter fields
• Typos in field names go undetected until compilation
• No autocomplete for engine, tools, permissions, safe-outputs, etc.

Example Pain Point:

---
engine: coppilot  # Typo - should be "copilot"
permissons: contents:read  # Typo - should be "permissions"
---

These errors aren't caught until gh aw compile runs, wasting developer time.

2. Inconsistent Engine Specifications

Found:

• 72 workflows use engine: copilot
• 25 use engine: claude
• 15 have blank engine: fields
• 7 use engine: codex
• 16 have malformed engine values (detected via grep)

Impact: Blank or malformed engine fields cause compilation failures or unexpected behavior.

3. Limited Workflow Templates

Current Templates in examples/:

• label-trigger-simple.md
• label-trigger-pull-request.md
• label-trigger-discussion.md
• network-python-project.md
• network-node-project.md
• network-multi-language.md

Missing Templates:

• Scheduled workflow with time-based triggers
• Issue triage workflow with safe-outputs
• PR review automation with Copilot engine
• Multi-repo campaign workflow
• Custom MCP server integration
• Playwright browser automation
• Complex tool configuration patterns

4. Tool Configuration Complexity

Analysis: 134 workflows configure tools, but tool syntax varies significantly:

# Simple tool usage
tools:
  bash:
    - "*"

# Complex tool configurations
tools:
  github:
    mode: remote
    toolsets: [default]
  playwright:
    version: "v1.41.0"
    allowed_domains: ["github.com"]

No standardized patterns or IDE support make this error-prone.

5. Validation Feedback Quality

Found: 177 lines of validation error handling in pkg/workflow/*.go, but:

• Error messages may lack actionable fix suggestions
• No line numbers pointing to YAML issues
• No validation before gh aw compile

6. Documentation Gaps

Current Documentation:

• Examples exist but lack progressive learning path
• No "Your First Workflow" tutorial
• Network configuration guide exists (docs/src/content/docs/guides/network-configuration.md)
• No comprehensive frontmatter field reference

Missing:

• Step-by-step workflow authoring guide
• Frontmatter field reference with examples
• Common patterns cookbook
• Troubleshooting guide for compilation errors

---

🤖 Tasks for Copilot Agent

NOTE TO PLANNER AGENT: The following tasks are designed for GitHub Copilot agent execution. Please split these into individual work items for Claude to process.

Improvement Tasks

The following code regions and tasks should be processed by the Copilot agent. Each section is marked for easy identification by the planner agent.

---

Task 1: Create JSON Schema for Workflow Frontmatter Validation

Priority: High
Estimated Effort: Medium
Focus Area: Workflow Markdown Authoring Experience

Description:

Create a comprehensive JSON schema file that validates workflow frontmatter YAML, enabling IDE autocomplete, hover documentation, and real-time validation. This schema should cover all frontmatter fields: description, on, permissions, engine, tools, safe-outputs, safe-inputs, network, timeout-minutes, imports, features, runtimes, and sandbox.

The schema should be placed in schemas/workflow-frontmatter.schema.json and referenced in .vscode/settings.json to enable YAML language server integration.

Acceptance Criteria:

• JSON schema validates all frontmatter fields with correct types
• Schema includes descriptions for each field (shown as hover documentation)
• Schema provides enum values for engine field (copilot, claude, codex, custom)
• Schema includes common patterns for tools, permissions, and safe-outputs
• .vscode/settings.json updated to associate *.md files in .github/workflows/ with the schema
• Documentation updated with schema location and usage

Code Region: schemas/workflow-frontmatter.schema.json (new file), .vscode/settings.json

Create a JSON schema file at `schemas/workflow-frontmatter.schema.json` that validates gh-aw workflow frontmatter YAML. The schema should:

1. Define all top-level frontmatter fields (see AGENTS.md and existing workflows in .github/workflows/)
2. Provide clear descriptions for each field
3. Include enum constraints where applicable (e.g., engine: copilot|claude|codex|custom)
4. Define nested object structures for tools, permissions, safe-outputs, network, etc.
5. Include examples as part of field descriptions

Then update `.vscode/settings.json` to associate this schema with workflow markdown files using the YAML language server extension.

Reference existing workflows to understand field usage patterns.

---

Task 2: Add Workflow Linting Command (gh aw lint)

Priority: High
Estimated Effort: Medium
Focus Area: Workflow Markdown Authoring Experience

Description:

Implement a new CLI command gh aw lint (workflow.md) that validates workflow markdown files without full compilation. This command should check:

• Frontmatter YAML syntax
• Required field presence (description, engine, etc.)
• Engine value validity
• Tool configuration structure
• Safe-outputs syntax
• Permission definitions
• Network configuration validity

The linting should provide clear, actionable error messages with line numbers.

Acceptance Criteria:

• gh aw lint command accepts workflow file path(s)
• Command validates YAML frontmatter syntax
• Command checks required fields and valid values
• Error messages include line numbers and fix suggestions
• Exit code 0 for valid workflows, non-zero for errors
• Support for linting multiple files: gh aw lint .github/workflows/*.md
• Documentation updated with lint command usage

Code Region: pkg/cli/lint_command.go (new file), cmd/gh-aw/main.go

Create a new CLI command `gh aw lint` in the gh-aw codebase. This command should:

1. Parse workflow markdown frontmatter YAML
2. Validate required fields (description, engine)
3. Check engine value against allowed values
4. Validate tool configurations structure
5. Check safe-outputs and permissions syntax
6. Report errors with line numbers and actionable fix suggestions
7. Support multiple file paths as arguments

Use existing validation logic from `pkg/workflow/validation.go` where possible. Follow CLI patterns from other commands like `compile_command.go`.

Add command registration in `cmd/gh-aw/main.go` following existing patterns.

---

Task 3: Create Workflow Starter Templates Library

Priority: Medium
Estimated Effort: Large
Focus Area: Workflow Markdown Authoring Experience

Description:

Expand the examples/ directory with 10+ comprehensive workflow starter templates covering common use cases. Each template should be fully documented with inline comments explaining frontmatter fields, tool configurations, and workflow logic.

Template Categories:

1. Scheduled Workflows: Daily reports, weekly summaries, automated maintenance
2. Issue Triage: Auto-labeling, assignment, prioritization
3. PR Automation: Review assistance, status checks, merge coordination
4. Multi-Repo Operations: Cross-repository updates, dependency tracking
5. MCP Integration: Custom MCP server usage examples
6. Browser Automation: Playwright-based testing and scraping

Acceptance Criteria:

• At least 10 new workflow templates created in examples/workflows/
• Each template includes comprehensive inline documentation
• Templates cover diverse trigger types (schedule, issue, PR, discussion, workflow_dispatch)
• Templates demonstrate different engines (copilot, claude, codex)
• Examples show tool configuration patterns (bash, edit, GitHub MCP, Playwright)
• Safe-outputs examples for issues, PRs, discussions, and comments
• examples/workflows/README.md created with template index and usage guide
• Main README.md updated with link to template library

Code Region: examples/workflows/*.md (new files), examples/workflows/README.md (new file)

Create a comprehensive library of workflow starter templates in `examples/workflows/`. Design 10+ templates covering:

**Scheduled Workflows:**
- `daily-summary.md` - Daily repository activity report with safe-outputs
- `weekly-maintenance.md` - Automated dependency updates and cleanup

**Issue Triage:**
- `issue-auto-labeler.md` - Auto-label issues based on content analysis
- `issue-assignment.md` - Auto-assign issues to team members

**PR Automation:**
- `pr-review-assistant.md` - Copilot-powered PR review suggestions
- `pr-merge-coordinator.md` - Coordinate PR merges with checks

**Multi-Repo:**
- `cross-repo-sync.md` - Synchronize files across repositories
- `dependency-tracker.md` - Track dependency versions across repos

**MCP Integration:**
- `custom-mcp-workflow.md` - Use custom MCP server in workflow

**Browser Automation:**
- `playwright-testing.md` - Browser-based UI testing with Playwright

Each template should:
1. Include comprehensive frontmatter with all relevant fields
2. Have inline comments explaining each configuration option
3. Demonstrate best practices (permissions, network, safe-outputs)
4. Include example workflow logic with clear documentation
5. Show error handling patterns

Create `examples/workflows/README.md` with an index and usage instructions.

---

Task 4: Build Workflow Scaffolding Tool (gh aw init)

Priority: Medium
Estimated Effort: Large
Focus Area: Workflow Markdown Authoring Experience

Description:

Implement an interactive CLI command gh aw init that guides users through creating a new workflow via prompts. The tool should ask about:

• Workflow name and description
• Trigger type (schedule, issue, PR, discussion, manual)
• Engine preference (copilot, claude, codex)
• Tool requirements (bash, edit, GitHub MCP, Playwright)
• Safe-outputs needs (issues, PRs, discussions)
• Network access requirements

After collecting inputs, generate a workflow markdown file with appropriate frontmatter and boilerplate.

Acceptance Criteria:

• gh aw init command provides interactive prompts
• Prompts guide users through workflow configuration
• Command generates .github/workflows/(workflow-name).md file
• Generated workflow includes valid frontmatter and boilerplate
• Support for --template flag to use starter templates
• Support for --non-interactive mode with CLI flags
• Documentation updated with init command usage

Code Region: pkg/cli/init_command.go (new file), cmd/gh-aw/main.go

Create an interactive workflow scaffolding tool `gh aw init` in the gh-aw CLI. This command should:

1. Prompt user for workflow details:
   - Name (validates against filename rules)
   - Description (required)
   - Trigger type (schedule, issue, PR, discussion, workflow_dispatch)
   - Engine (copilot, claude, codex, custom)
   - Required tools (bash, edit, github, playwright, etc.)
   - Safe-outputs configuration
   - Network access needs

2. Generate workflow markdown file at `.github/workflows/(name).md`

3. Include comprehensive frontmatter based on user choices

4. Add boilerplate workflow content with TODO comments

5. Support flags:
   - `--template (name)` - Use a template from examples/workflows/
   - `--non-interactive` - Accept all defaults or use CLI flags

Use a prompting library like `survey` or `promptui` for interactive mode. Follow CLI patterns from existing commands.

Add command registration in `cmd/gh-aw/main.go`.

---

Task 5: Enhance Validation Error Messages with Fix Suggestions

Priority: Medium
Estimated Effort: Medium
Focus Area: Workflow Markdown Authoring Experience

Description:

Improve validation error messages in pkg/workflow/validation.go to include:

• Exact line numbers where errors occur
• Specific field names causing issues
• Suggested fixes or valid values
• Links to relevant documentation

Transform generic errors like "invalid engine value" into actionable messages like "Invalid engine 'coppilot' at line 3. Did you mean 'copilot'? Valid engines: copilot, claude, codex, custom. See (redacted)

Acceptance Criteria:

• All validation errors include line numbers
• Error messages suggest corrections (e.g., "Did you mean X?")
• Errors reference specific frontmatter fields
• Messages include links to documentation where helpful
• String similarity used for typo suggestions (e.g., "coppilot" → "copilot")
• Validation errors tested with unit tests
• Error message style guide followed (see specs/error-messages.md)

Code Region: pkg/workflow/validation.go, pkg/workflow/frontmatter_parser.go

Enhance validation error messages throughout the gh-aw validation system. Focus on:

1. **Line Number Reporting**: Track YAML line numbers during parsing and include in errors

2. **Typo Detection**: Use string similarity (Levenshtein distance) to suggest corrections
   - "Invalid engine 'coppilot'. Did you mean 'copilot'?"

3. **Field-Specific Guidance**: Provide context for each error
   - "Missing required field 'description' in frontmatter"
   - "Invalid permission 'read-all'. Valid permissions: read, write, none"

4. **Documentation Links**: Add helpful references
   - "See (redacted) for engine configuration"

5. **Multi-Error Reporting**: Show all validation errors, not just the first

Modify functions in `pkg/workflow/validation.go` and `pkg/workflow/frontmatter_parser.go`. Follow error message patterns from `specs/error-messages.md` and use `console.FormatErrorMessage()` for output.

Add unit tests for error message quality in corresponding test files.

---

📊 Historical Context

Previous Focus Areas

Date	Focus Area	Type	Custom	Key Outcomes
2026-01-01	Workflow Markdown Authoring Experience	Custom	Y	Initial quality improvement run

This is the first focus area analysis run. Future runs will track diversity and impact.

---

🎯 Recommendations

Immediate Actions (This Week)

1. Create JSON Schema (Task 1) - Priority: High

   • Enables immediate IDE validation and autocomplete
   • Low effort, high impact for all workflow authors

2. Add Workflow Linting (Task 2) - Priority: High

   • Catches errors before compilation
   • Reduces iteration time for developers

Short-term Actions (This Month)

1. Expand Starter Templates (Task 3) - Priority: Medium

   • Provides learning resources for new users
   • Demonstrates best practices

2. Enhance Error Messages (Task 5) - Priority: Medium

   • Improves debugging experience
   • Reduces support burden

Long-term Actions (This Quarter)

1. Build Scaffolding Tool (Task 4) - Priority: Medium
   • Streamlines workflow creation
   • Reduces time-to-first-workflow

---

📈 Success Metrics

Track these metrics to measure improvement in Workflow Markdown Authoring Experience:

• Engine specification errors: 16 → 0 (target: 100% valid)
• Time to first workflow: (establish baseline) → -50% (via scaffolding tool)
• Compilation error rate: (establish baseline) → -40% (via linting + better validation)
• IDE autocomplete usage: 0% → 80% (via JSON schema)
• Starter template usage: (establish baseline) → 50+ uses/month

Tracking Method:

• Monitor compilation error logs
• Track gh aw lint usage via telemetry
• Survey users on authoring experience
• Count template file usage in new workflows

---

Next Steps

1. Review and prioritize the tasks above
2. Assign Task 1 (JSON Schema) to Copilot agent via planner agent - high priority, quick win
3. Assign Task 2 (Linting Command) to Copilot agent - high priority, medium effort
4. Track progress on improvement items via GitHub issues or project board
5. Re-evaluate this focus area in 2-3 months to measure impact

---

Generated by Repository Quality Improvement Agent
Next analysis: 2026-01-02 - Focus area will be selected based on diversity algorithm (60% custom, 30% standard, 10% reuse)

AI generated by Repository Quality Improvement Agent

[pelikhan]

/plan review existing commands

[github-actions[bot]]

🎉 Yo ho ho! Plan Command found the treasure and completed successfully! ⚓💰