[prompt-analysis] Copilot PR Prompt Analysis - 2026-01-02

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2026-01-02

Summary

Analysis Period: Last 30 days
Total PRs: 1000 | Merged: 755 (76.0%) | Closed: 239 (24.0%) | Open: 6

Overall Success Rate: 76.0% of completed PRs were merged

Prompt Categories and Success Rates

Category	Total	Merged	Success Rate
Bug Fix	956	741	77.5%
Feature Addition	894	683	76.4%
Testing	808	628	77.7%
Refactoring	455	347	76.3%
Documentation	874	648	74.1%

Full Report

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 456 words
• Average title length: 8.5 words
• Most common keywords: copilot, agent, coding, github, workflow, details, https, start, summary, actions
• Top action verbs: fix, add, update, create, implement
• Code references: 99.9% of merged PRs include code references
• File references: 84.1% of merged PRs reference specific files

Example successful prompts:

1. PR Convert Campaign specs ASCII diagram to Mermaid with icons #8601: Convert Campaign specs ASCII diagram to Mermaid with icons
   • Preview: Replaced the ASCII art mental model diagram in Campaign specs documentation with a Mermaid flowchart using Font Awesome icons for improved visual clarity and accessibility.

Changes

• **Diagram f...
  • Status: ✅ Merged

2. PR Add Gastown multi-agent orchestration concept mapping #8586: Add Gastown multi-agent orchestration concept mapping
   • Preview: Deep analysis of Steve Yegge's Gastown multi-agent orchestrator, mapping its concepts to GitHub Agentic Workflows architecture and identifying enhancement opportunities.

New Documentation

**`spec...

• Status: ✅ Merged

3. PR Fix fuzz test compilation errors from gateway function signature changes #8584: Fix fuzz test compilation errors from gateway function signature changes
   • Preview: ## Summary

Fixed fuzz test failures in CI by resolving compilation errors in test files.

Changes Made

✅ Fixed Gateway Test Function Signatures (pkg/workflow/gateway_test.go)

• Updated 6 func...
  • Status: ✅ Merged

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 472 words
• Average title length: 8.2 words
• Most common keywords: copilot, agent, workflow, coding, github, details, githubnext, start, https, workflows
• Top action verbs: fix, add, update, create, implement
• Code references: 87.0% of closed PRs include code references
• File references: 77.8% of closed PRs reference specific files

Example unsuccessful prompts:

1. PR Add safe output support for PR draft status and completion workflow instructions #8577: Add safe output support for PR draft status and completion workflow instructions

   • Preview: All 6 recent Copilot agent PRs remain stuck in draft status. Agents lack instructions on when/how to mark PRs ready for review, and the existing safe output mechanism didn't support changing draft sta...
   • Status: ❌ Closed without merge

2. PR [WIP] Fix repo-memory configuration in meta-orchestrator workflows #8576: [WIP] Fix repo-memory configuration in meta-orchestrator workflows

   • Preview: ## Fix repo-memory configuration in meta-orchestrator workflows

• Fix metrics-collector.md repo-memory config
  • Add explicit id: default
  • Change branch-name from `memory/meta-orc...
  • Status: ❌ Closed without merge

3. PR Document shell completion setup in README and installation docs #8560: Document shell completion setup in README and installation docs
   • Preview: The gh aw completion command exists and works correctly, but lacks user-facing documentation for setup and troubleshooting.

Changes

README.md

• Added Shell Completion section after Quick St...
  • Status: ❌ Closed without merge

Key Insights

Based on analysis of 1000 PRs from the last 30 days:

• Code references matter: Merged PRs have 99.9% code reference rate vs 87.0% in closed PRs—nearly universal in successful PRs
• File specificity helps: 84.1% of merged PRs reference specific files vs 77.8% in closed PRs
• Testing PRs excel: Testing-focused PRs show the highest success rate at 77.7%
• Similar prompt lengths: Both merged (456 words) and closed (472 words) PRs have comparable description lengths—suggesting quality over quantity
• Action verb preference: "Fix" and "add" dominate successful PRs, appearing 736 and 640 times respectively

Recommendations

Based on the data analysis:

1. DO: Include code snippets and backtick-formatted references—99.9% of merged PRs use this pattern
2. DO: Reference specific files when possible—84% of merged PRs mention file paths
3. DO: Focus on testing improvements—they have the highest success rate at 77.7%
4. CONSIDER: Using "fix" or "add" as your primary action verb—they're the most common in successful PRs
5. NOTE: Prompt length alone doesn't determine success—focus on clarity and specificity instead

Data Quality Notes

• Analysis based on 1000 PRs created by app/copilot-swe-agent in the last 30 days
• 6 PRs remain open and are excluded from success rate calculations
• Success rate calculated as: merged PRs / (merged + closed) = 76.0%
• Category assignments are not mutually exclusive—PRs can belong to multiple categories

---

References:

• §20657361979

AI generated by Copilot PR Prompt Pattern Analysis