[prompt-analysis] Copilot PR Prompt Analysis - 2026-01-04

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2026-01-04

Summary

Analysis Period: Last 30 days
Total PRs Analyzed: 998 (completed) | Merged: 754 (75.6%) | Closed: 244 (24.4%) | Still Open: 2

Prompt Categories and Success Rates

Category	Total	Merged	Success Rate
🧪 Test	647	501	77.4%
🔒 Security	141	108	76.6%
🐛 Bug Fix	708	537	75.8%
⚡ Performance	130	96	73.8%
✨ Feature	708	521	73.6%
🔄 Update	485	352	72.6%
🔧 Refactor	242	175	72.3%
📝 Documentation	569	405	71.2%
❓ Uncategorized	15	10	66.7%

Note: PRs can belong to multiple categories, so totals exceed 998.

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 453 words
• Code references: 99.7% include code blocks or inline code
• File references: 83.6% mention specific files or paths
• Most common keywords: copilot, agent, workflow, github, test, update, command

Standout characteristics:

• 13.7 percentage point advantage for PRs with code references
• 4.9 percentage point advantage for PRs with file references
• Test-related prompts have highest success rate (77.4%)
• Security-focused prompts also perform well (76.6%)

Example successful prompts:

1. PR Sync Sentry MCP server version reference with package.json #8100 - "Sync Sentry MCP server version reference with package.json" → Merged ✅

   • Why it worked: Specific file references, clear problem statement, focused scope
   • Preview: "Dependabot bumped @sentry/mcp-server from 0.24.0 to 0.26.0 in package.json, but the shared MCP configuration still referenced the old version..."

2. PR chore: fix formatting issues in devcontainer and test files #7981 - "chore: fix formatting issues in devcontainer and test files" → Merged ✅

   • Why it worked: Clear action items, specific files mentioned, code formatting context
   • Preview: "Fixed code formatting violations flagged by make lint. Changes: Consolidated VSCode extension array in .devcontainer/devcontainer.json..."

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 481 words (slightly longer but less focused)
• Code references: Only 86.1% include code blocks or inline code
• File references: Only 78.7% mention specific files or paths
• Issues identified: Vague scope, lack of code examples, unclear objectives

Example unsuccessful prompts:

1. PR [wip] test sample workflow #6926 - "[wip] test sample workflow" → Closed ❌

   • Why it failed: Minimal description (only 3 words), no context, no specific goals
   • Preview: "- Testing only..."

2. PR [WIP] Investigate failure in action run #7441 - "[WIP] Investigate failure in action run" → Closed ❌

   • Why it failed: No code references, no file references, investigative rather than action-oriented
   • Preview: "Thanks for asking me to work on this. I will get started on it and keep this PR's description up to date..."

Key Insights

Based on analysis of 998 completed PRs:

• 🎯 Code References Matter Most: PRs with code blocks or inline code references have a 13.7 percentage point higher success rate (99.7% vs 86.1%). Including actual code examples or referencing specific code patterns dramatically improves merge likelihood.

• 📁 Specific File References Help: PRs mentioning specific files or paths have a 4.9 percentage point advantage (83.6% vs 78.7%). Being explicit about which files need changes provides clear scope and direction.

• 🎪 Test & Security Prompts Excel: Test-related prompts achieve 77.4% success rate, and security-focused prompts reach 76.6%. These categories likely benefit from clear, measurable objectives and well-defined acceptance criteria.

• 📏 Brevity vs Context Balance: Merged PRs average 453 words while closed PRs average 481 words. This suggests that more words don't equal better outcomes—concise, focused prompts with relevant technical details outperform verbose, unfocused descriptions.

• 🔤 Generic Prompts Underperform: Uncategorized prompts (those lacking clear technical keywords) have the lowest success rate at 66.7%, suggesting that explicit framing of the task type helps Copilot generate more appropriate solutions.

Recommendations

Based on today's analysis:

✅ DO:

1. Include code examples or inline code references - Use backticks for functions, variables, or code snippets. This nearly guarantees inclusion in the 99.7% success group.

2. Mention specific files and paths - Reference exact filenames like pkg/workflow/validation.go or .github/workflows/test.yml to provide clear scope.

3. Frame tasks in clear categories - Use keywords like "fix bug in...", "add test for...", or "implement security check..." to help categorize the work.

4. Keep prompts focused and actionable - Aim for 300-500 words with clear objectives rather than lengthy, exploratory descriptions.

5. Include technical context - Reference error messages, test failures, or specific behaviors to provide concrete anchors for the implementation.

❌ AVOID:

1. Minimal descriptions - Three-word prompts like "Testing only" provide insufficient context for successful implementation.

2. Vague investigative requests - Prompts like "Investigate failure" without specific details, code references, or file mentions struggle to produce mergeable changes.

3. Generic improvement requests - Broad asks like "improve performance" without specific targets, metrics, or file references lead to unfocused changes.

Historical Trends

Date	PRs	Success Rate	Top Category
2026-01-04	998	75.6%	Test (77.4%)

Trend: First day of tracking—establishing baseline. Future reports will show week-over-week trends.

---

References:

• Workflow Run §20692537654

AI generated by Copilot PR Prompt Pattern Analysis