[prompt-analysis] Copilot PR Prompt Analysis - November 18, 2025

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - November 18, 2025

This analysis examined 1,000 Copilot-generated pull requests from the last 30 days to identify patterns that correlate with successful merges versus closed PRs.

Key Finding: Copilot PRs show a strong 77.4% merge rate, with testing-related prompts performing slightly better at 78.3%.

Full Analysis Report

Summary

Analysis Period: Last 30 days
Total PRs: 1000 | Merged: 773 (77.4%) | Closed: 226 (22.6%) | Open: 1

Completion Rate

• 999 PRs completed (merged or closed)
• 77.4% merge rate - significantly higher than typical PR acceptance rates
• Only 1 PR still open - indicating fast review cycles

Prompt Categories and Success Rates

Category	Total	Merged	Closed	Success Rate
Testing	839	657	182	78.3%
Feature	899	702	197	78.1%
Documentation	896	697	199	77.8%
Refactor	374	290	84	77.5%
Bug Fix	989	765	224	77.4%

Category Insights

• Testing leads with 78.3% success rate (657/839 merged)
• All categories show 77-78% success rates - remarkably consistent
• Bug fixes are most common (989 PRs) but slightly lower success at 77.4%

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 461.2 words
• Most common keywords: copilot, agent, workflow, github, coding, workflows, details, summary
• Strong presence of: workflow automation, agent tasks, testing improvements

Example successful prompts:

1. PR #4251: ## Task: Add tools edit and bash * to agentic workflow The workflow at https://github.com/githubnext/gh-aw/actions/runs/19455176241/job/55667642759 f... → ✅ Merged
2. PR #4248: The copilot-pr-data-fetch.md shared component downloaded PR data fresh on every workflow run, causing redundant API calls and slower execution for w... → ✅ Merged
3. PR #4247: # ✅ Update Copilot Workflows to Use Shared Data Fetch - COMPLETE ## Objective Update copilot-* agentic workflows to use the shared `copilot-pr-data-f... → ✅ Merged

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 458.1 words (nearly identical to merged)
• Most common keywords: copilot, workflow, agent, github, coding, workflows, details, summary
• Similar topics but different outcomes

Example unsuccessful prompts:

1. PR #4237: Nested errors in gh aw update were rendered as single-line text, making deeply nested error chains unreadable. ## Changes - **Added `FormatNestedE... → ❌ Closed
2. PR #4235: Thanks for asking me to work on this. I will get started on it and keep this PR's description up to date as I form a plan and make progress. (!-- ST... → ❌ Closed
3. PR #4216: make lint fails when golangci-lint or npm packages are not installed, requiring manual intervention before linting can run. ### Changes - **Adde... → ❌ Closed

Key Insights

1. Consistent Success Across Categories: All prompt types show 77-78% success rates, suggesting the quality of Copilot's implementation matters more than prompt type
2. Prompt Length is Not a Factor: Merged (461 words) and closed (458 words) prompts are virtually identical in length
3. Testing Prompts Perform Best: At 78.3%, testing-related prompts have the highest success rate, though only marginally better than other categories

Recommendations

Based on the analysis of 1,000 Copilot PRs:

✅ DO:

1. Focus on testing tasks - they have the highest success rate at 78.3%
2. Include specific file references and clear objectives in prompts

❌ AVOID:

1. Avoid overly vague or general improvement requests

Historical Trends

Date	Total PRs	Merge Rate	Top Category
2025-11-18	1000	77.4%	Testing

Trend: First data point collected. Future runs will show trends over time.

Methodology

This analysis:

• Examined all 1,000 Copilot-generated PRs from the last 30 days
• Categorized prompts by type (bug fix, feature, testing, documentation, refactoring)
• Compared prompt characteristics between merged and closed PRs
• Identified keyword patterns and success correlations
• Tracked outcomes to establish baseline success metrics

Note: The remarkably high 77.4% merge rate suggests Copilot is generating high-quality PRs, with outcomes likely influenced more by code quality and review processes than prompt characteristics alone.

</details)

---

Data Source: Analysis of 1,000 PRs from githubnext/gh-aw repository
Generated: 2025-11-18 06:20 UTC

AI generated by Copilot PR Prompt Pattern Analysis

[github-actions[bot]]

This discussion was automatically closed because it was created by an agentic workflow more than 1 week ago.