Copilot impact dashboard adds a return on investment section (#62701)

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Author: sophietheking

content/copilot/concepts/copilot-usage-metrics/copilot-metrics.md

@@ -130,6 +130,8 @@ To be grouped into a phase, a user must meet an engagement threshold of at least
 
 Phase assignment is recalculated daily from the trailing 28-day window, so a user's phase can shift from one day to the next as their activity within that window changes. This is expected behavior, not a data error.
 
+The impact dashboard counts every user who was active during the trailing 28-day window and groups each user by their current phase. The phase classification rules are the same, but the dashboard population is broader than the `totals_by_ai_adoption_phase.total_engaged_users` field in API and NDJSON reports, which counts users who were active on a specific day.
+
 Users who haven't met any phase's threshold are grouped into **Passive users**, which is not a measure of inactivity, but a signal that a user's engagement hasn't yet reached the level needed to reliably classify their adoption depth. For example, a user with fewer than two active days in the trailing 28-day window, or one who only lightly uses a surface like {% data variables.copilot.copilot_chat_short %} on {% data variables.product.prodname_dotcom_the_website %}, shows as a passive user.
 
 Because phase assignment is based on **which surfaces a user engages with**, not simply how many actions they take, a user with high completion volume but no agent usage stays in Phase 1, while a user with lighter but broader usage across agent surfaces progresses to Phase 2 or Phase 3.
@@ -142,6 +144,14 @@ This shows the relative impact of deeper adoption, independent of how many users
 
 For example, if users in the engaged cohorts averaged 20 pull requests/user/month and the passive cohort averaged 10, then the equation is 20 / 10 = 2x multiplier.
 
+### Estimating potential return on investment
+
+The impact dashboard's **Potential return on investment** section provides a comparison of cost and pull request output between **Phase 0-1 Passive and Code First Users** and **Phase 2-3 Agent First Users**.
+
+For each phase group, the dashboard shows the monthly {% data variables.product.prodname_copilot_short %} cost per developer, based on actual {% data variables.product.prodname_ai_credits_short %} consumption, and that cost as a percentage of developer compensation. It also shows average pull requests per developer per month.
+
+You can select a compensation band to update estimates that depend on developer compensation. These figures are estimates rather than exact financial results, so interpret them alongside the adoption multiplier's code-shipped and time-to-merge comparisons. The return-on-investment estimates are available only in the dashboard and are not included in the {% data variables.product.prodname_copilot_short %} usage metrics API or NDJSON exports.
+
 ### Interpreting pull request lifecycle metrics across scopes
 
 Pull request lifecycle metrics are available at both the organization and enterprise level. When comparing reports, keep the following in mind:

content/copilot/how-tos/administer-copilot/view-impact-dashboard.md

@@ -15,7 +15,7 @@ category:
 
 Instead of a flat active-user count, the impact dashboard groups users into adoption cohorts based on how they engage with {% data variables.product.prodname_copilot_short %}, and connects that engagement to pull request throughput. This gives you a more meaningful signal of adoption depth than daily or weekly active user counts alone.
 
-For a detailed explanation of what the dashboard shows, including adoption cohorts, engagement trends, and recommendations, see [AUTOTITLE](/copilot/concepts/copilot-usage-metrics/copilot-metrics).
+For a detailed explanation of what the dashboard shows, including adoption cohorts, engagement trends, potential return on investment, and recommendations, see [AUTOTITLE](/copilot/concepts/copilot-usage-metrics/copilot-metrics).
 
 ## Prerequisite
 
@@ -25,3 +25,12 @@ For a detailed explanation of what the dashboard shows, including adoption cohor
 
 {% data reusables.copilot.access-copilot-metrics-dashboard %}
 1. In the left sidebar, click **{% data variables.product.prodname_copilot_short %} impact**.
+
+## Estimating potential return on investment
+
+The "Potential return on investment" section provides a directional comparison of cost and pull request output across adoption phases.
+
+1. Under "Average developer cost in your organization", select a compensation band.
+1. In the "Transition your developers to be agent-first" card, compare the cost, payroll, and pull request estimates for "Phase 0-1 Passive and Code First Users" and "Phase 2-3 Agent First Users".
+
+Treat the figures as directional estimates rather than exact financial results. Use them with the adoption multiplier metrics for code shipped and time to merge pull requests.

content/copilot/reference/copilot-usage-metrics/interpret-copilot-metrics.md

@@ -86,6 +86,16 @@ The adoption multiplier compares engaged users (Phase 1, 2, or 3) against passiv
 > [!NOTE]
 > The adoption multiplier compares two different populations of users, not the same users over time. Differences in team composition, project complexity, or seniority between engaged and passive users can influence the comparison as much as {% data variables.product.prodname_copilot_short %} adoption itself.
 
+## Reviewing potential return on investment
+
+The **Potential return on investment** section compares {% data variables.product.prodname_copilot_short %} cost, estimated percentage of payroll, and pull requests per developer per month across earlier and agent-first adoption phases. Treat these comparisons as directional associations, not proof that a change in adoption caused a change in output.
+
+| Signal | What it suggests | Suggested action |
+|:--|:--|:--|
+| Agent-first users merge more pull requests with a similar or lower cost as a percentage of payroll | Agent-first usage is associated with more pull request output relative to the estimated investment. | Compare this signal with the adoption multiplier and trends over multiple periods before using it to inform enablement investment. |
+| Agent-first users merge fewer pull requests or have a higher cost as a percentage of payroll | Differences in work type, team composition, or adoption maturity may be affecting the comparison. | Review cohort distribution and team-level context before drawing conclusions about return on investment. |
+| The payroll percentage changes after you select a different developer cost | The dashboard is recalculating the estimate using a different compensation assumption, not showing a change in usage. | Use the same compensation band when comparing different periods or organizations. |
+
 ## Acting on your insights
 
 Use trends in usage, feature adoption, and language activity to guide enablement and rollout planning.

content/copilot/reference/copilot-usage-metrics/reconciling-usage-metrics.md

@@ -43,7 +43,7 @@ The dashboard and APIs use shared definitions for key metrics:
 | Acceptance rate | Code completion acceptance rate | `code_acceptance_activity_count` ÷ `code_generation_activity_count` | Both sources calculate acceptance rate the same way, though rounding may differ. |
 | Agent adoption | Agent adoption chart | `totals_by_feature` where feature = “agent” | Reflects users who interacted with the {% data variables.copilot.copilot_agent_short %}. |
 | Language usage | Language usage charts | `totals_by_language_feature` or `totals_by_language_model` | The dashboard visualizes these aggregated fields. |
-| Adoption cohort/phase | Adoption cohort distribution (impact dashboard) | `ai_adoption_phase`, `totals_by_ai_adoption_phase` | The dashboard and API read the same underlying field, so cohort counts should match between them. Users displayed as "Passive users" in the dashboard correspond to the `No Cohort` value in `totals_by_ai_adoption_phase`, and are included in that array like any other phase, so cohort percentages reflect all licensed users, not just those in Phase 1–3. |
+| Adoption cohort/phase | Adoption cohort distribution (impact dashboard) | `ai_adoption_phase`, `totals_by_ai_adoption_phase` | The phase definitions are shared, but the populations differ. The dashboard includes every user active during the trailing 28-day window, while the aggregated `totals_by_ai_adoption_phase` field counts users active on the record's specific day. Dashboard and API cohort counts and percentages therefore do not match. Users displayed as "Passive users" in the dashboard correspond to the `No Cohort` API value. |
 
 For complete field descriptions, see [AUTOTITLE](/copilot/reference/copilot-usage-metrics).
 
@@ -61,7 +61,7 @@ Each data source aggregates data differently.
 | APIs | Daily | Each record represents a single day per user, enabling daily trend analysis. |
 | NDJSON exports | Daily | Mirrors API output for BI tools and long-term reporting. |
 
-Aligning your reporting period with the dashboard’s 28-day window ensures consistent comparisons.
+Aligning your reporting period with the dashboard’s 28-day window makes most comparisons more consistent. However, it does not make adoption cohort counts match because the dashboard counts distinct users across the window and the aggregated API fields contain daily counts.
 
 ### Delayed telemetry