fix: acceptance rate now uses suggestion/acceptance count #2170
+7
−2
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix acceptance rate calculation
We are migrating from https://github.com/github-copilot-resources/copilot-metrics-viewer/issues to the Backstage plugin, and noticed one critical difference in our metrics: The "Acceptance Rate Average".
The copilot plugin reports acceptance rate incorrectly, using the lines metrics rather than acceptance/suggestion count. It seems like the same code slightly later down the line does it correctly (minus the rate function):
community-plugins/workspaces/copilot/plugins/copilot/src/components/Cards/DashboardCards.tsx
Lines 52 to 64 in c3bbe0f
The GitHub API will respond with the following:
In this case, we expect acceptance rate to be 33%, not 20%, as the Backstage plugin reports. This is because copilot can generate several lines worth of suggestions, and one accept will shoot up (or down) the average significantly, whereas in
total_*_count
this is just a +1 to the value. Making it more reliable to report as an acceptance rate as it directly is the number of accepts/rejects, and is not influenced by the size of the suggestion.For reference, here is how copilot-metrics-viewer does it: https://github.com/github-copilot-resources/copilot-metrics-viewer/blob/223cb4fc9f73bd6e7263f8246d9a701aca1983cf/src/components/MetricsViewer.vue#L276-L280
✔️ Checklist
Signed-off-by
line in the message. (more info)