Add Agentic Workflow Portfolio Yield workflow#31363
Conversation
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Comment MemoryNote This comment is managed by comment memory.It stores persistent context for this thread in the code block at the top of this comment. |
There was a problem hiding this comment.
Pull request overview
Adds an “Agentic Workflow Portfolio Yield” workflow that deterministically scores and analyzes the repository’s agentic workflows (yield, overlap/drag, governance signals) and then produces an issue report, with optional OTel-backed evidence.
Changes:
- Introduces deterministic precompute/postcompute Python scripts to generate portfolio metrics and a markdown report.
- Adds a new scheduled/dispatchable agentic workflow that runs the scripts and creates a single report issue via safe outputs.
- Adds shared OTel observability import and initial unit tests for the scoring/parsing logic.
Show a summary per file
| File | Description |
|---|---|
scripts/aw_yield_precompute.py |
New deterministic precompute: workflow discovery, frontmatter parsing, scoring, overlap detection, portfolio metrics. |
scripts/aw_yield_postcompute.py |
New deterministic postcompute: clamps scores, validates agent recommendations, generates report markdown + safe output payload. |
tests/test_aw_yield_precompute.py |
Unit tests for discovery/frontmatter parsing/import handling/scoring math/overlap clustering. |
tests/test_aw_yield_postcompute.py |
Unit tests for clamping, recommendation validation, and malformed agent-output handling. |
.github/workflows/aw-portfolio-yield.md |
New agentic workflow definition that runs weekly and on demand, producing a portfolio report issue. |
.github/workflows/shared/otel-observability.md |
Shared import that configures OTLP env/observability and an otel MCP server. |
.github/workflows/aw-portfolio-yield.lock.yml |
Compiled lockfile for the new workflow. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 7/7 changed files
- Comments generated: 3
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
|
Hey
If you'd like a hand drafting it, you can assign this prompt to your coding agent:
|
|
@copilot use JavaScript and actions/github-script in the generated code instead python or bash. |
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Implemented in |
Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
|
@copilot Replace the PR body for #31363 with a proper description. The description should:
|
New Feature
What does this feature do?
This PR adds Agentic Workflow Portfolio Yield, a scheduled/manual agentic workflow that evaluates the repository’s workflow portfolio in two deterministic stages and then asks the agent to interpret the results. The precompute stage inventories workflow metadata and calculates workflow yield, overlap drag, portfolio cost and risk, agentic vs. deterministic fractions, telemetry coverage, evidence quality, overlap clusters, recommendation seeds, and organizational health signals. The postcompute stage validates the agent output, recomputes bounded overlap data, and emits a single issue-safe portfolio report.
Why is this feature needed?
This workflow adds a recurring governance view to the gh-aw portfolio so the repository can identify which workflows are high-yield, where overlap or consolidation exists, which workflows may be retirement candidates, and where missing instrumentation weakens portfolio decisions.
Implementation details
Notable implementation choices include keeping the scoring math deterministic in the scripts so the agent focuses on interpretation, using
actions/github-scriptin the generated workflow, and importing optional OTel observability context from.github/workflows/shared/otel-observability.mdso telemetry can inform recommendations without requiring raw trace access.Key files:
.github/workflows/aw-portfolio-yield.md— source workflow definition and prompt.github/workflows/aw-portfolio-yield.lock.yml— compiled workflowscripts/aw_yield_precompute.py— deterministic scoring and signal extractionscripts/aw_yield_postcompute.py— report finalization and guardrailsscripts/aw_yield_precompute.cjs/scripts/aw_yield_postcompute.cjs—github-scriptwrapperstests/test_aw_yield_precompute.py/tests/test_aw_yield_postcompute.py— regression coverage