💡 Token Cost Observatory with Pre-Agentic Optimization

[github-actions[bot]]

Summary

Implement GitHub's Effective Tokens (ET) metric across all agentic workflows, add structured token-usage logging, and introduce pre-agentic context download steps that fetch issue/PR data via gh CLI before agent invocation. This reduces token consumption by 20-60% while providing cost visibility and budget alerting.

Market Signal

GitHub published its token efficiency framework with the Effective Tokens (ET) metric: ET = m × (1.0×I + 0.1×C + 4.0×O), showing 19-62% reductions across five internal workflows. TrueFoundry documented the "agentic token explosion" where costs grow O(n²) in the number of agent steps. Starting June 1, 2026, Copilot code review will consume GitHub Actions minutes under the new AI Credits billing model, adding a significant new cost vector. GitHub's approach of eliminating unused MCP tool registrations saved 8-12KB per turn (several thousand tokens per run).

User Signal

Issue #206 explicitly requests token usage monitoring and optimization. The org runs 8+ agentic workflows (claude-code, agent-shield, feature-ideation, compliance-audit, org-status, dependabot-automerge, pr-review-mention, dependency-audit). No token tracking infrastructure currently exists — costs are invisible. With Copilot's June 2026 billing change imminent, this gap is becoming a budget risk.

Technical Opportunity

Three implementation layers, each independently valuable:

1. Token logging — Claude Code's API proxy can emit per-call metrics (input tokens, output tokens, cache-read/write, model, timestamps) as a JSONL workflow artifact. No new infrastructure required.
2. ET metric calculation — A lightweight shell script applies the formula to the JSONL artifact, producing a weekly cost report that surfaces the top-3 optimization opportunities.
3. Pre-agentic context download — Add pre-fetch steps to claude-code-reusable.yml that download issue/PR context via gh CLI before invoking Claude. This eliminates MCP tool overhead and keeps credentials out of the LLM context window (a security co-benefit).

The weekly cost report can be appended to the existing daily-org-status report with minimal integration work.

Assessment

Dimension	Score	Rationale
Feasibility	high	Logging wrapper + shell script; no new infrastructure; each layer is independently deployable
Impact	high	20-60% cost reduction potential; directly addresses Issue #206; security co-benefit from credential isolation
Urgency	med	Issue #206 is an explicit request; Copilot billing change on June 1 creates a deadline

Adversarial Review

Strongest objection: The org is small with limited workflow runs. The cost isn't high enough to justify monitoring infrastructure. YAGNI applies.

Rebuttal: Issue #206 is an explicit user request, not speculative demand. The org runs 8+ agentic workflows, several daily. GitHub's own team achieved 62% reduction on their auto-triage workflow — similar in scope to this org's compliance-audit. The implementation is deliberately lightweight (logging wrapper + aggregation script, not a full observability platform). The pre-agentic download pattern provides a security benefit beyond cost savings by keeping GitHub tokens out of the LLM context.

Suggested Next Step

Add token-usage logging to claude-code-reusable.yml by capturing the Claude API response token counts and writing them to a JSONL artifact. Implement the ET metric formula in a small shell script (scripts/lib/token-metrics.sh). Create a weekly token report that aggregates costs by workflow and surfaces the top-3 optimization opportunities.

[github-actions[bot]]

Weekly Update — 2026-05-15

What Changed

Copilot code review will start consuming GitHub Actions minutes on June 1, 2026

GitHub announced (changelog) that Copilot code review — currently a "free" add-on — will begin consuming GitHub Actions minutes starting June 1, 2026. This is a significant cost signal for organizations like petry-projects that run Copilot review alongside Claude Code and CodeRabbit on every PR.

Impact on this org:

• Every PR currently triggers 3 AI review systems: Claude Code, CodeRabbit, and Copilot
• After June 1, Copilot reviews will consume Actions minutes in addition to the existing Claude Code and CodeRabbit costs
• Combined with the claude-code-action minutes usage and agent-shield runs, total Actions minute consumption per PR will increase materially

Code with Claude SF 2026 pricing changes:

• Anthropic doubled five-hour rate limits across Pro, Max, Team, and Enterprise plans
• Removed peak-hours throttling for Pro and Max accounts
• Increased Opus API rate limits "considerably" with per-tier specifications
• These changes may affect cost-per-review calculations for Claude Code in CI

GitHub Actions pricing context:

• GitHub reduced pricing for hosted runners effective January 1, 2026 (changelog)
• The combination of reduced runner pricing + new Copilot minute consumption creates a complex cost model that warrants monitoring

Updated Assessment

Dimension	Previous	Current	Delta
Feasibility	med	med	→
Impact	med	high	↑ — Copilot minutes add concrete, measurable cost pressure with a June 1 deadline
Urgency	med	high	↑ — June 1 deadline means cost increase hits in 17 days without visibility tooling

Recommendation

Advance — prioritize before June 1. At minimum, establish baseline metrics for current Actions minute consumption per PR (across all AI review tools) so the Copilot minutes impact can be measured when it hits. A lightweight first step: add a step to daily-org-status.yml that reports total Actions minutes consumed per repo in the past 24 hours.

[don-petry]

🔀 Discussion Moved

This discussion has been moved to petry-projects/.github-private#332 since the implementation lives in the private infra repo.

Please continue the conversation there.

[github-actions[bot]]

Weekly Update

What Changed

Copilot code review billing change (June 1, 2026): GitHub announced that Copilot code review runs will consume GitHub Actions minutes starting June 1, 2026. This fundamentally changes the cost model — the org now needs to track Copilot review minutes alongside Anthropic API costs.

Copilot usage metrics API (May 14, 2026): Team-level Copilot usage metrics are now available via API, providing the programmatic data source needed for automated cost tracking across all repos.

New Discussion #338 proposes expanding the Token Cost Observatory concept into a unified dashboard covering all three AI systems (Claude Code, Copilot, CodeRabbit). See: #338

Updated Assessment

Dimension	Previous	Current	Delta
Feasibility	high	high	→
Impact	high	high	→
Urgency	med	high	↑ (June 1 billing change is imminent)

Recommendation

Advance — the Copilot billing change makes cost observability urgent. Consider merging this idea with Discussion #338 to create a unified approach covering all agent costs.