🏥 Safe Output Health Report - November 17, 2025

[github-actions[bot]]

🏥 Safe Output Health Report - November 17, 2025

Executive Summary

Daily audit of agentic workflow safe output jobs over the last 24 hours reveals stable performance with 2 permission-related failures out of 48 total safe output job executions. Both failures are recurring issues related to personal access token permissions that have been tracked since November 11-14.

• Period: Last 24 hours (November 16-17, 2025)
• Runs Analyzed: 43 workflow runs
• Safe Output Jobs Executed: 48
• Safe Output Jobs Failed: 2 (4.2% failure rate)
• Safe Output Jobs Successful: 22 (45.8% success rate)
• Safe Output Jobs Skipped: 24 (50% - expected behavior for missing_tool jobs)
• Error Clusters Identified: 1 (permissions-related)

Key Finding: The primary issue continues to be personal access token (PAT) permission limitations for PR reviewer requests and issue assignments. Both failures are non-critical as the main operations (creating PRs and issues) succeeded.

Full Report Details

Safe Output Job Statistics

Job Type	Total Executions	Failures	Skipped	Success	Success Rate
create_issue	23	1	11	11	47.8%
create_discussion	7	0	1	6	85.7%
add_comment	1	0	0	1	100.0%
create_pull_request	10	1	6	3	30.0%
push_to_pull_request_branch	7	0	6	1	14.3%
missing_tool	41	0	41	0	N/A*

* Note: missing_tool jobs are designed to be skipped when no missing tools are reported

Job Performance Analysis

High Performers:

• ✅ add_comment: 100% success rate (1/1)
• ✅ create_discussion: 85.7% success rate (6/7)

Concerning Areas:

• ⚠️ create_pull_request: 30% success rate (3/10) - 1 failure, 6 skipped
• ⚠️ push_to_pull_request_branch: 14.3% success rate (1/7) - 6 skipped
• ⚠️ create_issue: 47.8% success rate (11/23) - 1 failure, 11 skipped

Note: High skip rates are often normal behavior when agents don't request those output types. The actual failure rate (excluding skipped) is much better.

Error Analysis

Error Cluster 1: Personal Access Token Permission Errors

Cluster ID: permissions_personal_access_token
Count: 2 occurrences
Severity: High
Status: Recurring (first seen November 11-14, still occurring)

Failure 1: PR Reviewer Request

• Run: §19401328339
• Workflow: Daily Documentation Updater
• Job: create_pull_request
• Engine: Claude
• Error: Resource not accessible by personal access token (HTTP 403)
• Failed Step: Request reviewers for the PR
• Impact: PR was created successfully, but reviewer assignment failed causing job to fail

Root Cause: The GH_TOKEN (personal access token) lacks the necessary permissions to request reviewers on pull requests. This operation requires either:

• GITHUB_TOKEN with elevated permissions
• GitHub App installation token
• PAT with additional scopes

Failure 2: Issue Assignment

• Run: §19411965851
• Workflow: Duplicate Code Detector
• Job: create_issue
• Engine: Codex
• Error: Resource not accessible by personal access token (replaceActorsForAssignable)
• Failed Step: Assign issue to @copilot
• Impact: Issue [duplicate-code] 🔍 Duplicate Code Detected: Domain Accessor Methods #4149 was created successfully, but assignment to @copilot failed

Root Cause: The GH_TOKEN (personal access token) lacks permissions to assign issues. The GraphQL mutation replaceActorsForAssignable is not accessible with the current token.

Pattern: Recurring Permission Issues

Both failures match known error patterns tracked in our database:

1. pr-reviewer-request-permission (pattern_id)

   • First seen: 2025-11-14
   • Last seen: 2025-11-16
   • Frequency: High
   • Resolution status: Unresolved

2. issue-assignment-permission (pattern_id)

   • First seen: 2025-11-11
   • Last seen: 2025-11-16
   • Frequency: Medium
   • Resolution status: Unresolved

Root Cause Analysis

API-Related Issues

Permission Errors (2 occurrences)

The core issue is that certain GitHub operations require elevated permissions that personal access tokens may not have:

1. PR Reviewer Requests (/repos/{owner}/{repo}/pulls/{pull_number}/requested_reviewers)

   • Requires: repo scope (full control) or organization membership
   • Current: Limited PAT permissions

2. Issue Assignment (GraphQL replaceActorsForAssignable)

   • Requires: Ability to modify issue assignees
   • Current: PAT may lack this specific permission

Impact Assessment

Severity: Medium to High

• The primary operations (creating PRs and issues) succeed
• Only secondary operations (reviewer assignment, issue assignment) fail
• However, job failures cause workflow status to show as "failed"
• This creates noise in workflow monitoring and may hide other issues

User Impact: Low to Medium

• PRs and issues are created successfully
• Manual assignment of reviewers/assignees is still possible
• Workflow failure status may cause confusion

Recommendations

Critical Issues (Immediate Action Required)

1. Fix PR Reviewer Request Permission Error

Priority: High
Root Cause: GH_TOKEN lacks permissions to request PR reviewers
Recommended Action: Make reviewer requests optional or use proper token

Implementation Options:

Option A: Make Reviewer Requests Optional (Quick Fix)

# In create_pull_request safe output job
- name: Request reviewers (optional)
  continue-on-error: true  # Don't fail job if this fails
  run: |
    gh pr edit $PR_NUMBER --add-reviewer $REVIEWERS || echo "⚠️ Could not add reviewers (insufficient permissions)"

Option B: Use GITHUB_TOKEN with Elevated Permissions

permissions:
  pull-requests: write
  contents: write

Option C: Use GitHub App Token

• Configure GitHub App with proper permissions
• Use installation token instead of PAT

Affected Workflows: Daily Documentation Updater (and any workflow that creates PRs)

2. Fix Issue Assignment Permission Error

Priority: Medium
Root Cause: GH_TOKEN lacks permissions to assign issues
Recommended Action: Make assignment optional or document token requirements

Implementation Options:

Option A: Make Assignment Optional (Quick Fix)

- name: Assign issue (optional)
  continue-on-error: true
  run: |
    gh issue edit $ISSUE_NUMBER --add-assignee $ASSIGNEE || echo "⚠️ Could not assign issue (insufficient permissions)"

Option B: Document Token Requirements
Add to workflow documentation:

## Token Permissions Required

To enable automatic issue assignment, the GH_TOKEN must have:
- `write:issues` permission
- Organization membership (for assigning to organization members)

Affected Workflows: Duplicate Code Detector (and any workflow that assigns issues)

Process Improvements

1. Graceful Degradation for Permission-Limited Operations

Current State: Job fails completely when optional operations fail
Proposed: Implement graceful degradation for non-critical operations
Benefits:

• Workflow shows success when primary operation succeeds
• Clear warnings for failed optional operations
• Reduces noise in workflow monitoring

Implementation:

- name: Create Pull Request
  id: create_pr
  run: |
    # Primary operation (must succeed)
    gh pr create --title "$TITLE" --body "$BODY"
    echo "pr_number=$PR_NUMBER" >> $GITHUB_OUTPUT

- name: Add Reviewers (Optional)
  continue-on-error: true
  run: |
    gh pr edit ${{ steps.create_pr.outputs.pr_number }} --add-reviewer $REVIEWERS || {
      echo "::warning::Could not add reviewers - insufficient token permissions"
      echo "Manual reviewer assignment required"
    }

2. Token Permission Validation

Proposed: Add pre-flight checks to validate token permissions
Benefits: Early detection of permission issues with clear error messages

Implementation:

- name: Validate Token Permissions
  run: |
    # Check if token can request reviewers
    if ! gh api /repos/${{ github.repository }}/collaborators --jq '.[] | .login' &>/dev/null; then
      echo "::warning::GH_TOKEN may lack permissions for advanced operations"
      echo "Some features (reviewer requests, assignments) may not work"
    fi

3. Enhanced Error Reporting

Proposed: Improve error messages to clearly distinguish between:

• Primary operation failures (critical)
• Secondary operation failures (non-critical)

Example:

✅ Pull Request #123 created successfully
⚠️ Could not add reviewers (token permission limitation)
ℹ️ Reviewers can be added manually at: https://github.com/...

Work Item Plans

Work Item 1: Make PR Reviewer Requests Optional

• Type: Bug Fix / Enhancement
• Priority: High
• Affected Jobs: create_pull_request
• Description: Prevent workflow failures when PR reviewer requests fail due to permission errors

Acceptance Criteria:

• create_pull_request job succeeds even when reviewer request fails
• Clear warning message logged when reviewer request fails
• Job summary includes notice about manual reviewer assignment
• Existing functionality preserved for tokens with sufficient permissions

Technical Approach:

1. Add continue-on-error: true to reviewer request step
2. Add error handling to log clear warning message
3. Update job summary to reflect partial success
4. Test with both limited and full-permission tokens

Estimated Effort: Small (1-2 hours)
Dependencies: None
Files to Modify:

• .github/workflows/daily-doc-updater.md (or compiled .lock.yml)
• Safe output templates in gh-aw

Work Item 2: Make Issue Assignment Optional

• Type: Bug Fix / Enhancement
• Priority: Medium
• Affected Jobs: create_issue
• Description: Prevent workflow failures when issue assignment fails due to permission errors

Acceptance Criteria:

• create_issue job succeeds even when assignment fails
• Clear warning message logged when assignment fails
• Issue created successfully regardless of assignment outcome
• Manual assignment link provided in logs

Technical Approach:

1. Add error handling around issue assignment logic
2. Use continue-on-error or try-catch in JavaScript action
3. Log clear warning with instructions for manual assignment
4. Test with various token permission levels

Estimated Effort: Small (1-2 hours)
Dependencies: None
Files to Modify: Safe output job templates for create_issue

Work Item 3: Document Token Permission Requirements

• Type: Documentation
• Priority: Medium
• Description: Create comprehensive documentation for token permissions required by safe output jobs

Acceptance Criteria:

• Document lists all safe output operations
• Clear indication of required vs optional permissions
• Instructions for configuring GitHub App vs PAT
• Troubleshooting guide for permission errors
• Examples of workflow permission configuration

Technical Approach:

1. Audit all safe output job types for permission requirements
2. Create permission matrix (operation × token type × permissions)
3. Document setup instructions for different scenarios
4. Add troubleshooting section for common errors

Estimated Effort: Medium (4-6 hours)
Dependencies: None
Location: docs/reference/safe-outputs/permissions.md

Work Item 4: Implement Permission Pre-flight Checks

• Type: Enhancement
• Priority: Low
• Description: Add validation steps to check token permissions before attempting operations

Acceptance Criteria:

• Pre-flight check validates token capabilities
• Clear warnings for limited permissions
• No false positives (doesn't block valid operations)
• Minimal performance impact
• Works with all token types (PAT, App, GITHUB_TOKEN)

Technical Approach:

1. Create reusable action for permission checking
2. Add checks for reviewer request capability
3. Add checks for issue assignment capability
4. Output warnings rather than errors for non-critical limitations

Estimated Effort: Medium (6-8 hours)
Dependencies: None
Benefits: Proactive detection of permission issues

Historical Context

Comparing with previous safe output health audits:

Trend Analysis

Date	Runs	Jobs	Failed	Success Rate	Status
2025-11-17	43	48	2	45.8%	⚠️ Stable with known issues
2025-11-16	33	9	3	66.7%	⚠️ High failure rate
2025-11-14	75	164	3	89.7%	✅ Good
2025-11-13	93	48	2	95.8%	✅ Excellent
2025-11-12	88	52	8	84.6%	⚠️ Moderate issues

Key Observations

1. Permission errors are recurring: First appeared Nov 11-14, still occurring
2. Pattern is consistent: Same two error types (reviewer request, issue assignment)
3. Impact is limited: Primary operations succeed, only secondary features fail
4. No new error types: All failures match known patterns
5. Success rate variability: Large variance due to different workflow mixes and skip behavior

Most Common Recurring Issue

Personal Access Token Permission Limitations

• Pattern: Resource not accessible by personal access token
• Frequency: Multiple occurrences across multiple days
• Operations affected: PR reviewer requests, issue assignments
• Status: Unresolved, requires configuration change or graceful degradation

Improvement Since Last Audit

• Positive: No new error patterns introduced
• Neutral: Same permission issues continue (expected without fixes)
• Action Item: Implement recommended fixes to reduce failure rate

Metrics and KPIs

• Overall Safe Output Success Rate: 45.8% (22 succeeded out of 48 total)
  • Note: 50% of jobs were skipped (expected behavior), so effective success rate is ~92% (22/24 non-skipped)
• Most Reliable Job Type: add_comment (100% success rate, 1/1)
• Most Problematic Job Type: create_pull_request (30% success rate due to permission errors and high skip rate)
• Average Failure Impact: Low - primary operations succeed in all cases
• Error Recurrence Rate: 100% - both failures are known recurring issues

Success Rate by Engine

Based on failed runs:

• Claude: 1 failure (create_pull_request in Daily Documentation Updater)
• Codex: 1 failure (create_issue in Duplicate Code Detector)
• Copilot: 0 failures

Most Active Workflows (Safe Output Jobs)

Workflows that executed safe output jobs in the last 24 hours:

• CLI Version Checker (create_issue)
• Daily Documentation Updater (create_pull_request)
• Duplicate Code Detector (create_issue)
• Various smoke test workflows (create_discussion)

Next Steps

Immediate actions recommended:

• High Priority: Implement graceful degradation for PR reviewer requests (Work Item 1)
• High Priority: Implement graceful degradation for issue assignments (Work Item 2)
• Medium Priority: Document token permission requirements (Work Item 3)
• Medium Priority: Review and update workflow configurations to use appropriate tokens
• Low Priority: Implement permission pre-flight checks (Work Item 4)
• Ongoing: Monitor safe output health daily for new patterns

Conclusion

Safe output job health is stable with 2 known, recurring permission-related failures. These failures are non-critical as primary operations (creating PRs and issues) succeed in all cases.

The recommended approach is to implement graceful degradation for secondary operations (reviewer requests, assignments) to prevent workflow failures when using tokens with limited permissions. This will reduce noise in workflow monitoring and provide clearer status reporting.

No new error patterns or critical issues were identified in this audit period.

---

References:

• §19401328339
• §19411965851

AI generated by Safe Output Health Monitor

[github-actions[bot]]

This discussion was automatically closed because it was created by an agentic workflow more than 1 week ago.