[Schema Consistency] 🔍 Schema Consistency Check - 2025-11-18: Metadata Completeness Audit

[github-actions[bot]]

🔍 Schema Consistency Check - 2025-11-18

Summary

This analysis focused on schema metadata completeness and internal consistency across all three JSON schemas (main_workflow_schema, included_file_schema, mcp_config_schema). Strategy-008 was selected from cache (last used 23 days ago) to audit schema quality dimensions often invisible to code comparison strategies.

Key Results:

• Inconsistencies Found: 9 (3 critical, 4 moderate, 2 informational)
• Categories Analyzed: Schema metadata, descriptions, examples, defaults, patterns, internal references
• Strategy Used: strategy-008 (Schema Completeness & Internal Reference Analysis)
• New Strategy: No (proven strategy from cache)

Overall Assessment: Schema internal consistency is excellent (zero broken references, no dead definitions), but metadata completeness has significant gaps affecting IDE integration and documentation quality.

Full Report Details

Critical Issues

1. Missing Top-Level Schema Metadata ⚠️

Impact: High - Affects schema discoverability, IDE integration, and external tooling

All three schemas lack critical metadata fields:

• $id: null (should be unique identifier/URL)
• title: null (should be descriptive name)
• description: null (should explain schema purpose)

Affected Files:

• pkg/parser/schemas/main_workflow_schema.json
• pkg/parser/schemas/included_file_schema.json
• pkg/parser/schemas/mcp_config_schema.json

Why This Matters:

• IDEs can't properly identify schemas without $id
• Schema registries can't catalog schemas without metadata
• Developers browsing schemas can't understand purpose at a glance
• External validation tools may have difficulty referencing schemas

Recommended Fix:

{
  "$schema": "(redacted)",
  "$id": "(redacted)",
  "title": "GitHub Agentic Workflows - Main Workflow Schema",
  "description": "JSON Schema for agentic workflow frontmatter configuration in .md workflow files. Defines structure, validation rules, and documentation for all frontmatter fields including triggers (on), engine configuration, permissions, tools, and safe-outputs."
}

Similar metadata should be added to:

• included_file_schema.json: Schema for shared/reusable workflow components
• mcp_config_schema.json: Schema for Model Context Protocol server configuration

2. Missing Description: applyTo Field

Impact: Medium - Reduces IDE autocomplete quality

Field applyTo in included_file_schema.json has no description property.

Location: pkg/parser/schemas/included_file_schema.json (property: applyTo)

Current State:

"applyTo": {
  "oneOf": [...]
  // No "description" field
}

Recommended Fix: Add description explaining what applyTo controls (presumably which workflows or contexts the included file applies to).

3. Widespread Missing Examples

Impact: Medium - Reduces schema documentation value

89+ fields lack examples field, including critical top-level fields:

Major fields without examples:

• on - Workflow trigger configuration
• engine - AI engine selection and configuration
• tools - Tool availability configuration
• permissions - GitHub token permissions
• jobs - Job definitions
• All safe-outputs methods: create-issue, add-comment, create-pull-request, etc.

Why This Matters:

• Examples are primary learning tool for schema consumers
• IDEs can use examples for intelligent defaults
• Documentation generation tools rely on examples
• Complex nested structures especially need examples

Recommended Priority:

1. P0: Add examples to top 10 most-used fields (on, engine, permissions, tools)
2. P1: Add examples to all safe-outputs methods
3. P2: Add examples to remaining complex nested structures

Moderate Issues

4. Descriptions Claim Defaults Without Explicit default Field

Impact: Medium - Schema structure doesn't match description claims

10 fields describe default values in prose but lack explicit JSON Schema default field:

Field	Described Default	Has default Field?
name	filename without extension	❌
timeout-minutes	20 minutes	❌
timeout_minutes	20 minutes (deprecated)	❌
concurrency	sequential queue	❌
roles	['admin', 'maintainer', 'write']	❌
strict	false	❌
github-token	GH_AW_GITHUB_TOKEN || GITHUB_TOKEN	❌
safety-prompt	true	❌
timeout	varies by engine	❌
runs-on	ubuntu-slim	❌

Why This Matters:

• JSON Schema validators can't apply defaults automatically
• IDE tools can't populate default values
• Schema consumers must parse descriptions to find defaults
• Inconsistent with fields that DO have explicit defaults (engine: copilot, max-patch-size: 1024)

Recommended Approach:

For constant defaults (strict, safety-prompt, runs-on):

"strict": {
  "type": "boolean",
  "description": "...",
  "default": false
}

For dynamic/computed defaults (name, github-token):

"name": {
  "type": "string",
  "description": "...",
  "$comment": "Default computed at runtime: filename without extension"
}

For engine-specific defaults (timeout):

"timeout": {
  "type": "integer",
  "description": "...",
  "$comment": "Default varies by engine: Claude 60s, Codex 120s"
}

5. Nested oneOf Fields Missing Descriptions

Impact: Low - Affects deeply nested autocomplete

36 fields in nested oneOf schemas lack descriptions, mainly permission enum variants:

• forks.oneOf[].forks.forks
• names.oneOf[].names.names
• id-token.oneOf[].id-token
• etc.

Assessment: These are intentional structural artifacts of the oneOf pattern used to support both string and object types for permissions. Not a true documentation gap.

Recommendation: Low priority - acceptable schema design pattern.

6. Limited Pattern Constraints

Impact: Low - Validation could be stricter

Only 6 pattern constraints total across main schema:

• campaign: ^[a-zA-Z0-9_-]+$ ✅

Fields that might benefit from patterns:

• Engine IDs: claude, copilot, codex, custom
• Domain patterns: *.example.com
• GitHub usernames/slugs: alphanumeric with hyphens
• Campaign names (already has pattern) ✅

Recommendation: Consider adding patterns for string fields with known formats to provide earlier validation feedback.

7. Zero Format Constraints

Impact: Low - Missing semantic validation hints

No format constraints in any schema (no format: uri, format: email, etc.).

Potential candidates:

• URLs in MCP server configurations: format: "uri"
• GitHub API endpoints: format: "uri"
• Email addresses in configuration: format: "email"

Recommendation: Add format hints where applicable to enable semantic validation.

Informational

8. Internal $ref Usage Pattern ✅

Finding: Schemas use $ref: "#/properties/..." pattern to reuse definitions (permissions, concurrency, githubActionsStep).

Assessment: This is valid JSON Schema and creates DRY (Don't Repeat Yourself) definitions. All referenced properties exist and are valid.

Example:

"jobs": {
  "properties": {
    "permissions": {
      "$ref": "#/properties/permissions"
    }
  }
}

This allows the permissions definition to be reused in multiple contexts without duplication.

9. additionalProperties Coverage ✅

Finding: 96 additionalProperties: false constraints in main schema.

Assessment: Excellent strict validation providing comprehensive typo protection. This is a best practice.

Breakdown:

• main_workflow_schema: 96 constraints
• included_file_schema: 21 constraints
• mcp_config_schema: 4 constraints

Positive Findings

✅ Zero broken $ref references - All internal references resolve correctly
✅ All $defs are used - No dead definitions (engine_config, github_token, http_mcp_tool, stdio_mcp_tool all referenced)
✅ Root-level descriptions 100% complete - All top-level properties have descriptions
✅ Excellent additionalProperties discipline - 96 constraints prevent typos
✅ Good pattern coverage for campaign field - Critical field validated
✅ Intentionally minimal explicit defaults - Only 2 explicit defaults (engine: copilot, max-patch-size: 1024)
✅ Valid DRY pattern - $ref to #/properties/ creates reusable definitions without duplication

Schema Statistics

main_workflow_schema.json

• Root properties: 34
• $defs: 4 (engine_config, github_token, http_mcp_tool, stdio_mcp_tool)
• Explicit defaults: 2
• Pattern constraints: 1
• additionalProperties constraints: 96
• Description coverage (root): 100%

included_file_schema.json

• Root properties: 12
• $defs: 2
• Explicit defaults: 1
• Pattern constraints: 2
• additionalProperties constraints: 21
• Description coverage (root): 91% (missing: applyTo)

mcp_config_schema.json

• Root properties: 12
• $defs: 0
• Explicit defaults: 0
• Pattern constraints: 2
• additionalProperties constraints: 4
• Description coverage (root): 100%

Recommendations by Priority

P0 - Critical (Do Immediately)

1. Add top-level metadata to all schemas

   • Add $id, title, description to main_workflow_schema.json
   • Add $id, title, description to included_file_schema.json
   • Add $id, title, description to mcp_config_schema.json
   • Impact: Enables proper schema identification and IDE integration

2. Add description to applyTo field

   • File: included_file_schema.json
   • Impact: Completes description coverage for included schema

P1 - High (Next Sprint)

3. Formalize default values

   • Add explicit default fields for constant defaults (strict, safety-prompt, runs-on)
   • Add $comment fields explaining dynamic defaults (name, github-token, timeout)
   • Impact: Makes defaults machine-readable and consistent

4. Add examples to top-tier fields

   • Priority fields: on, engine, permissions, tools, safe-outputs methods
   • Impact: Major documentation quality improvement

P2 - Medium (Backlog)

5. Expand validation constraints

   • Add pattern constraints for known string formats
   • Add format constraints for URIs, emails, dates
   • Impact: Earlier validation feedback, better error messages

6. Add examples to remaining fields

   • All remaining fields without examples
   • Impact: Complete documentation coverage

Strategy Performance

• Strategy Used: strategy-008 (Schema Completeness & Internal Reference Analysis)
• Last Used: 2025-10-26 (23 days ago)
• Findings: 9 (3 critical, 4 moderate, 2 informational)
• Effectiveness: Very High
• Should Reuse: Yes - every 4-5 analyses

Key Strength: Unique focus on schema metadata quality complements code-comparison strategies. Finds documentation completeness issues invisible to validation logic analysis.

Pairs Well With:

• strategy-007 (Example Validation & Documentation Freshness)
• strategy-012 (Dynamic Validation & Error Pattern Analysis)

Next Steps

• Immediate: Add $id, title, description to all three schemas
• Immediate: Add description to applyTo field in included_file_schema
• This Week: Add explicit default fields or $comment for 10 fields with described defaults
• This Sprint: Add examples to top 10 most-used fields (on, engine, permissions, tools, jobs, safe-outputs)
• Backlog: Consider pattern/format constraints for additional validation strictness
• Backlog: Add examples to remaining 79+ fields

Conclusion

The schemas demonstrate excellent internal consistency (zero broken references, comprehensive additionalProperties discipline), but have significant metadata completeness gaps affecting IDE integration and documentation quality. Priority should be adding top-level schema metadata ($id, title, description) and formalizing default values that are currently only described in prose.

Overall Grade: B+ (excellent structure, needs metadata polish)

AI generated by Schema Consistency Checker

[github-actions[bot]]

This discussion was automatically closed because it was created by an agentic workflow more than 1 week ago.