Skip to content

Commit fa4e4e5

Browse files
committed
feat: add skill frontmatter schema validation
1 parent c4cf277 commit fa4e4e5

6 files changed

Lines changed: 322 additions & 41 deletions

File tree

.github/workflows/lint-skills.yml

Lines changed: 2 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -13,42 +13,5 @@ jobs:
1313
- name: Checkout repository
1414
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
1515

16-
- name: Validate frontmatter in skill and role files
17-
run: |
18-
EXIT_CODE=0
19-
REQUIRED_FIELDS=("name" "description" "version" "author" "license" "injection-hardened" "allowed-tools" "tags" "role" "phase" "frameworks" "difficulty" "time_estimate")
20-
21-
FILES=$(find skills/ roles/ -name 'SKILL.md' 2>/dev/null || true)
22-
23-
if [ -z "$FILES" ]; then
24-
echo "No .md files found in skills/ or roles/."
25-
exit 0
26-
fi
27-
28-
while IFS= read -r file; do
29-
echo "Checking: $file"
30-
31-
FRONTMATTER=$(awk '/^---$/{if(++c==2) exit} c==1' "$file")
32-
33-
if [ -z "$FRONTMATTER" ]; then
34-
echo " ERROR: No YAML frontmatter found (missing --- delimiters)"
35-
EXIT_CODE=1
36-
continue
37-
fi
38-
39-
for field in "${REQUIRED_FIELDS[@]}"; do
40-
if ! echo "$FRONTMATTER" | grep -qE "^${field}:"; then
41-
echo " ERROR: Missing required field: $field"
42-
EXIT_CODE=1
43-
fi
44-
done
45-
done <<< "$FILES"
46-
47-
if [ "$EXIT_CODE" -ne 0 ]; then
48-
echo ""
49-
echo "FAIL: One or more files have missing required frontmatter fields."
50-
exit 1
51-
fi
52-
53-
echo ""
54-
echo "All frontmatter checks passed."
16+
- name: Validate skill schema
17+
run: ruby scripts/validate_skill_schema.rb

CONTRIBUTING.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -144,7 +144,13 @@ argument-hint: "[target-file-or-directory]"
144144
```
145145

146146
Use [SKILL_TEMPLATE.md](SKILL_TEMPLATE.md) as the source of truth for the
147-
required body sections and submission checklist.
147+
required body sections and submission checklist. The machine-readable
148+
frontmatter contract lives in [schemas/skill.schema.json](schemas/skill.schema.json)
149+
and is enforced by CI. Run it locally before opening a PR:
150+
151+
```bash
152+
ruby scripts/validate_skill_schema.rb
153+
```
148154

149155
---
150156

README.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,14 @@ argument-hint: "[target-file-or-directory]"
9292
# context: fork # optional
9393
```
9494

95+
The machine-readable schema for this frontmatter lives at
96+
[`schemas/skill.schema.json`](schemas/skill.schema.json). Validate all skills
97+
and role bundles locally with:
98+
99+
```bash
100+
ruby scripts/validate_skill_schema.rb
101+
```
102+
95103
### Progressive disclosure (keep `SKILL.md` lean)
96104

97105
Claude's skill guidance: when a `SKILL.md` would exceed ~500 lines, **don't inline everything** — split detail into sibling reference files in the same directory and link to them from `SKILL.md`. The agent loads a reference only when it needs it, so the entrypoint stays cheap to load.

schemas/skill.schema.json

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
{
2+
"$schema": "https://json-schema.org/draft/2020-12/schema",
3+
"$id": "https://github.com/UnitOneAI/SecuritySkills/schemas/skill.schema.json",
4+
"title": "SecuritySkills SKILL.md Frontmatter",
5+
"description": "Canonical machine-readable contract for SecuritySkills SKILL.md frontmatter. The Markdown body remains the place for detailed outputs, evidence, remediation, and references, while this schema validates the metadata agents and CI need for discovery.",
6+
"type": "object",
7+
"additionalProperties": false,
8+
"required": [
9+
"name",
10+
"description",
11+
"tags",
12+
"role",
13+
"phase",
14+
"frameworks",
15+
"difficulty",
16+
"time_estimate",
17+
"version",
18+
"author",
19+
"license",
20+
"allowed-tools",
21+
"injection-hardened"
22+
],
23+
"properties": {
24+
"name": {
25+
"type": "string",
26+
"pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$",
27+
"description": "Kebab-case skill identifier. For skills, this must match the skill directory name."
28+
},
29+
"description": {
30+
"type": "string",
31+
"minLength": 40,
32+
"description": "Agent-facing summary of what the skill does and when it should be invoked."
33+
},
34+
"tags": {
35+
"type": "array",
36+
"minItems": 1,
37+
"items": {
38+
"type": "string",
39+
"pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$"
40+
},
41+
"description": "Discovery tags covering domain, activity, technology, or role context."
42+
},
43+
"role": {
44+
"type": "array",
45+
"minItems": 1,
46+
"items": {
47+
"type": "string",
48+
"pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$"
49+
},
50+
"description": "Role bundles that should include or invoke this skill."
51+
},
52+
"phase": {
53+
"type": "array",
54+
"minItems": 1,
55+
"items": {
56+
"type": "string",
57+
"pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$"
58+
},
59+
"description": "Lifecycle phases where the skill applies."
60+
},
61+
"frameworks": {
62+
"type": "array",
63+
"minItems": 1,
64+
"items": {
65+
"type": "string",
66+
"minLength": 2
67+
},
68+
"description": "Frameworks, standards, taxonomies, or benchmark versions cited by the skill. Control IDs in findings must resolve to these references."
69+
},
70+
"difficulty": {
71+
"type": "string",
72+
"enum": ["beginner", "intermediate", "advanced"],
73+
"description": "Expected operator skill level."
74+
},
75+
"time_estimate": {
76+
"type": "string",
77+
"minLength": 3,
78+
"description": "Expected time to run the skill for a typical target."
79+
},
80+
"version": {
81+
"type": "string",
82+
"pattern": "^\\d+\\.\\d+\\.\\d+$",
83+
"description": "Semantic version for the skill contract and content."
84+
},
85+
"author": {
86+
"type": "string",
87+
"minLength": 2,
88+
"description": "GitHub handle, organization, or maintainer identity."
89+
},
90+
"license": {
91+
"type": "string",
92+
"minLength": 2,
93+
"description": "License covering the skill content."
94+
},
95+
"allowed-tools": {
96+
"oneOf": [
97+
{
98+
"type": "string",
99+
"minLength": 2
100+
},
101+
{
102+
"type": "array",
103+
"minItems": 1,
104+
"items": {
105+
"type": "string",
106+
"minLength": 2
107+
}
108+
}
109+
],
110+
"description": "Tool names the skill may use. Existing skills may use a comma-separated string; new tooling may normalize this to an array."
111+
},
112+
"injection-hardened": {
113+
"type": "boolean",
114+
"description": "True only after the skill has been reviewed against prompt injection guidance."
115+
},
116+
"argument-hint": {
117+
"type": "string",
118+
"minLength": 2,
119+
"description": "Optional invocation argument hint shown to users or agents."
120+
},
121+
"context": {
122+
"type": "string",
123+
"minLength": 2,
124+
"description": "Optional execution context hint."
125+
},
126+
"disable-model-invocation": {
127+
"type": "boolean",
128+
"description": "Optional role-bundle guard for workflows that should not directly invoke a model."
129+
}
130+
},
131+
"x-securityskills": {
132+
"bodySections": {
133+
"outputs": "Document expected findings, evidence, remediation, or deliverables in the Markdown body.",
134+
"references": "Document authoritative framework and control references in the Markdown body or sibling reference files."
135+
},
136+
"referenceFiles": "Long framework tables, tool rules, benchmark checklists, and language-specific guidance should live in sibling Markdown files linked from SKILL.md."
137+
}
138+
}

scripts/validate_skill_schema.rb

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
#!/usr/bin/env ruby
2+
# frozen_string_literal: true
3+
4+
require "json"
5+
require "yaml"
6+
7+
ROOT = File.expand_path("..", __dir__)
8+
SCHEMA_PATH = File.join(ROOT, "schemas", "skill.schema.json")
9+
DEFAULT_GLOBS = [
10+
File.join(ROOT, "skills", "*", "*", "SKILL.md"),
11+
File.join(ROOT, "roles", "*", "SKILL.md")
12+
].freeze
13+
14+
def usage
15+
warn "Usage: ruby scripts/validate_skill_schema.rb [SKILL.md ...]"
16+
end
17+
18+
def load_schema
19+
JSON.parse(File.read(SCHEMA_PATH))
20+
rescue Errno::ENOENT
21+
abort "Schema not found: #{SCHEMA_PATH}"
22+
rescue JSON::ParserError => e
23+
abort "Invalid JSON schema #{SCHEMA_PATH}: #{e.message}"
24+
end
25+
26+
def skill_files(args)
27+
files = args.empty? ? DEFAULT_GLOBS.flat_map { |pattern| Dir.glob(pattern) } : args
28+
files.map { |path| File.expand_path(path, Dir.pwd) }.sort
29+
end
30+
31+
def frontmatter_for(path)
32+
text = File.read(path)
33+
match = text.match(/\A---\s*\n(.*?)\n---\s*(?:\n|\z)/m)
34+
raise "missing YAML frontmatter delimited by ---" unless match
35+
36+
YAML.safe_load(match[1], permitted_classes: [], aliases: false) || {}
37+
rescue Psych::SyntaxError => e
38+
raise "invalid YAML frontmatter: #{e.message}"
39+
end
40+
41+
def type_name(value)
42+
case value
43+
when String then "string"
44+
when Array then "array"
45+
when Hash then "object"
46+
when TrueClass, FalseClass then "boolean"
47+
when Integer then "integer"
48+
when Float then "number"
49+
when NilClass then "null"
50+
else value.class.name
51+
end
52+
end
53+
54+
def validate_type(value, expected)
55+
Array(expected).include?(type_name(value))
56+
end
57+
58+
def validate_string(path, value, schema, errors)
59+
return unless value.is_a?(String)
60+
61+
min = schema["minLength"]
62+
errors << "#{path} must be at least #{min} characters" if min && value.length < min
63+
64+
pattern = schema["pattern"]
65+
return unless pattern
66+
67+
errors << "#{path} must match /#{pattern}/" unless Regexp.new(pattern).match?(value)
68+
end
69+
70+
def validate_array(path, value, schema, errors)
71+
return unless value.is_a?(Array)
72+
73+
min = schema["minItems"]
74+
errors << "#{path} must contain at least #{min} item(s)" if min && value.length < min
75+
76+
item_schema = schema["items"]
77+
return unless item_schema
78+
79+
value.each_with_index do |item, index|
80+
validate_value("#{path}[#{index}]", item, item_schema, errors)
81+
end
82+
end
83+
84+
def validate_value(path, value, schema, errors)
85+
if schema["oneOf"]
86+
nested = schema["oneOf"].map do |candidate|
87+
candidate_errors = []
88+
validate_value(path, value, candidate, candidate_errors)
89+
candidate_errors
90+
end
91+
errors << "#{path} must match one allowed schema" if nested.none?(&:empty?)
92+
return
93+
end
94+
95+
expected_type = schema["type"]
96+
if expected_type && !validate_type(value, expected_type)
97+
errors << "#{path} must be #{Array(expected_type).join(' or ')}, got #{type_name(value)}"
98+
return
99+
end
100+
101+
enum = schema["enum"]
102+
errors << "#{path} must be one of #{enum.join(', ')}" if enum && !enum.include?(value)
103+
104+
validate_string(path, value, schema, errors)
105+
validate_array(path, value, schema, errors)
106+
end
107+
108+
def validate_document(frontmatter, schema)
109+
errors = []
110+
111+
schema.fetch("required", []).each do |field|
112+
errors << "missing required field: #{field}" unless frontmatter.key?(field)
113+
end
114+
115+
properties = schema.fetch("properties", {})
116+
unless schema.fetch("additionalProperties", true)
117+
frontmatter.each_key do |key|
118+
errors << "unknown field: #{key}" unless properties.key?(key)
119+
end
120+
end
121+
122+
frontmatter.each do |key, value|
123+
next unless properties.key?(key)
124+
125+
validate_value(key, value, properties[key], errors)
126+
end
127+
128+
errors
129+
end
130+
131+
def validate_name_matches_path(path, frontmatter)
132+
return [] unless path.include?("#{File::SEPARATOR}skills#{File::SEPARATOR}")
133+
134+
expected = File.basename(File.dirname(path))
135+
actual = frontmatter["name"]
136+
actual == expected ? [] : ["name must match skill directory '#{expected}', got '#{actual}'"]
137+
end
138+
139+
schema = load_schema
140+
files = skill_files(ARGV)
141+
142+
if files.empty?
143+
usage
144+
abort "No SKILL.md files found."
145+
end
146+
147+
failed = false
148+
files.each do |path|
149+
relative = path.delete_prefix("#{ROOT}#{File::SEPARATOR}")
150+
begin
151+
frontmatter = frontmatter_for(path)
152+
errors = validate_document(frontmatter, schema) + validate_name_matches_path(path, frontmatter)
153+
rescue StandardError => e
154+
errors = [e.message]
155+
end
156+
157+
if errors.empty?
158+
puts "OK: #{relative}"
159+
else
160+
failed = true
161+
puts "FAIL: #{relative}"
162+
errors.each { |error| puts " - #{error}" }
163+
end
164+
end
165+
166+
exit(failed ? 1 : 0)

skills/compliance/iso27001-gap/SKILL.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ description: >
1010
tags: [compliance, iso27001, isms]
1111
role: [vciso, security-engineer]
1212
phase: [assess, operate]
13-
frameworks: [ISO/IEC-27001:2022, ISO/IEC-27002:2022]
13+
frameworks: ["ISO/IEC-27001:2022", "ISO/IEC-27002:2022"]
1414
difficulty: intermediate
1515
time_estimate: "90-180min"
1616
version: "1.0.0"

0 commit comments

Comments
 (0)