-
Notifications
You must be signed in to change notification settings - Fork 5
feat(cli): route hol-guard binary to combined mode for scanner+guard commands #700
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| --- | ||
| name: hol-guard-runner | ||
| description: Runs HOL Guard scanner and guard operations headlessly. Use for CI/CD scanning, guard checks, and automated security analysis. | ||
| model: inherit | ||
| tools: ["Execute"] | ||
| --- | ||
|
|
||
| # HOL Guard Runner | ||
|
|
||
| You are a headless runner for HOL Guard (AI Antivirus). Your job is to execute hol-guard CLI commands safely and report results. | ||
|
|
||
| ## Execution Rules | ||
|
|
||
| 1. All commands MUST run from the hol-guard project root. | ||
| 2. Use `uv run hol-guard` for all invocations. Never invoke Python modules directly. | ||
| 3. Before running, verify environment: `uv sync --frozen --extra dev` | ||
| 4. For scanner operations, use absolute paths to target directories. | ||
| 5. For guard operations, use `--dry-run` and `--default-action allow` flags for safe testing. | ||
| 6. Prefer `--json` output for machine readability. | ||
|
|
||
| ## Test Fixture Locations | ||
|
|
||
| - `tests/fixtures/good-plugin/` - clean Codex plugin | ||
| - `tests/fixtures/bad-plugin/` - plugin with security issues | ||
| - `tests/fixtures/malicious-skill-plugin/` - malicious skill patterns | ||
| - `tests/fixtures/claude-plugin-good/` - clean Claude plugin | ||
| - `tests/fixtures/multi-ecosystem-repo/` - multi-ecosystem repo | ||
|
|
||
| ## Output Format | ||
|
|
||
| Report results as: | ||
| ``` | ||
| PASS: <test-name> | ||
| FAIL: <test-name> - <reason> | ||
| EXIT_CODE: <code> | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| --- | ||
| name: hol-guard | ||
| description: Run HOL Guard scanner and guard operations via `uv run hol-guard`. Use when the user asks to scan plugins/MCP/skills for security, quality, or ecosystem compliance, or when they ask to run guard detect/install/protect workflows for local AI harnesses. | ||
| --- | ||
|
|
||
| # HOL Guard | ||
|
|
||
| HOL Guard is an AI Antivirus scanner that checks plugins, MCP servers, skills, and local AI harnesses for security, quality, and ecosystem compliance. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - Always run from the `hol-guard` project root. | ||
| - Use `uv run hol-guard` to invoke the CLI. Never invoke Python modules directly. | ||
| - Ensure `uv sync --frozen --extra dev` has been run before invoking. | ||
|
|
||
| ## Scanner Operations | ||
|
|
||
| Scan a plugin or skill directory: | ||
|
|
||
| ``` | ||
| uv run hol-guard scan <directory> [--format json|text|markdown|sarif] [--profile default|public-marketplace|strict-security] [--fail-on-severity critical|high|medium|low|info|none] | ||
| ``` | ||
|
|
||
| Lint rules: | ||
|
|
||
| ``` | ||
| uv run hol-guard lint <directory> [--list-rules] [--explain <rule-id>] | ||
| ``` | ||
|
|
||
| Verify runtime: | ||
|
|
||
| ``` | ||
| uv run hol-guard verify <directory> [--online] | ||
| ``` | ||
|
|
||
| List ecosystems: | ||
|
|
||
| ``` | ||
| uv run hol-guard --list-ecosystems | ||
| ``` | ||
|
|
||
| ## Guard Operations | ||
|
|
||
| Detect harnesses: | ||
|
|
||
| ``` | ||
| uv run hol-guard detect [codex|claude|cursor|gemini|opencode] [--json] | ||
| ``` | ||
|
|
||
| Run guard in dry-run mode: | ||
|
|
||
| ``` | ||
| uv run hol-guard run <harness> --dry-run --default-action allow --json | ||
| ``` | ||
|
|
||
| Check guard status: | ||
|
|
||
| ``` | ||
| uv run hol-guard status [--json] | ||
| ``` | ||
|
|
||
| ## Common Test Fixtures | ||
|
|
||
| Test fixtures live in `tests/fixtures/`: | ||
| - `good-plugin/` - clean Codex plugin with all required fields | ||
| - `bad-plugin/` - plugin with secrets, missing fields, bad practices | ||
| - `malicious-skill-plugin/` - skill with malicious patterns | ||
| - `multi-ecosystem-repo/` - repo with Codex, Claude, and Gemini configs | ||
| - `claude-plugin-good/` - clean Claude plugin | ||
| - `opencode-good/` - clean OpenCode plugin | ||
| - `gemini-extension-good/` - clean Gemini extension | ||
|
|
||
| ## Verification | ||
|
|
||
| After each operation, verify: | ||
| - Exit code 0 for clean targets | ||
| - Exit code non-zero for targets with findings | ||
| - Output is valid JSON when `--format json` or `--json` is used | ||
| - Scanner reports findings with correct rule IDs and severities |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,138 @@ | ||
| #!/usr/bin/env python3 | ||
| """Headless droid exec e2e test script for hol-guard.""" | ||
| from __future__ import annotations | ||
|
|
||
| import json | ||
| import os | ||
| import shutil | ||
| import subprocess | ||
| import sys | ||
| from pathlib import Path | ||
|
|
||
| PROJECT_ROOT = Path(__file__).resolve().parent.parent | ||
|
|
||
| def _scan_cmd(fixture_path: Path, fmt: str) -> list[str]: | ||
| """Build scanner invocation, bypassing guard preflight if active.""" | ||
| if os.environ.get("GUARD_PREFLIGHT_DISABLE") == "1" or shutil.which("uv") is None: | ||
| return ["python3", "-m", "codex_plugin_scanner.cli", "scan", str(fixture_path), "--format", fmt] | ||
| return ["uv", "run", "hol-guard", "scan", str(fixture_path), "--format", fmt] | ||
|
|
||
| def run(cmd: list[str], cwd: Path | None = None) -> tuple[int, str, str]: | ||
| env = dict(os.environ) | ||
| env["GUARD_PRE_SCAN_DISABLE"] = "1" | ||
| env["GUARD_PREFLIGHT_DISABLE"] = "1" | ||
| result = subprocess.run(cmd, capture_output=True, text=True, timeout=120, env=env, cwd=cwd) | ||
| return result.returncode, result.stdout, result.stderr | ||
|
kantorcodes marked this conversation as resolved.
|
||
|
|
||
| def test_scanner(): | ||
| fixtures_dir = Path(__file__).resolve().parent.parent / "tests" / "fixtures" | ||
| test_cases = [ | ||
| (fixtures_dir / "good-plugin", 0, "json", False), | ||
| (fixtures_dir / "good-plugin", 0, "text", False), | ||
| (fixtures_dir / "good-plugin", 0, "sarif", False), | ||
| (fixtures_dir / "bad-plugin", 0, "json", True), # expect low score | ||
| (fixtures_dir / "malicious-skill-plugin", 0, "json", False), # can score high w/few findings | ||
| (fixtures_dir / "multi-plugin-repo" / "plugins" / "alpha-plugin", 0, "json", False), | ||
|
greptile-apps[bot] marked this conversation as resolved.
|
||
| ] | ||
| failures: list[str] = [] | ||
|
|
||
| for fixture_path, expected_code, fmt, expect_issues in test_cases: | ||
| cmd = _scan_cmd(fixture_path, fmt) | ||
| code, stdout, stderr = run(cmd, cwd=PROJECT_ROOT) | ||
|
|
||
| payload = None | ||
| if fmt == "json": | ||
| try: | ||
| payload = json.loads(stdout) | ||
| except json.JSONDecodeError: | ||
| failures.append(f"{fixture_path.name}: invalid JSON") | ||
|
|
||
| if payload is not None: | ||
| score = payload.get("score", 0) | ||
| grade = payload.get("grade", "") | ||
| num_findings = len(payload.get("findings", [])) | ||
| else: | ||
| score, grade, num_findings = 0, "", 0 | ||
| if fmt == "json": | ||
| failures.append(f"{fixture_path.name}: no payload") | ||
|
|
||
| # Determine if test passed | ||
| passed = code == expected_code | ||
| if fmt == "json" and num_findings == 0 and expect_issues: | ||
| passed = False | ||
| if fmt == "json" and not expect_issues and payload is not None and score < 60: | ||
| passed = False # Good plugin should score well | ||
| if fmt == "json" and expect_issues and payload is not None and score > 60: | ||
| passed = False # Bad plugin should score poorly | ||
|
kantorcodes marked this conversation as resolved.
|
||
|
|
||
| if not passed: | ||
| failures.append( | ||
| f"{fixture_path.name} fmt={fmt} code={code} expected={expected_code} score={score}" | ||
| ) | ||
|
|
||
| status = "PASS" if passed else "FAIL" | ||
| label = fixture_path.name.split("/")[-1] | ||
| print(f" [{status}] scan {label} --format {fmt} (exit={code}, expected={expected_code})") | ||
| if fmt == "json" and payload is not None: | ||
| print(f" score={score}, grade={grade}, findings={num_findings}") | ||
| if not passed: | ||
| print(f" DEBUG stdout: {stdout[:300]}") | ||
| print(f" DEBUG stderr: {stderr[:300]}") | ||
|
|
||
| return failures | ||
|
|
||
| def test_guard(): | ||
| guard_cases = [ | ||
| (["uv", "run", "hol-guard", "detect", "opencode", "--json"], None), | ||
| (["uv", "run", "hol-guard", "status", "--json"], None), | ||
| (["uv", "run", "hol-guard", "--version"], None), | ||
| ] | ||
| failures: list[str] = [] | ||
| for cmd, _expected_code in guard_cases: | ||
| code, _stdout, stderr = run(cmd, cwd=PROJECT_ROOT) | ||
| # Guard commands generally return 0 or 2 (not installed = not error) | ||
| passed = code in (0, 2) | ||
| status = "PASS" if passed else "FAIL" | ||
| print(f" [{status}] {' '.join(cmd[3:])} (exit={code})") | ||
| if not passed: | ||
| print(f" DEBUG stderr: {stderr[:300]}") | ||
| failures.append(f"guard: {' '.join(cmd[3:])} exit={code}") | ||
| return failures | ||
|
|
||
| def test_droid_exec(): | ||
| prompt = ( | ||
| "Run hol-guard scan against tests/fixtures/good-plugin with --json" | ||
| " and report only the score." | ||
| ) | ||
| cmd = [ | ||
| "droid", "exec", | ||
| "--cwd", str(PROJECT_ROOT), | ||
| "--auto", "medium", | ||
| prompt, | ||
| ] | ||
| code, stdout, stderr = run(cmd) | ||
| # Check droid exec ran successfully (stdout contains score number) | ||
| passed = code == 0 and (len(stdout.strip()) > 0 or "score" in stderr.lower()) | ||
| status = "PASS" if passed else "FAIL" | ||
| print(f" [{status}] droid exec headless run (exit={code})") | ||
| if not passed: | ||
| print(f" DEBUG stdout: {stdout[:500]}") | ||
| print(f" DEBUG stderr: {stderr[:500]}") | ||
| return [] if passed else ["droid exec headless run failed"] | ||
|
|
||
| if __name__ == "__main__": | ||
| all_failures: list[str] = [] | ||
| print("=== Scanner E2E ===") | ||
| all_failures.extend(test_scanner()) | ||
| print("\n=== Guard E2E ===") | ||
| all_failures.extend(test_guard()) | ||
| print("\n=== Droid Exec E2E ===") | ||
| all_failures.extend(test_droid_exec()) | ||
|
|
||
| if all_failures: | ||
| print(f"\nFAILED ({len(all_failures)}):") | ||
| for f in all_failures: | ||
| print(f" - {f}") | ||
| sys.exit(1) | ||
| print("\nSUCCESS: all tests passed") | ||
| sys.exit(0) | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.