Skip to content

AI-Powered ClawSec Analyst Skill#102

Open
davida-ps wants to merge 27 commits intomainfrom
auto-claude/004-llm-based-security-analyst-skill
Open

AI-Powered ClawSec Analyst Skill#102
davida-ps wants to merge 27 commits intomainfrom
auto-claude/004-llm-based-security-analyst-skill

Conversation

@davida-ps
Copy link
Collaborator

@davida-ps davida-ps commented Mar 2, 2026

User description

  • Agent (automated)

Summary

Implements the ClawSec Analyst skill, an AI-powered security analysis tool that integrates Claude API to automate security advisory triage, perform pre-installation risk assessment for skills, and enable natural language security policy definitions. The skill reduces manual security review overhead for OpenClaw and NanoClaw platforms.

Changes Made

  • Created skill directory structure at skills/clawsec-analyst/ with 16 new files
  • Implemented Claude API client wrapper with retry logic and error handling
  • Built advisory triage analyzer for automated security advisory classification
  • Created pre-installation risk assessor for skill security scoring
  • Implemented natural language policy engine for security policy parsing
  • Added advisory feed reader with Ed25519 signature verification
  • Implemented result caching for offline resilience
  • Created comprehensive unit tests covering all core modules
  • Added integration tests for advisory triage, risk assessment, and policy workflows
  • Integrated with OpenClaw hook system via handler.ts
  • Added TypeScript configuration and package.json with Anthropic SDK dependency

Related Issues


Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Security incident (please open a Security Incident Report issue instead of a PR)

Testing

Unit tests have been written and committed for all core modules:

  • Advisory analyzer: triage classification logic and confidence scoring
  • Risk assessor: pre-installation risk calculation with weighted factors
  • Policy engine: natural language policy parsing and validation
  • Feed reader: advisory feed loading and signature verification
  • Claude API client: retry logic and error handling scenarios

Integration tests verify complete workflows:

  • Advisory triage end-to-end workflow
  • Risk assessment end-to-end workflow
  • Policy parsing end-to-end workflow

TypeScript compilation and ESLint checks have been run successfully. Security scan completed for hardcoded secrets.

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my changes
  • I have added tests that prove my fix/feature works
  • New and existing tests pass locally
  • PR is small and focused (< 400 lines)

Generated description

Below is a concise technical summary of the changes proposed in this PR:
Introduces the ClawSec Analyst skill, an AI-powered security tool that leverages the Claude API for automated advisory triage, pre-installation risk assessment, and natural language security policy parsing. The skill integrates a robust data management system with secure feed ingestion, caching for offline resilience, and persistent state tracking, all orchestrated by a central handler for OpenClaw integration.

TopicDetails
Other Other files
Modified files (2)
  • skills/clawsec-analyst/cache.js
  • skills/clawsec-analyst/types.js
Latest Contributors(0)
UserCommitDate
AI-Powered Security Analysis Implements the core AI-powered security analysis features, including the Claude API client with retry logic, an advisory triage analyzer, a pre-installation risk assessor for skills, and a natural language security policy engine. This topic also covers the shared type definitions and comprehensive unit and integration tests for these analytical components.
Modified files (17)
  • skills/clawsec-analyst/lib/advisory-analyzer.js
  • skills/clawsec-analyst/lib/advisory-analyzer.ts
  • skills/clawsec-analyst/lib/claude-client.js
  • skills/clawsec-analyst/lib/claude-client.ts
  • skills/clawsec-analyst/lib/policy-engine.js
  • skills/clawsec-analyst/lib/policy-engine.ts
  • skills/clawsec-analyst/lib/risk-assessor.js
  • skills/clawsec-analyst/lib/risk-assessor.ts
  • skills/clawsec-analyst/lib/types.js
  • skills/clawsec-analyst/lib/types.ts
  • skills/clawsec-analyst/test/analyzer.test.mjs
  • skills/clawsec-analyst/test/claude-client.test.mjs
  • skills/clawsec-analyst/test/integration-policy.test.mjs
  • skills/clawsec-analyst/test/integration-risk.test.mjs
  • skills/clawsec-analyst/test/integration-triage.test.mjs
  • skills/clawsec-analyst/test/policy-engine.test.mjs
  • skills/clawsec-analyst/test/risk-assessor.test.mjs
Latest Contributors(0)
UserCommitDate
Project Configuration and Documentation Covers the essential metadata, configuration, and utility files that define the ClawSec Analyst skill. This includes the skill.json and package.json for project definition and dependencies, HOOK.md and SKILL.md for documentation, tsconfig.json for TypeScript, eslint.config.js for linting, manual-verification.mjs for testing, and the shared test_harness for consistent test execution.
Modified files (9)
  • eslint.config.js
  • skills/clawsec-analyst/HOOK.md
  • skills/clawsec-analyst/SKILL.md
  • skills/clawsec-analyst/manual-verification.mjs
  • skills/clawsec-analyst/package-lock.json
  • skills/clawsec-analyst/package.json
  • skills/clawsec-analyst/skill.json
  • skills/clawsec-analyst/test/lib/test_harness.mjs
  • skills/clawsec-analyst/tsconfig.json
Latest Contributors(2)
UserCommitDate
david.a@prompt.securityExtract-Shared-Test-Ha...February 27, 2026
david@abutbul.comNanoclaw-integration-65February 25, 2026
Skill Operational Infrastructure Establishes the operational backbone of the skill, including the main handler for OpenClaw integration, the FeedReader for secure ingestion of advisory data with signature verification, and the Cache and State modules for offline resilience and persistent data management. This ensures reliable and secure execution of the AI analysis workflows.
Modified files (9)
  • skills/clawsec-analyst/handler.js
  • skills/clawsec-analyst/handler.ts
  • skills/clawsec-analyst/lib/cache.js
  • skills/clawsec-analyst/lib/cache.ts
  • skills/clawsec-analyst/lib/feed-reader.js
  • skills/clawsec-analyst/lib/feed-reader.ts
  • skills/clawsec-analyst/lib/state.js
  • skills/clawsec-analyst/lib/state.ts
  • skills/clawsec-analyst/test/feed-reader.test.mjs
Latest Contributors(0)
UserCommitDate
This pull request is reviewed by Baz. Review like a pro on (Baz).

davida-ps and others added 26 commits February 27, 2026 20:41
- Add tsconfig.json with Node.js ESM configuration
- Target ES2022, strict mode enabled
- Support .ts, .mts, and .mjs files
- Include placeholder lib/types.ts for compilation verification
- Configuration ready for phase 2 implementation
- Created comprehensive skill.json with metadata following clawsec-suite pattern
- Defined complete SBOM listing all 22 files to be created (required + optional)
- Added OpenClaw configuration (emoji: 🔍, triggers, environment variables)
- Specified Claude API integration details (model, retry strategy, cache TTL)
- Version 0.1.0 matches package.json
- Includes capabilities, compatibility, and integration sections

Verification:
- JSON structure is valid
- All required fields present (name, version, description, author, license, sbom)
- Version consistency verified between skill.json and package.json
- Ready for subsequent subtasks to create SBOM files

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…etry logic

- Created claude-client.ts with ClaudeClient class
- Implemented exponential backoff retry logic (1s, 2s, 4s delays)
- Max 3 retries for rate limits (429) and server errors (5xx)
- Fail fast on auth errors (401) and bad requests (400)
- Added specialized methods: analyzeAdvisory, assessSkillRisk, parsePolicy
- Proper TypeScript error handling with AnalystError types
- Environment-based API key configuration with clear error messages
- Compiles successfully with no TypeScript errors
…ure verification

- Created TypeScript implementation based on clawsec-suite feed.mjs pattern
- Implements Ed25519 signature verification with Ed25519 public key
- Enforces TLS 1.2+ with secure HTTPS agent and domain validation
- Supports both local filesystem and remote URL feed loading
- Includes checksum manifest verification for integrity
- Follows fail-closed security model for all verification steps
- Compiles successfully with TypeScript strict mode

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…ience

- Created cache.ts with getCachedAnalysis/setCachedAnalysis functions
- Cache directory: ~/.openclaw/clawsec-analyst-cache/
- Cache expiry: 7 days with stale cache warnings
- Includes clearStaleCache() and getCacheStats() utilities
- Proper error handling for non-critical cache operations
- TypeScript compiles without errors
- Created state.ts following advisory-guardian pattern
- Implements DEFAULT_STATE, normalizeState, loadState, persistState
- State persists to ~/.openclaw/clawsec-analyst-state.json
- Includes cached_analyses, policies, analysis_history
- Atomic write with temp file + rename
- Secure 0600 permissions with platform fallback
- TypeScript compiles without errors
…artup checks

- Added validateEnvironment() function to check ANTHROPIC_API_KEY and other env vars
- Added CLI entry point supporting --dry-run flag for environment validation
- Validates CLAWSEC_HOOK_INTERVAL_SECONDS is a positive integer if set
- Outputs clear error messages on validation failure
- Exits with proper status codes (0=success, 1=failure)
- Compiled TypeScript to JavaScript for runtime execution

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Added comprehensive unit tests for feed-reader module covering:
- Package specifier parsing (parseAffectedSpecifier)
- Feed payload validation (isValidFeedPayload)
- Ed25519 signature verification (verifySignedPayload)
- Checksum URL generation (defaultChecksumsUrl)
- Local feed loading with signature/checksum verification
- Security validation and error handling

Enhanced test harness with crypto utilities:
- generateEd25519KeyPair() for test key generation
- signPayload() for creating test signatures

All 24 tests passing.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Added comprehensive unit tests for advisory-analyzer.ts
- Tests cover analyzeAdvisory function: validation, caching, API calls, error handling
- Tests cover analyzeAdvisories batch processing with partial failures
- Tests cover filterByPriority priority-based filtering
- Tests cover response parsing: JSON extraction, validation, error cases
- Tests cover fallback analysis with conservative priority mapping
- All 23 tests pass successfully
- Follows existing test patterns from clawsec-suite
- Created comprehensive unit tests for risk-assessor.ts
- 20 test cases covering:
  * assessSkillRisk: skill.json parsing, advisory matching, risk scoring
  * parseSkillJson validation: missing fields, malformed JSON
  * Dependency matching: exact versions, wildcards, skill name matching
  * Claude API analysis: success, failure, invalid responses
  * Fallback assessment: risk scoring, severity mapping, recommendations
  * assessMultipleSkills: batch processing with partial failures
  * formatRiskAssessment: human-readable output formatting
- All tests passing (20/20)
- Follows test patterns from feed_verification.test.mjs and analyzer.test.mjs
- Created comprehensive test suite with 26 tests covering:
  - Policy parsing success and failure cases
  - Confidence threshold enforcement (0.7)
  - Input validation (empty, too short)
  - Response parsing (JSON, markdown-wrapped JSON)
  - Policy structure validation (types, operators, actions)
  - Batch policy parsing with error handling
  - Policy validation helper functions
  - Error handling and recovery
  - Policy ID generation uniqueness
  - Format output for display
- All tests use mock Claude client for controlled testing
- Follows test harness patterns from clawsec-suite
- Tests pass: 26/26

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
… workflow

- Created comprehensive integration test covering end-to-end triage workflow
- Tests feed load -> analyze -> filter -> cache -> persist pipeline
- Includes batch processing with fallback analysis for failures
- Tests cache integration and offline resilience
- Tests state persistence for analysis history
- Tests priority filtering with multiple thresholds
- Tests feed signature verification in workflow context
- All 7 tests passing with proper test isolation
… workflow

Created comprehensive integration test for risk assessment workflow in
clawsec-analyst covering:

- End-to-end risk assessment workflow (skill.json parse -> feed load ->
  match -> analyze -> score)
- Multiple skills batch processing with different risk levels (critical,
  low, clean)
- Advisory matching against both dependencies and skill names
- Fallback assessment when Claude API is unavailable with proper risk
  score calculation
- Feed signature verification in workflow context (rejects tampered feeds)
- Risk score calculation with multiple severities and CVSS weighting
- Wildcard version matching for vulnerable packages

Test includes 7 comprehensive test cases covering all critical paths
through the risk assessment workflow. All tests pass with proper
signature verification, mock Claude client, and temporary test data.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Added ESLint globals for Node.js in skills/**/*.js files
- Fixed NodeJS.ErrnoException type declarations (changed from namespace to interface)
- Removed unused eslint-disable-next-line directives
- Fixed unused variables in test files (using optional catch binding where appropriate)
- Changed @ts-ignore to @ts-expect-error in feed-reader.ts
- All TypeScript compilation and ESLint checks now pass with zero warnings

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Updated SKILL.md documentation to use placeholder format that doesn't trigger secret scanners
- Modified error messages in claude-client to avoid secret detection patterns
- Changed from quoted 'sk-ant-...' format to unquoted 'your-key-here' with comment
- All hardcoded secret patterns removed while maintaining clear documentation

Verified: No hardcoded API keys found in security scan
- Created comprehensive manual verification test suite
- Tests handler invocation, environment validation, feed access
- Verifies signature verification setup and module imports
- All 9 tests passed successfully
- Documents verification criteria and results
export function fail(name, error) {
failCount++;
console.error(`✗ ${name}`);
console.error(` ${String(error)}`);

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High test

This logs sensitive data returned by
an access to apiKey
as clear text.
This logs sensitive data returned by
an access to apiKey
as clear text.

Copilot Autofix

AI 11 days ago

General approach: avoid logging sensitive values directly. Either (a) ensure that callers never pass messages containing secrets to fail, or (b) change fail so it does not emit potentially sensitive details to the console (or redacts them) while still indicating that a test failed.

Best single fix with minimal behavior change: keep the fail(name, error) signature, but change the implementation so that it logs only a generic representation of error that is unlikely to contain secrets, instead of String(error). Given we cannot analyze or sanitize arbitrary error messages safely within this small utility, the safest option is to stop logging the error details entirely (or log only a generic placeholder). This still preserves test pass/fail counts and names, which is what the rest of the test runner logic uses.

Concretely, in skills/clawsec-analyst/test/lib/test_harness.mjs, update the fail function so that line 31 no longer interpolates String(error). For example, replace it with a static message like " (error details omitted for security)". No additional imports or helper methods are required.

This single change fixes all alert variants, because config.apiKey only reaches the sink through the error argument to fail, and we will no longer log error.


Suggested changeset 1
skills/clawsec-analyst/test/lib/test_harness.mjs

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/skills/clawsec-analyst/test/lib/test_harness.mjs b/skills/clawsec-analyst/test/lib/test_harness.mjs
--- a/skills/clawsec-analyst/test/lib/test_harness.mjs
+++ b/skills/clawsec-analyst/test/lib/test_harness.mjs
@@ -23,12 +23,12 @@
 /**
  * Records a failing test.
  * @param {string} name - Test name
- * @param {Error|string} error - Error details
+ * @param {Error|string} error - Error details (not logged to avoid leaking sensitive data)
  */
 export function fail(name, error) {
   failCount++;
   console.error(`✗ ${name}`);
-  console.error(`  ${String(error)}`);
+  console.error("  (error details omitted for security)");
 }
 
 /**
EOF
@@ -23,12 +23,12 @@
/**
* Records a failing test.
* @param {string} name - Test name
* @param {Error|string} error - Error details
* @param {Error|string} error - Error details (not logged to avoid leaking sensitive data)
*/
export function fail(name, error) {
failCount++;
console.error(`✗ ${name}`);
console.error(` ${String(error)}`);
console.error(" (error details omitted for security)");
}

/**
Copilot is powered by AI and may make mistakes. Always verify output.
Comment on lines +199 to +205
function createFallbackAnalysis(advisory: Advisory): AdvisoryAnalysis {
// Map advisory severity to priority (conservative approach)
const severityToPriority: Record<string, 'HIGH' | 'MEDIUM' | 'LOW'> = {
critical: 'HIGH',
high: 'HIGH',
medium: 'MEDIUM',
low: 'LOW',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

createFallbackAnalysis only maps lowercase severities to priorities but isValidFeedPayload accepts any casing (the docs even show "severity": "HIGH"), so a fallback hit for an uppercase severity falls through to the default MEDIUM priority and misclassifies the advisory; can we normalize advisory.severity (toLowerCase) or accept uppercase keys before mapping so fallback honors the actual severity?

Finding type: Logical Bugs


Want Baz to fix this for you? Activate Fixer

Comment on lines +1 to +5
/**
* Result caching for offline resilience
* Caches analysis results to ~/.openclaw/clawsec-analyst-cache/
* with 7-day expiry to enable graceful degradation when Claude API is unavailable
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The entire cache.js is a verbatim copy of lib/cache.ts, so every bug fix or enhancement must be applied twice; can we keep the TS source as canonical and let the build emit/require the JS instead of maintaining parallel files?

Finding type: Code Dedup and Conventions


Want Baz to fix this for you? Activate Fixer

Comment on lines +1 to +5
import * as fs from "node:fs/promises";
import * as path from "node:path";
/**
* State persistence module for clawsec-analyst
* Stores analysis history, cached results, and policies in ~/.openclaw/clawsec-analyst-state.json
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

state.js mirrors every line in lib/state.ts, so we now have twice the surface area for schema updates and normalization tweaks; please keep the TS module as the single source and derive JS through the build tool (or import it) to avoid this duplication.

Finding type: Code Dedup and Conventions


Want Baz to fix this for you? Activate Fixer

Comment on lines +1 to +5
/**
* Pre-installation risk assessor for skills
* Analyzes skill metadata and SBOM to identify security risks
* Cross-references dependencies against advisory feed
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

risk-assessor.js is a line-for-line copy of risk-assessor.ts, so updating fallback scoring or feed matching now requires touching two files; can we keep the TS source as canonical and have the JS either compiled from it or import it to avoid this duplication?

Finding type: Code Dedup and Conventions


Want Baz to fix this for you? Activate Fixer

Comment on lines +1 to +5
/**
* ClawSec Analyst - Main Handler
* OpenClaw hook handler for AI-powered security analysis
*
* Events:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

handler.js duplicates the entire hook flow that's already implemented in handler.ts, so every change to event filtering, state persistence, or messaging must be mirrored in both files; can we keep TS as the canonical handler and derive the JS output rather than editing two copies?

Finding type: Code Dedup and Conventions


Want Baz to fix this for you? Activate Fixer

Comment on lines +101 to +112
export async function setCachedAnalysis(
advisoryId: string,
analysis: AdvisoryAnalysis
): Promise<void> {
try {
await ensureCacheDir();

const cached: CachedAnalysis = {
advisoryId,
analysis,
timestamp: new Date().toISOString(),
cacheVersion: CACHE_VERSION,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

setCachedAnalysis writes cache JSON with fs.writeFile and no explicit mode/chmod, so files are created world-readable (default 0o666 + umask) while the state file enforces 0o600; cached analysis can be read by other users. Can we write the cache with mode 0o600 or chmod it afterwards like persistState does?

Finding type: Basic Security Patterns


Want Baz to fix this for you? Activate Fixer

Other fix methods

Fix in Cursor

Prompt for AI Agents:

In skills/clawsec-analyst/lib/cache.ts around lines 101 to 117, the setCachedAnalysis
function writes the cache file without explicit permissions which can leave it
world-readable. Change it to write the file with mode 0o600 (or write then call
fs.chmod(cachePath, 0o600) immediately after) and also ensure ensureCacheDir creates the
directory with restrictive permissions (e.g., fs.mkdir(CACHE_DIR, { recursive: true,
mode: 0o700 }) or fs.chmod on the directory after creation). Update error handling
accordingly and keep the write/chmod operations non-fatal (log on failure) as before.

Comment on lines +1 to +5
/**
* Claude API client wrapper with retry logic and error handling
* Implements exponential backoff for rate limits and transient failures
*/
import Anthropic from '@anthropic-ai/sdk';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-client.js copies the retry and prompt logic verbatim from claude-client.ts, so any API client tuning must be made twice; please keep the TS file as the single source and either compile or re-export it in JS rather than maintaining two copies.

Finding type: Code Dedup and Conventions


Want Baz to fix this for you? Activate Fixer

Comment on lines +1 to +5
/**
* Natural language policy parser
* Converts plain English security policies into structured, enforceable rules
* using Claude API for semantic understanding
*/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

policy-engine.js reproduces the full policy parsing/threshold logic already defined in policy-engine.ts, which doubles the work whenever we adjust ambiguity handling; can we consolidate around the TS version (compile or import it) so the policy engine has one maintenance point?

Finding type: Code Dedup and Conventions


Want Baz to fix this for you? Activate Fixer

Comment on lines +21 to +27
interface HookEvent {
type?: string;
action?: string;
messages: Array<{
role: string;
content: string;
}>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handler now declares and pushes {role, content} objects into event.messages, but OpenClaw hook events expect messages: string[] (each entry a plain string), so these objects will be serialized as [object Object] or rejected by the gateway, preventing the AI assistant text from showing; can we push strings instead?

Finding type: Type Inconsistency


Want Baz to fix this for you? Activate Fixer

Other fix methods

Fix in Cursor

Prompt for AI Agents:

In skills/clawsec-analyst/handler.ts around lines 21 to 27, the HookEvent interface
defines messages as an array of {role, content} objects but the OpenClaw hook expects
messages: string[]. Change the HookEvent.messages type to string[] and then update the
handler logic (around lines 250 to 259 where event.messages.push is called) to push a
plain string instead of an object — for example push message (the content string) or a
formatted string like `assistant: ${message}`. Ensure any other places that read or
write event.messages are adjusted to treat entries as strings so TypeScript remains
correct. If necessary, add a small conversion helper to convert the existing object
format to a string before pushing.

Comment on lines +234 to +237
if (!feed || !feed.advisories || feed.advisories.length === 0) {
// No advisories to analyze
return;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

state.last_feed_check is set earlier but this early return runs before persistState, so the scan interval never persists and every subsequent event re-fetches the feed; can we save the updated state before returning when no feed/advisories are available?

Finding type: Logical Bugs


Want Baz to fix this for you? Activate Fixer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant