From a4277cc469fe9039a4de26d0468a55306f38c323 Mon Sep 17 00:00:00 2001 From: Anthony Ettinger Date: Tue, 28 Jul 2026 13:49:50 +0000 Subject: [PATCH] refactor(modules): fold dep-scanner into code-scanner as the deps subsystem MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Resolves the open module-boundary question in PRD 0002. PRD.md already assigns "dependency CVEs" to code-scanner, so shipping a separate dep-scanner module contradicted the product doc. Two modules would also have walked the same trees, held two inventories of the same packages, and given the operator two path lists to keep in sync — for a user-visible surface (`threatcrush scan --deps`) that was always meant to be one command. The arguments for splitting were real but all internal: a package-graph data model, an external advisory database, a per-ecosystem parser surface that will keep growing. Those are preserved as a subsystem boundary under src/deps/ rather than a module boundary, alongside stubs for the rest of code-scanner's remit (secrets/, sast/, config/). modules/dep-scanner/src/{inventory,osv,scripts,semver}.ts -> modules/code-scanner/src/deps/ src/deps/index.ts is the subsystem entry point: scanDependencies() takes options and returns a result with no ModuleContext, so the same code serves the scheduled daemon scan and the one-shot `scan --deps`. src/index.ts is now a thin host that runs enabled subsystems and owns alerting and dedupe. Dependency-specific config is namespaced deps_* so sibling subsystems can be configured without collision; paths, min_severity, scan interval and the fail_on_unparseable honesty switch stay shared. Behaviour is unchanged — 41 tests pass, and a scan of the reference pnpm workspace still resolves 21 roots and 101 packages rather than the "0 dependencies, coupling 0/100 (good)" that motivated the PRD. Co-Authored-By: Claude Opus 4.8 --- .../{dep-scanner => code-scanner}/.gitignore | 0 modules/{dep-scanner => code-scanner}/LICENSE | 0 .../{dep-scanner => code-scanner}/README.md | 28 +- .../config/example.conf.toml | 8 +- modules/code-scanner/mod.toml | 63 ++++ .../package.json | 8 +- .../src/__tests__/inventory.test.ts | 4 +- .../src/__tests__/scripts.test.ts | 2 +- .../src/__tests__/semver.test.ts | 2 +- modules/code-scanner/src/deps/index.ts | 131 +++++++ .../src/deps}/inventory.ts | 0 .../src => code-scanner/src/deps}/osv.ts | 0 .../src => code-scanner/src/deps}/scripts.ts | 0 .../src => code-scanner/src/deps}/semver.ts | 0 modules/code-scanner/src/index.ts | 299 ++++++++++++++++ .../tsconfig.json | 0 modules/dep-scanner/mod.toml | 64 ---- modules/dep-scanner/src/index.ts | 338 ------------------ pnpm-lock.yaml | 16 + ...licious-dependencies-on-running-servers.md | 74 ++-- prd/README.md | 2 +- 21 files changed, 585 insertions(+), 454 deletions(-) rename modules/{dep-scanner => code-scanner}/.gitignore (100%) rename modules/{dep-scanner => code-scanner}/LICENSE (100%) rename modules/{dep-scanner => code-scanner}/README.md (77%) rename modules/{dep-scanner => code-scanner}/config/example.conf.toml (94%) create mode 100644 modules/code-scanner/mod.toml rename modules/{dep-scanner => code-scanner}/package.json (76%) rename modules/{dep-scanner => code-scanner}/src/__tests__/inventory.test.ts (98%) rename modules/{dep-scanner => code-scanner}/src/__tests__/scripts.test.ts (99%) rename modules/{dep-scanner => code-scanner}/src/__tests__/semver.test.ts (99%) create mode 100644 modules/code-scanner/src/deps/index.ts rename modules/{dep-scanner/src => code-scanner/src/deps}/inventory.ts (100%) rename modules/{dep-scanner/src => code-scanner/src/deps}/osv.ts (100%) rename modules/{dep-scanner/src => code-scanner/src/deps}/scripts.ts (100%) rename modules/{dep-scanner/src => code-scanner/src/deps}/semver.ts (100%) create mode 100644 modules/code-scanner/src/index.ts rename modules/{dep-scanner => code-scanner}/tsconfig.json (100%) delete mode 100644 modules/dep-scanner/mod.toml delete mode 100644 modules/dep-scanner/src/index.ts diff --git a/modules/dep-scanner/.gitignore b/modules/code-scanner/.gitignore similarity index 100% rename from modules/dep-scanner/.gitignore rename to modules/code-scanner/.gitignore diff --git a/modules/dep-scanner/LICENSE b/modules/code-scanner/LICENSE similarity index 100% rename from modules/dep-scanner/LICENSE rename to modules/code-scanner/LICENSE diff --git a/modules/dep-scanner/README.md b/modules/code-scanner/README.md similarity index 77% rename from modules/dep-scanner/README.md rename to modules/code-scanner/README.md index ee5bd1f..f906b62 100644 --- a/modules/dep-scanner/README.md +++ b/modules/code-scanner/README.md @@ -1,15 +1,31 @@ -# dep-scanner +# code-scanner -Detects vulnerable and malicious dependencies on **running servers**. +Static analysis on codebases. `PRD.md` scopes this module to "vulnerabilities, +secrets, misconfigs, dependency CVEs" — four different detection models over the +same directories, so the module is a thin host and each one is a **subsystem**: -Implements [PRD 0002](../../prd/0002-detect-vulnerable-and-malicious-dependencies-on-running-servers.md). +| Subsystem | Covers | Status | +| --- | --- | --- | +| `deps/` | Dependency advisories, malicious install scripts, lockfile drift | **implemented** ([PRD 0002](../../prd/0002-detect-vulnerable-and-malicious-dependencies-on-running-servers.md)) | +| `secrets/` | Hardcoded credentials | not yet built | +| `sast/` | Source-level vulnerability analysis | not yet built | +| `config/` | Misconfiguration checks | not yet built | ```bash -threatcrush modules install dep-scanner +threatcrush modules install code-scanner threatcrush scan --deps /srv/app ``` -## Why another dependency scanner +PRD 0002 originally proposed the dependency work as a standalone `dep-scanner` +module and left the boundary open. It is resolved in favour of folding in: two +modules would have walked the same trees, held two inventories of the same +packages, and given the operator two path lists to keep in sync — for a +user-visible surface (`threatcrush scan --deps`) that was always meant to be one +command. What survives the fold is the *internal* separation, because `deps/` +owns an external advisory database and a per-ecosystem parser surface that has +nothing in common with a secrets regex pass. + +## Why the deps subsystem exists Two failures motivated this module, and both are things the existing tools do badly or cannot do at all. @@ -27,7 +43,7 @@ Every number was wrong. It read the root `package.json`, never resolved `pnpm-workspace.yaml`, found nothing, and called the nothing *good*. For a productivity tool that wastes an afternoon; for a security tool it is a clean bill of health issued without an examination. Run against the same workspace, -this module reports **21 manifests, 59 packages**, and marks the root `partial` +this subsystem reports **21 project roots and 101 packages**, and marks the root `partial` with the reason — because a pnpm lockfile is not parsed yet, and saying so is the entire point. diff --git a/modules/dep-scanner/config/example.conf.toml b/modules/code-scanner/config/example.conf.toml similarity index 94% rename from modules/dep-scanner/config/example.conf.toml rename to modules/code-scanner/config/example.conf.toml index ffc9b9a..c0135f8 100644 --- a/modules/dep-scanner/config/example.conf.toml +++ b/modules/code-scanner/config/example.conf.toml @@ -1,4 +1,4 @@ -# Example: /etc/threatcrush/threatcrushd.conf.d/dep-scanner.conf +# Example: /etc/threatcrush/threatcrushd.conf.d/code-scanner.conf # # Values here override [module.config.defaults] in mod.toml. @@ -27,7 +27,7 @@ fail_on_unparseable = true # Set false on an air-gapped host: install-script scoring and lockfile drift # detection both keep working without network access, and they are the two # detectors that catch a malicious package before an advisory exists. -advisories_enabled = true +deps_advisories = true # Alerting floor. Findings below this are still counted and logged, just not # alerted on. @@ -40,7 +40,7 @@ min_severity = "medium" # There is deliberately no "ignore" — a finding nobody ever sees is # indistinguishable from one that was missed. Build tooling still executes on # your build host, and that host usually holds deploy credentials. -dev_dependencies = "rank_down" +deps_dev_dependencies = "rank_down" # --- Behavioural detection ------------------------------------------------- # Score preinstall/install/postinstall scripts on what they do: fetching remote @@ -49,7 +49,7 @@ dev_dependencies = "rank_down" # This is the detector that does not wait for a CVE. event-stream (2018), # ua-parser-js (2021) and node-ipc (2022) were all advisory-clean at the moment # they were installed; all three executed from a lifecycle script. -install_scripts = true +deps_install_scripts = true # --- Reporting ------------------------------------------------------------- # Cap alerts per scan so the first run on a long-neglected host summarizes diff --git a/modules/code-scanner/mod.toml b/modules/code-scanner/mod.toml new file mode 100644 index 0000000..597fb22 --- /dev/null +++ b/modules/code-scanner/mod.toml @@ -0,0 +1,63 @@ +[module] +name = "code-scanner" +version = "0.1.0" +description = "Static analysis on codebases — dependency advisories, malicious install scripts, and lockfile drift" +author = "Profullstack, Inc." +license = "MIT" +homepage = "https://github.com/profullstack/threatcrush/tree/master/modules/code-scanner" + +[module.pricing] +type = "free" + +[module.requirements] +threatcrush = ">=0.2.0" +os = ["linux", "darwin"] +capabilities = ["fs:read", "network:outbound"] + +[module.config.defaults] +enabled = true + +# Where to look for code. Each path is walked for project roots, skipping +# node_modules, VCS and build directories. Shared by every subsystem. +paths = ["/srv", "/var/www", "/opt"] + +# How deep to walk below each configured path when looking for project roots. +max_depth = 4 + +# Dependencies change on deploy, not by the minute. Six hours keeps advisory +# data fresh without turning the scanner into background load. +scan_interval_seconds = 21600 + +# Alerting floor, shared by every subsystem. Findings below this are still +# counted and logged, just not alerted on. +min_severity = "medium" + +# Cap alerts per scan so a first run on a neglected host summarizes instead of +# flooding the alert channel. Findings over the cap carry to the next scan. +max_alerts = 25 + +# --- Honesty --------------------------------------------------------------- +# Alert when a project could not be parsed at all. Leave this on: the failure +# this scanner was written to prevent is reporting "nothing found" when the +# truth is "nothing was read". An unexamined project is not a clean one. +fail_on_unparseable = true + +# --- deps subsystem: dependency and supply-chain scanning (PRD 0002) ------- +deps_enabled = true + +# Look up installed versions against OSV.dev (aggregates GHSA, PyPA, RustSec, +# Go vulndb). Set false for a fully offline host; install-script scoring and +# drift detection continue to work without network access. +deps_advisories = true + +# Score preinstall/install/postinstall scripts on what they do. This is what +# catches a malicious package before an advisory exists, which is the window in +# which event-stream, ua-parser-js and node-ipc all did their damage. +deps_install_scripts = true + +# How to treat findings reachable only through devDependencies: +# rank_down demote one severity level (default) +# equal treat identically to runtime dependencies +# There is deliberately no "ignore": a finding an operator never sees is +# indistinguishable from one that was missed. +deps_dev_dependencies = "rank_down" diff --git a/modules/dep-scanner/package.json b/modules/code-scanner/package.json similarity index 76% rename from modules/dep-scanner/package.json rename to modules/code-scanner/package.json index 1196aca..1ae2348 100644 --- a/modules/dep-scanner/package.json +++ b/modules/code-scanner/package.json @@ -1,17 +1,17 @@ { - "name": "threatcrush-module-dep-scanner", + "name": "threatcrush-module-code-scanner", "version": "0.1.0", - "description": "Scans installed dependencies for known advisories, malicious install scripts and lockfile drift", + "description": "Static analysis on codebases — dependency advisories, malicious install scripts, and lockfile drift", "type": "module", "main": "dist/index.js", "types": "dist/index.d.ts", "license": "MIT", "author": "Profullstack, Inc.", - "homepage": "https://threatcrush.com/store/dep-scanner", + "homepage": "https://threatcrush.com/store/code-scanner", "repository": { "type": "git", "url": "git+https://github.com/profullstack/threatcrush.git", - "directory": "modules/dep-scanner" + "directory": "modules/code-scanner" }, "keywords": [ "threatcrush", diff --git a/modules/dep-scanner/src/__tests__/inventory.test.ts b/modules/code-scanner/src/__tests__/inventory.test.ts similarity index 98% rename from modules/dep-scanner/src/__tests__/inventory.test.ts rename to modules/code-scanner/src/__tests__/inventory.test.ts index 5d9b493..4ec3806 100644 --- a/modules/dep-scanner/src/__tests__/inventory.test.ts +++ b/modules/code-scanner/src/__tests__/inventory.test.ts @@ -8,12 +8,12 @@ import { inventoryRoot, parsePackageLock, readWorkspacePatterns, -} from '../inventory.js'; +} from '../deps/inventory.js'; let dir: string; beforeEach(async () => { - dir = await mkdtemp(join(tmpdir(), 'dep-scanner-')); + dir = await mkdtemp(join(tmpdir(), 'code-scanner-')); }); afterEach(async () => { diff --git a/modules/dep-scanner/src/__tests__/scripts.test.ts b/modules/code-scanner/src/__tests__/scripts.test.ts similarity index 99% rename from modules/dep-scanner/src/__tests__/scripts.test.ts rename to modules/code-scanner/src/__tests__/scripts.test.ts index cfd270a..1b6b9e8 100644 --- a/modules/dep-scanner/src/__tests__/scripts.test.ts +++ b/modules/code-scanner/src/__tests__/scripts.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from 'vitest'; -import { extractScripts, rankScripts, scoreScript, type InstallScript } from '../scripts.js'; +import { extractScripts, rankScripts, scoreScript, type InstallScript } from '../deps/scripts.js'; const script = (command: string, stage: InstallScript['stage'] = 'postinstall'): InstallScript => ({ packageName: 'example', diff --git a/modules/dep-scanner/src/__tests__/semver.test.ts b/modules/code-scanner/src/__tests__/semver.test.ts similarity index 99% rename from modules/dep-scanner/src/__tests__/semver.test.ts rename to modules/code-scanner/src/__tests__/semver.test.ts index c549c7b..2d80f86 100644 --- a/modules/dep-scanner/src/__tests__/semver.test.ts +++ b/modules/code-scanner/src/__tests__/semver.test.ts @@ -4,7 +4,7 @@ import { fixedVersionFor, isVersionAffected, parseVersion, -} from '../semver.js'; +} from '../deps/semver.js'; /** * Version comparison is the load-bearing primitive: every advisory match runs diff --git a/modules/code-scanner/src/deps/index.ts b/modules/code-scanner/src/deps/index.ts new file mode 100644 index 0000000..f31738a --- /dev/null +++ b/modules/code-scanner/src/deps/index.ts @@ -0,0 +1,131 @@ +/** + * Dependency and supply-chain scanning — the `deps` subsystem of + * `code-scanner`. + * + * Implements PRD 0002, which originally proposed this as a standalone + * `dep-scanner` module and left the boundary open. It is folded in here + * because `PRD.md` already assigns "dependency CVEs" to `code-scanner`, and a + * separate module would have meant two things scanning the same directories, + * holding two inventories of the same tree, and two places for an operator to + * configure a path list. + * + * The subsystem keeps its own directory rather than dissolving into the + * module: it owns an external advisory database, a per-ecosystem parser + * surface that will keep growing, and a detection model (a package graph) + * that has nothing in common with the source-level scanning `code-scanner` + * will add for secrets and SAST. Those are separable concerns that happen to + * share a filesystem walk. + */ + +export * from './inventory.js'; +export * from './osv.js'; +export * from './scripts.js'; +export * from './semver.js'; + +import { readFile } from 'node:fs/promises'; +import { join } from 'node:path'; + +import { buildInventory, type Inventory } from './inventory.js'; +import { queryAdvisories, severityRank, type Finding } from './osv.js'; +import { extractScripts, rankScripts, type ScriptVerdict } from './scripts.js'; + +export interface DepsOptions { + paths: readonly string[]; + maxDepth: number; + advisories: boolean; + installScripts: boolean; + /** `rank_down` demotes dev-only findings; `equal` leaves them alone. */ + devDependencies: 'rank_down' | 'equal'; + /** Called when advisory lookup fails, so the caller can surface it. */ + onDegraded?: (reason: string) => void; +} + +export interface DepsResult { + inventory: Inventory; + findings: Finding[]; + scripts: ScriptVerdict[]; + drift: { root: string; name: string; locked: string; installed: string }[]; + /** True when any root failed to parse — the caller must not report "clean". */ + incomplete: boolean; +} + +export const DEPS_DEFAULTS: DepsOptions = { + paths: ['/srv', '/var/www', '/opt'], + maxDepth: 4, + advisories: true, + installScripts: true, + devDependencies: 'rank_down', +}; + +/** One step down the severity ladder, floored at low. */ +export function demote(severity: Finding['severity']): Finding['severity'] { + const order: Finding['severity'][] = ['info', 'low', 'medium', 'high', 'critical']; + return order[Math.max(1, order.indexOf(severity) - 1)]!; +} + +/** + * Run the whole dependency pass: inventory, advisories, install scripts, drift. + * + * Pure with respect to the daemon — it takes options and returns a result, + * with no `ModuleContext` — so the same entry point serves the scheduled + * daemon scan and the one-shot `threatcrush scan --deps`. + */ +export async function scanDependencies(options: DepsOptions): Promise { + const inventory = await buildInventory(options.paths, options.maxDepth); + const scripts = options.installScripts ? await collectInstallScripts(inventory) : []; + + let findings: Finding[] = []; + if (options.advisories && inventory.packages.length > 0) { + try { + findings = await queryAdvisories( + inventory.packages.map((p) => ({ name: p.name, version: p.version })), + ); + } catch (err) { + // Advisory lookup is the only part needing the network. Losing it must + // degrade the scan, not void it — and must be visible, because a silent + // skip is indistinguishable from "nothing found". + options.onDegraded?.(String(err)); + inventory.incomplete = true; + } + } + + // Dev-only findings are ranked down, never dropped (PRD 0002 R9). + if (options.devDependencies !== 'equal') { + const devOnly = new Set( + inventory.packages.filter((p) => p.dev).map((p) => `${p.name}@${p.version}`), + ); + findings = findings.map((f) => + devOnly.has(`${f.package}@${f.version}`) && severityRank(f.severity) > 1 + ? { ...f, severity: demote(f.severity) } + : f, + ); + } + + const drift = inventory.roots.flatMap((root) => + root.drift.map((entry) => ({ root: root.root, ...entry })), + ); + + return { inventory, findings, scripts, drift, incomplete: inventory.incomplete }; +} + +/** Read lifecycle scripts out of every installed package and score them. */ +async function collectInstallScripts(inventory: Inventory): Promise { + const scripts = []; + + for (const report of inventory.roots) { + for (const record of report.packages) { + if (!record.path.startsWith('node_modules')) continue; + try { + const manifest = JSON.parse( + await readFile(join(report.root, record.path, 'package.json'), 'utf8'), + ) as { name?: string; version?: string; scripts?: Record }; + scripts.push(...extractScripts(manifest, record.name)); + } catch { + // A package whose manifest cannot be read contributes no scripts; the + // root's own parse status already records the trouble. + } + } + } + + return rankScripts(scripts); +} diff --git a/modules/dep-scanner/src/inventory.ts b/modules/code-scanner/src/deps/inventory.ts similarity index 100% rename from modules/dep-scanner/src/inventory.ts rename to modules/code-scanner/src/deps/inventory.ts diff --git a/modules/dep-scanner/src/osv.ts b/modules/code-scanner/src/deps/osv.ts similarity index 100% rename from modules/dep-scanner/src/osv.ts rename to modules/code-scanner/src/deps/osv.ts diff --git a/modules/dep-scanner/src/scripts.ts b/modules/code-scanner/src/deps/scripts.ts similarity index 100% rename from modules/dep-scanner/src/scripts.ts rename to modules/code-scanner/src/deps/scripts.ts diff --git a/modules/dep-scanner/src/semver.ts b/modules/code-scanner/src/deps/semver.ts similarity index 100% rename from modules/dep-scanner/src/semver.ts rename to modules/code-scanner/src/deps/semver.ts diff --git a/modules/code-scanner/src/index.ts b/modules/code-scanner/src/index.ts new file mode 100644 index 0000000..88daff8 --- /dev/null +++ b/modules/code-scanner/src/index.ts @@ -0,0 +1,299 @@ +/** + * code-scanner — static analysis on codebases. + * + * `PRD.md` scopes this module to "vulnerabilities, secrets, misconfigs, + * dependency CVEs". Those are four different detection models over the same + * directories, so the module is a thin host and each one is a **subsystem** + * under `src/`: + * + * deps/ dependency and supply-chain scanning — PRD 0002, implemented + * secrets/ hardcoded credential detection — not yet built + * sast/ source-level vulnerability analysis — not yet built + * config/ misconfiguration checks — not yet built + * + * PRD 0002 originally proposed dependency scanning as a standalone + * `dep-scanner` module and left the boundary as an open question. It is + * resolved here in favour of folding in: two modules would have walked the + * same trees, held two inventories of the same packages, and given the + * operator two path lists to keep in sync — for a user-visible surface + * (`threatcrush scan --deps`) that was always meant to be one command. + * + * What survives the fold is the *internal* separation. `deps/` owns an + * external advisory database and a per-ecosystem parser surface that will keep + * growing; none of that belongs in a secrets regex pass. + * + * Scope of this release is Discover, Detect and Report for `deps/` only. + * Remediation is deliberately absent: upgrading a transitive dependency under + * a running production service is more likely to cause the outage than to + * prevent one (PRD 0002 Non-Goals). + */ + +import type { Alert, ModuleContext, ThreatCrushModule, ThreatEvent } from '@threatcrush/sdk'; + +import { + DEPS_DEFAULTS, + scanDependencies, + severityRank, + type DepsOptions, + type DepsResult, + type Finding, +} from './deps/index.js'; + +const STATE_REPORTED = 'reported_keys'; + +const DEFAULTS = { + scanIntervalSeconds: 6 * 60 * 60, + minSeverity: 'medium' as Finding['severity'], + maxAlerts: 25, +}; + +export interface ScanResult { + deps: DepsResult | null; + /** True when any subsystem could not complete — never report "clean". */ + incomplete: boolean; +} + +export default class CodeScannerModule implements ThreatCrushModule { + name = 'code-scanner'; + version = '0.1.0'; + description = + 'Static analysis on codebases — dependency advisories, malicious install scripts and lockfile drift'; + + private ctx!: ModuleContext; + private timer: NodeJS.Timeout | null = null; + private running = false; + + async init(ctx: ModuleContext): Promise { + this.ctx = ctx; + const enabled = [this.depsEnabled() ? 'deps' : null].filter(Boolean); + ctx.logger.info( + '[%s] initialized — subsystems: %s; paths: %s', + this.name, + enabled.join(', ') || 'none', + this.depsOptions().paths.join(', '), + ); + } + + async start(): Promise { + const interval = + (this.ctx.config.scan_interval_seconds as number | undefined) ?? DEFAULTS.scanIntervalSeconds; + + this.running = true; + this.ctx.logger.info('[%s] scanning every %ds', this.name, interval); + + void this.tick(); + this.timer = setInterval(() => void this.tick(), interval * 1000); + } + + async stop(): Promise { + this.running = false; + if (this.timer) { + clearInterval(this.timer); + this.timer = null; + } + this.ctx.logger.info('[%s] stopped', this.name); + } + + private async tick(): Promise { + if (!this.running) return; + + try { + this.report(await this.scan()); + } catch (err) { + // A scanner that dies quietly is worse than one that never ran: the + // operator keeps believing they are covered. + this.ctx.logger.error('[%s] scan failed: %s', this.name, String(err)); + this.ctx.emit( + this.event('scan', 'medium', `code-scanner: scan failed — ${String(err)}`, { + error: String(err), + }), + ); + } + } + + /** Run every enabled subsystem. */ + async scan(): Promise { + const deps = this.depsEnabled() ? await scanDependencies(this.depsOptions()) : null; + return { deps, incomplete: Boolean(deps?.incomplete) }; + } + + private depsEnabled(): boolean { + return this.ctx.config.deps_enabled !== false; + } + + private depsOptions(): DepsOptions { + const cfg = this.ctx.config; + return { + paths: this.paths(), + maxDepth: (cfg.max_depth as number | undefined) ?? DEPS_DEFAULTS.maxDepth, + advisories: cfg.deps_advisories !== false, + installScripts: cfg.deps_install_scripts !== false, + devDependencies: + (cfg.deps_dev_dependencies as DepsOptions['devDependencies'] | undefined) ?? + DEPS_DEFAULTS.devDependencies, + onDegraded: (reason) => { + this.ctx.logger.warn('[%s] advisory lookup failed: %s', this.name, reason); + this.ctx.emit( + this.event('scan', 'low', 'code-scanner: advisory lookup unavailable', { error: reason }), + ); + }, + }; + } + + private report(result: ScanResult): void { + const deps = result.deps; + if (!deps) return; + + const floor = severityRank( + (this.ctx.config.min_severity as Finding['severity'] | undefined) ?? DEFAULTS.minSeverity, + ); + const maxAlerts = (this.ctx.config.max_alerts as number | undefined) ?? DEFAULTS.maxAlerts; + const reported = new Set(this.readState(STATE_REPORTED, [])); + + const notable = deps.findings.filter((f) => severityRank(f.severity) >= floor); + const failed = deps.inventory.roots.filter((r) => r.status === 'failed'); + + // Deduplicate per (package, version, advisory) so a vulnerability present + // for 90 days alerts on discovery, not once per scan for three months. + for (const finding of notable + .filter((f) => !reported.has(`${f.package}@${f.version}:${f.id}`)) + .slice(0, maxAlerts)) { + const headline = `code-scanner: ${finding.package}@${finding.version} — ${finding.id}`; + this.ctx.emit(this.event('scan', finding.severity, headline, { ...finding })); + this.ctx.alert({ + title: headline, + severity: finding.severity, + body: [ + finding.summary, + finding.fixedIn ? `Fixed in ${finding.fixedIn}` : 'No fix available yet', + finding.url, + ].join('\n'), + } satisfies Alert); + reported.add(`${finding.package}@${finding.version}:${finding.id}`); + } + + for (const verdict of deps.scripts + .filter( + (v) => + v.severity !== 'info' && + !reported.has(`script:${v.script.packageName}@${v.script.version}:${v.script.stage}`), + ) + .slice(0, maxAlerts)) { + const { packageName, version, stage } = verdict.script; + const headline = `code-scanner: ${packageName}@${version} ${stage} script looks dangerous`; + this.ctx.emit( + this.event('scan', verdict.severity, headline, { + package: packageName, + version, + stage, + score: verdict.score, + command: verdict.script.command.slice(0, 400), + signals: verdict.signals, + }), + ); + this.ctx.alert({ + title: headline, + severity: verdict.severity, + body: verdict.signals.map((s) => `• ${s.message}`).join('\n'), + } satisfies Alert); + reported.add(`script:${packageName}@${version}:${stage}`); + } + + for (const entry of deps.drift + .filter((d) => !reported.has(`drift:${d.name}:${d.locked}->${d.installed}`)) + .slice(0, maxAlerts)) { + const headline = `code-scanner: ${entry.name} on disk is ${entry.installed}, lockfile says ${entry.locked}`; + this.ctx.emit(this.event('scan', 'high', headline, entry)); + this.ctx.alert({ + title: headline, + severity: 'high', + body: 'Installed bytes do not match the resolved lockfile — a manual hotfix, or tampering.', + } satisfies Alert); + reported.add(`drift:${entry.name}:${entry.locked}->${entry.installed}`); + } + + // The honesty requirement (PRD 0002 R6): an unexamined root must be louder + // than a quiet one, and must never be folded into a clean result. + if (failed.length > 0 && this.ctx.config.fail_on_unparseable !== false) { + const headline = `code-scanner: ${failed.length} project(s) could not be scanned — this is not a clean result`; + this.ctx.emit( + this.event('scan', 'medium', headline, { + roots: failed.map((r) => ({ root: r.root, reason: r.reason })), + }), + ); + this.ctx.alert({ + title: headline, + severity: 'medium', + body: failed.map((r) => `• ${r.root}: ${r.reason ?? 'unknown reason'}`).join('\n'), + } satisfies Alert); + } + + this.writeState(STATE_REPORTED, [...reported].slice(-2000)); + + this.ctx.logger.info( + '[%s] deps: %d packages across %d root(s) — %d advisories, %d risky scripts, %d drifted%s', + this.name, + deps.inventory.packages.length, + deps.inventory.roots.length, + notable.length, + deps.scripts.length, + deps.drift.length, + deps.incomplete ? ' — INCOMPLETE, some roots unreadable' : '', + ); + } + + private paths(): string[] { + const configured = this.ctx.config.paths; + if (Array.isArray(configured) && configured.length > 0) return configured as string[]; + if (typeof configured === 'string' && configured.trim()) { + return configured.split(',').map((p) => p.trim()); + } + return [...DEPS_DEFAULTS.paths]; + } + + private event( + category: ThreatEvent['category'], + severity: ThreatEvent['severity'], + message: string, + details: Record, + ): ThreatEvent { + return { timestamp: new Date(), module: this.name, category, severity, message, details }; + } + + private readState(key: string, fallback: T): T { + return (this.ctx.getState(key) as T) ?? fallback; + } + + private writeState(key: string, value: unknown): void { + this.ctx.setState(key, value); + } +} + +/** Human-readable one-shot summary, used by `threatcrush scan --deps`. */ +export function summarize(result: ScanResult): string { + const deps = result.deps; + if (!deps) return 'no subsystems enabled'; + + const bySeverity = new Map(); + for (const finding of deps.findings) { + bySeverity.set(finding.severity, (bySeverity.get(finding.severity) ?? 0) + 1); + } + + const lines = [ + `${deps.inventory.packages.length} packages across ${deps.inventory.roots.length} root(s)`, + `advisories: ${[...bySeverity.entries()].map(([s, n]) => `${n} ${s}`).join(', ') || 'none'}`, + `risky install scripts: ${deps.scripts.length}`, + `lockfile drift: ${deps.drift.length}`, + ]; + + const failed = deps.inventory.roots.filter((r) => r.status === 'failed'); + if (failed.length > 0) { + lines.push(''); + for (const root of failed) lines.push(` FAILED ${root.root} — ${root.reason}`); + lines.push('This result is not a clean bill of health.'); + } + + return lines.join('\n'); +} + +export * from './deps/index.js'; diff --git a/modules/dep-scanner/tsconfig.json b/modules/code-scanner/tsconfig.json similarity index 100% rename from modules/dep-scanner/tsconfig.json rename to modules/code-scanner/tsconfig.json diff --git a/modules/dep-scanner/mod.toml b/modules/dep-scanner/mod.toml deleted file mode 100644 index ae8706b..0000000 --- a/modules/dep-scanner/mod.toml +++ /dev/null @@ -1,64 +0,0 @@ -[module] -name = "dep-scanner" -version = "0.1.0" -description = "Detects vulnerable and malicious dependencies on running servers — advisories, install-script behaviour, and lockfile drift" -author = "Profullstack, Inc." -license = "MIT" -homepage = "https://github.com/profullstack/threatcrush/tree/master/modules/dep-scanner" - -[module.pricing] -type = "free" - -[module.requirements] -threatcrush = ">=0.2.0" -os = ["linux", "darwin"] -capabilities = ["fs:read", "network:outbound"] - -[module.config.defaults] -enabled = true - -# Where to look for projects. Each path is walked for directories containing a -# package.json, skipping node_modules and VCS directories. -paths = ["/srv", "/var/www", "/opt"] - -# How deep to walk below each configured path when looking for project roots. -max_depth = 4 - -# Dependencies change on deploy, not on the minute. Six hours keeps advisory -# data fresh without turning the scanner into background load. -scan_interval_seconds = 21600 - -# --- Honesty --------------------------------------------------------------- -# Alert when a project could not be parsed at all. Leave this on: the failure -# this module was written to prevent is a scanner reporting "nothing found" -# when the truth is "nothing was read". An unexamined root is not a clean one. -fail_on_unparseable = true - -# --- Advisories ------------------------------------------------------------ -# Look up installed versions against OSV.dev (aggregates GHSA, PyPA, RustSec, -# Go vulndb). Set false for a fully offline host; install-script and drift -# detection continue to work without network access. -advisories_enabled = true - -# Alerting floor. Findings below this are still counted and logged. -min_severity = "medium" - -# How to treat findings reachable only through devDependencies: -# rank_down demote one severity level (default) -# equal treat identically to runtime dependencies -# There is deliberately no "ignore": a finding an operator never sees is -# indistinguishable from one that was missed. -dev_dependencies = "rank_down" - -# --- Behavioural detection ------------------------------------------------- -# Score preinstall/install/postinstall scripts on what they do — fetching -# remote code, decoding blobs, reading credential paths. This is what catches a -# malicious package before an advisory exists, which is the window in which -# event-stream, ua-parser-js and node-ipc all did their damage. -install_scripts = true - -# --- Reporting ------------------------------------------------------------- -# Cap alerts per scan so a first run on a neglected host summarizes instead of -# flooding the alert channel. Findings beyond the cap are logged and carried to -# the next scan. -max_alerts = 25 diff --git a/modules/dep-scanner/src/index.ts b/modules/dep-scanner/src/index.ts deleted file mode 100644 index fd35351..0000000 --- a/modules/dep-scanner/src/index.ts +++ /dev/null @@ -1,338 +0,0 @@ -/** - * dep-scanner — detect vulnerable and malicious dependencies on running - * servers. - * - * Implements PRD 0002. Motivated by two things: - * - * - A general-purpose analyzer reported "0 dependencies, coupling 0/100 - * (good)" for a 13-package pnpm workspace, because it never resolved the - * workspace globs. A productivity tool that does that wastes an afternoon; a - * security tool that does it issues a clean bill of health without an - * examination. - * - `event-stream`, `ua-parser-js` and `node-ipc` were all advisory-clean at - * the moment they were installed. Matching versions against a vulnerability - * database cannot catch a package that is malicious before anyone has - * written the advisory — so this module also reads what install scripts - * actually *do*. - * - * Scope of this release is Discover, Detect and Report. Remediation is - * deliberately absent: upgrading a transitive dependency under a running - * production service is more likely to cause the outage than to prevent one - * (PRD 0002 Non-Goals). See README, "What this does not do". - */ - -import { readFile } from 'node:fs/promises'; -import { join } from 'node:path'; - -import type { Alert, ModuleContext, ThreatCrushModule, ThreatEvent } from '@threatcrush/sdk'; - -import { buildInventory, type Inventory, type RootReport } from './inventory.js'; -import { queryAdvisories, severityRank, type Finding } from './osv.js'; -import { extractScripts, rankScripts, type ScriptVerdict } from './scripts.js'; - -const STATE_REPORTED = 'reported_keys'; - -const DEFAULTS = { - paths: ['/srv', '/var/www', '/opt'], - scanIntervalSeconds: 6 * 60 * 60, - minSeverity: 'medium' as Finding['severity'], - maxAlerts: 25, - maxDepth: 4, -}; - -export interface ScanResult { - inventory: Inventory; - findings: Finding[]; - scripts: ScriptVerdict[]; - /** True when any root failed to parse — never report "clean" if set. */ - incomplete: boolean; -} - -export default class DepScannerModule implements ThreatCrushModule { - name = 'dep-scanner'; - version = '0.1.0'; - description = - 'Scans installed dependencies for known advisories, malicious install scripts and lockfile drift'; - - private ctx!: ModuleContext; - private timer: NodeJS.Timeout | null = null; - private running = false; - - async init(ctx: ModuleContext): Promise { - this.ctx = ctx; - const paths = this.paths(); - ctx.logger.info('[%s] initialized — scanning %s', this.name, paths.join(', ')); - } - - async start(): Promise { - const interval = - (this.ctx.config.scan_interval_seconds as number | undefined) ?? DEFAULTS.scanIntervalSeconds; - - this.running = true; - this.ctx.logger.info('[%s] scanning every %ds', this.name, interval); - - void this.tick(); - this.timer = setInterval(() => void this.tick(), interval * 1000); - } - - async stop(): Promise { - this.running = false; - if (this.timer) { - clearInterval(this.timer); - this.timer = null; - } - this.ctx.logger.info('[%s] stopped', this.name); - } - - private async tick(): Promise { - if (!this.running) return; - - try { - const result = await this.scan(); - this.report(result); - } catch (err) { - // A scanner that dies quietly is worse than one that never ran: the - // operator keeps believing they are covered. - this.ctx.logger.error('[%s] scan failed: %s', this.name, String(err)); - this.ctx.emit( - this.event('scan', 'medium', `dep-scanner: scan failed — ${String(err)}`, { - error: String(err), - }), - ); - } - } - - /** Inventory, advisory lookup and script scoring for the configured paths. */ - async scan(): Promise { - const inventory = await buildInventory( - this.paths(), - (this.ctx.config.max_depth as number | undefined) ?? DEFAULTS.maxDepth, - ); - - const scripts = await this.scanInstallScripts(inventory); - - let findings: Finding[] = []; - if (inventory.packages.length > 0 && this.ctx.config.advisories_enabled !== false) { - try { - findings = await queryAdvisories( - inventory.packages.map((p) => ({ name: p.name, version: p.version })), - ); - } catch (err) { - // Advisory lookup is the one part that needs the network. Losing it - // must degrade the scan, not void it — and must be visible, because a - // silent skip looks identical to "nothing found". - this.ctx.logger.warn('[%s] advisory lookup failed: %s', this.name, String(err)); - this.ctx.emit( - this.event('scan', 'low', 'dep-scanner: advisory lookup unavailable', { - error: String(err), - }), - ); - inventory.incomplete = true; - } - } - - // Dev-only findings are ranked down, never dropped (PRD 0002 R9): a - // suppressed finding the operator never sees is indistinguishable from one - // that was missed. - const devOnly = new Set( - inventory.packages.filter((p) => p.dev).map((p) => `${p.name}@${p.version}`), - ); - if (this.ctx.config.dev_dependencies !== 'equal') { - findings = findings.map((f) => - devOnly.has(`${f.package}@${f.version}`) && severityRank(f.severity) > 1 - ? { ...f, severity: demote(f.severity) } - : f, - ); - } - - return { inventory, findings, scripts, incomplete: inventory.incomplete }; - } - - /** Read lifecycle scripts out of every installed package and score them. */ - private async scanInstallScripts(inventory: Inventory): Promise { - if (this.ctx.config.install_scripts === false) return []; - - const scripts = []; - for (const report of inventory.roots) { - for (const record of report.packages) { - if (!record.path.startsWith('node_modules')) continue; - try { - const manifest = JSON.parse( - await readFile(join(report.root, record.path, 'package.json'), 'utf8'), - ) as { name?: string; version?: string; scripts?: Record }; - scripts.push(...extractScripts(manifest, record.name)); - } catch { - // A package whose manifest cannot be read contributes no scripts; - // the root's own status already records parse trouble. - } - } - } - - return rankScripts(scripts); - } - - private report(result: ScanResult): void { - const floor = severityRank( - (this.ctx.config.min_severity as Finding['severity'] | undefined) ?? DEFAULTS.minSeverity, - ); - const reported = new Set(this.readState(STATE_REPORTED, [])); - const maxAlerts = (this.ctx.config.max_alerts as number | undefined) ?? DEFAULTS.maxAlerts; - - const notable = result.findings.filter((f) => severityRank(f.severity) >= floor); - const drift = result.inventory.roots.flatMap((r) => - r.drift.map((d) => ({ root: r.root, ...d })), - ); - const failed = result.inventory.roots.filter((r) => r.status === 'failed'); - - // Deduplicate per (package, version, advisory) so a vulnerability present - // for 90 days alerts on discovery, not 360 times. - const fresh = notable.filter((f) => !reported.has(`${f.package}@${f.version}:${f.id}`)); - const freshScripts = result.scripts.filter( - (v) => - v.severity !== 'info' && - !reported.has(`script:${v.script.packageName}@${v.script.version}:${v.script.stage}`), - ); - const freshDrift = drift.filter( - (d) => !reported.has(`drift:${d.name}:${d.locked}->${d.installed}`), - ); - - for (const finding of fresh.slice(0, maxAlerts)) { - const headline = `dep-scanner: ${finding.package}@${finding.version} — ${finding.id}`; - this.ctx.emit(this.event('scan', finding.severity, headline, { ...finding })); - this.ctx.alert({ - title: headline, - severity: finding.severity, - body: [ - finding.summary, - finding.fixedIn ? `Fixed in ${finding.fixedIn}` : 'No fix available yet', - finding.url, - ].join('\n'), - } satisfies Alert); - reported.add(`${finding.package}@${finding.version}:${finding.id}`); - } - - for (const verdict of freshScripts.slice(0, maxAlerts)) { - const { packageName, version, stage } = verdict.script; - const headline = `dep-scanner: ${packageName}@${version} ${stage} script looks dangerous`; - this.ctx.emit( - this.event('scan', verdict.severity, headline, { - package: packageName, - version, - stage, - score: verdict.score, - command: verdict.script.command.slice(0, 400), - signals: verdict.signals, - }), - ); - this.ctx.alert({ - title: headline, - severity: verdict.severity, - body: verdict.signals.map((s) => `• ${s.message}`).join('\n'), - } satisfies Alert); - reported.add(`script:${packageName}@${version}:${stage}`); - } - - for (const entry of freshDrift.slice(0, maxAlerts)) { - const headline = `dep-scanner: ${entry.name} on disk is ${entry.installed}, lockfile says ${entry.locked}`; - this.ctx.emit(this.event('scan', 'high', headline, entry)); - this.ctx.alert({ - title: headline, - severity: 'high', - body: 'Installed bytes do not match the resolved lockfile — a manual hotfix, or tampering.', - } satisfies Alert); - reported.add(`drift:${entry.name}:${entry.locked}->${entry.installed}`); - } - - // The honesty requirement (PRD 0002 R6): an unexamined root must be louder - // than a quiet one, and must never be folded into a clean result. - if (failed.length > 0 && this.ctx.config.fail_on_unparseable !== false) { - const headline = `dep-scanner: ${failed.length} project(s) could not be scanned — this is not a clean result`; - this.ctx.emit( - this.event('scan', 'medium', headline, { - roots: failed.map((r) => ({ root: r.root, reason: r.reason })), - }), - ); - this.ctx.alert({ - title: headline, - severity: 'medium', - body: failed.map((r) => `• ${r.root}: ${r.reason ?? 'unknown reason'}`).join('\n'), - } satisfies Alert); - } - - this.writeState(STATE_REPORTED, [...reported].slice(-2000)); - - this.ctx.logger.info( - '[%s] %d packages across %d root(s): %d advisories, %d risky scripts, %d drifted%s', - this.name, - result.inventory.packages.length, - result.inventory.roots.length, - notable.length, - result.scripts.length, - drift.length, - result.incomplete ? ' — INCOMPLETE, some roots unreadable' : '', - ); - } - - private paths(): string[] { - const configured = this.ctx.config.paths; - if (Array.isArray(configured) && configured.length > 0) return configured as string[]; - if (typeof configured === 'string' && configured.trim()) { - return configured.split(',').map((p) => p.trim()); - } - return DEFAULTS.paths; - } - - private event( - category: ThreatEvent['category'], - severity: ThreatEvent['severity'], - message: string, - details: Record, - ): ThreatEvent { - return { timestamp: new Date(), module: this.name, category, severity, message, details }; - } - - private readState(key: string, fallback: T): T { - return (this.ctx.getState(key) as T) ?? fallback; - } - - private writeState(key: string, value: unknown): void { - this.ctx.setState(key, value); - } -} - -/** One step down the severity ladder, floored at low. */ -export function demote(severity: Finding['severity']): Finding['severity'] { - const order: Finding['severity'][] = ['info', 'low', 'medium', 'high', 'critical']; - return order[Math.max(1, order.indexOf(severity) - 1)]!; -} - -/** Human-readable one-shot summary, used by `threatcrush scan --deps`. */ -export function summarize(result: ScanResult): string { - const lines: string[] = []; - const bySeverity = new Map(); - for (const finding of result.findings) { - bySeverity.set(finding.severity, (bySeverity.get(finding.severity) ?? 0) + 1); - } - - lines.push( - `${result.inventory.packages.length} packages across ${result.inventory.roots.length} root(s)`, - ); - lines.push( - `advisories: ${[...bySeverity.entries()].map(([s, n]) => `${n} ${s}`).join(', ') || 'none'}`, - ); - lines.push(`risky install scripts: ${result.scripts.length}`); - - const failed = result.inventory.roots.filter((r: RootReport) => r.status === 'failed'); - if (failed.length > 0) { - lines.push(''); - for (const root of failed) lines.push(` FAILED ${root.root} — ${root.reason}`); - lines.push('This result is not a clean bill of health.'); - } - - return lines.join('\n'); -} - -export * from './inventory.js'; -export * from './osv.js'; -export * from './scripts.js'; -export * from './semver.js'; diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index eca5857..620bb53 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -393,6 +393,22 @@ importers: specifier: ^2.1.9 version: 2.1.9(@types/node@20.19.39)(jsdom@29.1.1)(lightningcss@1.32.0)(terser@5.46.1) + modules/code-scanner: + dependencies: + '@threatcrush/sdk': + specifier: workspace:* + version: link:../../apps/sdk + devDependencies: + '@types/node': + specifier: ^22.0.0 + version: 22.19.17 + typescript: + specifier: ^5.9.3 + version: 5.9.3 + vitest: + specifier: ^3.0.0 + version: 3.2.7(@types/debug@4.1.13)(@types/node@22.19.17)(jiti@2.6.1)(jsdom@29.1.1)(lightningcss@1.32.0)(terser@5.46.1)(yaml@2.8.3) + modules/dep-scanner: dependencies: '@threatcrush/sdk': diff --git a/prd/0002-detect-vulnerable-and-malicious-dependencies-on-running-servers.md b/prd/0002-detect-vulnerable-and-malicious-dependencies-on-running-servers.md index e8594d5..13d2fde 100644 --- a/prd/0002-detect-vulnerable-and-malicious-dependencies-on-running-servers.md +++ b/prd/0002-detect-vulnerable-and-malicious-dependencies-on-running-servers.md @@ -9,8 +9,8 @@ created: 2026-07-28 updated: 2026-07-28 repo: profullstack/threatcrush discussion: -implementation: -tags: dep-scanner, supply-chain, sbom, cve, osv, install-scripts, drift, code-scanner, modules +implementation: modules/code-scanner (deps subsystem) +tags: code-scanner, deps, supply-chain, sbom, cve, osv, install-scripts, drift, supply-chain-scanning, modules supersedes: superseded-by: --- @@ -101,8 +101,9 @@ gives it three signals nobody in CI has: against a repository; this inspects a running host. Complementary, not competing — and this PRD does not propose ThreatCrush open PRs. - **Not a general SAST engine.** Source-level vulnerability analysis, secret - detection and misconfiguration scanning stay with `code-scanner`. This module - owns the dependency graph only (see the boundary question in Risks). + detection and misconfiguration scanning are `code-scanner`'s other subsystems + (`sast/`, `secrets/`, `config/`) and are out of scope for this PRD. This work + owns the dependency graph only — a package inventory, not source. - **Not a license-compliance tool.** SBOM output must be reusable for that, but license policy enforcement is out of scope. - **Not automatic remediation in v1.** No auto-upgrading, no auto-`npm install`, @@ -210,47 +211,50 @@ gives it three signals nobody in CI has: ## UX Notes -Ships as a core module, `dep-scanner`, following the existing module contract -(`mod.toml`, config in `threatcrushd.conf.d/dep-scanner.conf`, alerts routed -through `alert-system`), matching `spend-guard`. +Ships as the `deps` subsystem of the core `code-scanner` module, following the +existing module contract (`mod.toml`, config in +`threatcrushd.conf.d/code-scanner.conf`, alerts routed through `alert-system`), +matching `spend-guard`. Dependency-specific settings are namespaced `deps_*` so +the sibling subsystems — `secrets`, `sast`, `config` — can be configured +alongside without collision, while `paths`, `min_severity` and the scan interval +stay shared. ```bash threatcrush scan --deps # one-shot, current directory threatcrush scan --deps /srv/app --json # machine-readable threatcrush scan --deps --fail-on high # CI gate (non-zero exit) -threatcrush dep-scanner inventory # what is installed, per root -threatcrush dep-scanner scripts # every install script in the tree -threatcrush dep-scanner drift # changes since last scan -threatcrush dep-scanner sbom --format cyclonedx > sbom.json -threatcrush dep-scanner sync # refresh the advisory database -threatcrush dep-scanner why lodash # who pulls this in, and is it runtime +threatcrush code-scanner deps inventory # what is installed, per root +threatcrush code-scanner deps scripts # every install script in the tree +threatcrush code-scanner deps drift # changes since last scan +threatcrush code-scanner deps sbom --format cyclonedx > sbom.json +threatcrush code-scanner deps sync # refresh the advisory database +threatcrush code-scanner deps why lodash # who pulls this in, and is it runtime ``` Config sketch: ```toml -[dep-scanner] +[code-scanner] enabled = true paths = ["/srv", "/var/www", "/opt/app"] scan_interval = "6h" +min_severity = "medium" # alerting floor, shared by every subsystem # Never report a pass for a root that could not be parsed. fail_on_unparseable = true -[dep-scanner.advisories] -source = "osv" +[code-scanner.deps] +enabled = true +advisories = "osv" sync_interval = "12h" -offline_ok = true # match against last sync rather than skipping - -[dep-scanner.detect] -min_severity = "medium" # alerting floor -dev_dependencies = "rank_down" # rank_down | ignore | equal +offline_ok = true # match against last sync rather than skipping +dev_dependencies = "rank_down" # rank_down | equal — deliberately no "ignore" install_scripts = true integrity_mismatch = "critical" drift = "high" reputation_heuristics = true -[dep-scanner.report] +[code-scanner.report] dedupe_window = "30d" max_alerts_per_scan = 25 # summarize beyond this rather than flooding ``` @@ -259,7 +263,7 @@ An alert must lead with exposure and remediation, and must state its own blind spots: ``` -[HIGH] dep-scanner · /srv/qrypt-chat · 3 runtime advisories, 1 integrity mismatch +[HIGH] code-scanner · /srv/qrypt-chat · 3 runtime advisories, 1 integrity mismatch CRITICAL integrity mismatch ua-parser-js@0.7.29 on-disk hash != lockfile integrity — installed bytes are not the @@ -309,15 +313,19 @@ Design constraints: ## Risks & Open Questions -- **Module boundary with `code-scanner` is unresolved.** `PRD.md` assigns - "dependency CVEs" to `code-scanner`, and this PRD proposes a separate - `dep-scanner`. The argument for splitting: different data model (a package - graph, not source), different cadence (scheduled and on-change, not one-shot), - a large external database dependency, and a maintenance surface per ecosystem - that would dominate `code-scanner`. **Open:** split as proposed and amend - `PRD.md`, or ship as a `code-scanner` subsystem sharing the `scan` CLI? The - CLI (`scan --deps`) should look identical to the user either way, so this is - an internal boundary decision that should not leak. +- ~~**Module boundary with `code-scanner` is unresolved.**~~ **Resolved + 2026-07-28: folded into `code-scanner` as the `deps/` subsystem.** Two + modules would have walked the same trees, held two inventories of the same + packages, and given the operator two path lists to keep in sync — for a + user-visible surface (`scan --deps`) that was always meant to be one command. + `PRD.md` already assigned "dependency CVEs" to `code-scanner`, so folding in + makes the module tree match the product doc instead of contradicting it, and + no amendment is needed. The arguments for splitting (a package-graph data + model, an external advisory database, a per-ecosystem parser surface that will + keep growing) were real but are all *internal*, so they are preserved as a + subsystem boundary under `src/deps/` rather than a module boundary. Siblings + `secrets/`, `sast/` and `config/` are stubs for the rest of `code-scanner`'s + remit. - **Reachability analysis is where scanners lose credibility in both directions.** Too eager and it suppresses a real finding; too timid and the operator drowns. v1 uses it only to *rank* (R9), never to suppress — but that @@ -342,7 +350,7 @@ Design constraints: - **Drift detection (R14) needs a deploy signal it does not own.** Without knowing when a legitimate deploy happened, every deploy looks like drift. **Open:** infer from process restart and mtime clustering, or require an - explicit `threatcrush dep-scanner ack-deploy` hook in the user's deploy + explicit `threatcrush code-scanner deps ack-deploy` hook in the user's deploy script? The latter is accurate and adds integration burden. - **This module reads dependency trees, which frequently contain credentials in adjacent files** (`.npmrc`, `.env`). It must never log or transmit them, and diff --git a/prd/README.md b/prd/README.md index 7442b7c..545a412 100644 --- a/prd/README.md +++ b/prd/README.md @@ -14,4 +14,4 @@ these numbered PRDs cover individual changes to it. | ID | Title | Status | Tags | | --- | --- | --- | --- | | [0001](./0001-detect-and-contain-balance-drain-attacks-on-third-party-services.md) | Detect and contain balance-drain attacks on third-party services | Draft | spend-guard, billing, fraud, sms-pumping, irsf, auto-recharge, containment, modules | -| [0002](./0002-detect-vulnerable-and-malicious-dependencies-on-running-servers.md) | Detect vulnerable and malicious dependencies on running servers | Draft | dep-scanner, supply-chain, sbom, cve, osv, install-scripts, drift, code-scanner, modules | +| [0002](./0002-detect-vulnerable-and-malicious-dependencies-on-running-servers.md) | Detect vulnerable and malicious dependencies on running servers | Draft | code-scanner, deps, supply-chain, sbom, cve, osv, install-scripts, drift, modules |