Skip to content

Latest commit

 

History

History
420 lines (326 loc) · 14.4 KB

File metadata and controls

420 lines (326 loc) · 14.4 KB

Other integrations

Recipes for hook managers, containers, Rust embedding, and notifications. The dedicated pre-commit and CI guides own those workflows. Their headings remain below as stable entry points, not duplicate specifications.

Install the release with the verified installer, which records the host's autoroute evidence. A source-built multi-backend binary used outside the GitHub Action must run keyhog calibrate-autoroute before its first automatic scan. A portable single-backend build has no routing choice. The lightweight local-hook recipes use an explicit --backend cpu so they do not depend on machine-local routing state.

For the full contract behind a command, use the focused reference instead of treating a copied snippet as a second specification:

Task Start here
Protect local commits keyhog hook install
Gate a pull request CI integration
Scan a large tree or choose a policy Detection settings and hardware
Suppress an accepted finding Suppressions
Interpret a failure Exit codes

If you only need one section, jump to:

Pre-commit hook (Git)

Use the canonical pre-commit guide for installation, hook ownership, staged-content semantics, bypass auditing, performance, and removal.

Pre-push hook (Git)

Pre-commit is the fastest local gate. A pre-push history scan also finds a credential introduced by an earlier commit on the checked-out branch. Save this as .git/hooks/pre-push and make it executable:

#!/usr/bin/env bash
set -euo pipefail

keyhog scan --git-history . --backend cpu

This scans added lines across all commits reachable from local HEAD. It does not depend on the remote name, upstream branch, or network access. It is broader and slower than a staged scan. KeyHog's nonzero status is returned unchanged, so findings and incomplete scans both block the push. CI remains the authoritative gate because git push --no-verify bypasses local pre-push hooks.

pre-commit framework

The pre-commit framework recipe lives with the raw Git hook workflow so both installation paths share one behavioral contract.

Husky / lefthook

Husky (.husky/pre-commit)

#!/usr/bin/env sh
. "$(dirname -- "$0")/_/husky.sh"

keyhog scan --fast --git-staged --backend cpu

Lefthook (lefthook.yml)

pre-commit:
  parallel: true
  commands:
    keyhog:
      run: keyhog scan --fast --git-staged --backend cpu
      fail_text: "secrets detected - see output above"

GitHub Actions

Use the GitHub Action guide for the composite Action, inputs and outputs, baseline adoption, monorepo partitions, SARIF publication, and failure behavior. Use the CI guide when a GitHub workflow needs direct CLI flags such as --git-history or --git-blobs.

GitLab CI

Use the canonical GitLab CI workflow. It owns installation, GitLab SAST output, artifact retention, and exit semantics.

CircleCI

Use the canonical CircleCI workflow. It owns shell setup, scan status, and artifact handling.

Drone CI

Use the canonical Drone workflow. For another CI runner, use the generic shell workflow.

Buildkite

Use the canonical Buildkite workflow.

Docker / Docker Compose

Scan a repo from a one-shot container without installing anything on the host:

# No published registry image yet - build once from the repo (the Dockerfile
# ships in the repo root), then run the scan:
docker build -t keyhog:local https://github.com/santhreal/keyhog.git
docker run --rm -v "$PWD":/src keyhog:local \
  scan /src --backend cpu --format text

docker-compose.yml:

services:
  keyhog:
    build: https://github.com/santhreal/keyhog.git
    volumes:
      - ./:/src:ro
    command: scan /src --backend cpu --format json-envelope

To scan a built image, use the Docker/OCI source so layers, manifests, and source coverage are handled by KeyHog instead of manually unpacking an archive:

keyhog scan --docker-image my-image:latest

Jenkins

Use the canonical Jenkins workflow.

As a library (Rust)

Add to Cargo.toml:

[dependencies]
keyhog-core = "0.5"        # detector specs + Chunk/ChunkMetadata
keyhog-scanner = "0.5"     # CompiledScanner

(Detectors ship inside keyhog-core as a static-embedded TOML corpus; there is no separate keyhog-detectors crate.)

Minimal scan:

use keyhog_core::{Chunk, ChunkMetadata, RawMatch};
use keyhog_scanner::CompiledScanner;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Built-in embedded detectors - no disk I/O, fail-closed on corrupt bundled TOML.
    let specs = keyhog_core::load_embedded_detectors_or_fail()?;
    // …or load from a directory of TOMLs:
    // let specs = load_detectors(std::path::Path::new("detectors"))?;

    let scanner = CompiledScanner::compile(specs)?;

    let bytes = std::fs::read("config.yaml")?;
    let chunk = Chunk {
        data: String::from_utf8_lossy(&bytes).into_owned().into(),
        metadata: ChunkMetadata {
            source_type: "filesystem".into(),
            path: Some("config.yaml".into()),
            ..Default::default()
        },
    };
    let matches = scanner.scan(&chunk)?;
    for m in &matches {
        println!(
            "{}:{} (detector {})",
            m.location.file_path.as_deref().unwrap_or("<memory>"),
            m.location.line.unwrap_or(0),
            m.detector_id
        );
    }
    // RawMatch stays in process; this projection is safe to serialize or report.
    let _report_safe: Vec<_> = matches.iter().map(RawMatch::to_redacted).collect();
    Ok(())
}

For directory-tree / git / docker walking, drive keyhog-sources or shell out to the CLI - CompiledScanner is one chunk at a time by design.

The no-backend scan and scan_coalesced methods are deterministic portable CPU calls. Explicit scan_with_backend and scan_coalesced_with_backend calls return typed ScanError values when a selected backend cannot initialize or finish. They never terminate the embedding process and never substitute a different engine. You can probe startup eligibility with warm_backend; the CLI owns the separate mapping from terminal scanner errors to process exit status.

Successful calls return Vec<RawMatch> inside the typed Result. Credential, SensitiveString, raw or deduplicated matches, and source Chunk values can contain plaintext or encoded secret bytes and therefore refuse implicit serde output. Convert raw matches with RawMatch::to_redacted, or emit the verification pipeline's VerifiedFinding, before JSON, logging, disk, or network output. Only a protected private protocol should explicitly reveal secret bytes.

For finer-grained control of individual detector features:

[dependencies]
keyhog-scanner = { version = "0.5", default-features = false, features = ["ml", "decode", "entropy"] }

Embedded in another CLI

Shell out:

use std::process::Command;
let out = Command::new("keyhog")
    .args(["scan", "--format", "jsonl-envelope", "--min-confidence", "0.4", "."])
    .output()?;
if !matches!(out.status.code(), Some(0 | 1)) {
    return Err(std::io::Error::other(format!(
        "keyhog did not complete the requested scan: {}",
        String::from_utf8_lossy(&out.stderr)
    )).into());
}
for line in out.stdout.split(|b| *b == b'\n') {
    if line.is_empty() { continue; }
    let record: serde_json::Value = serde_json::from_slice(line)?;
    if matches!(record.get("record_type").and_then(|v| v.as_str()), Some("header" | "summary")) {
        continue;
    }
    let finding = record;
    // ... do whatever
}

Or invoke the scan subcommand directly from a wrapper script:

keyhog scan /path/to/project --format jsonl-envelope --min-confidence 0.4

SARIF for GitHub Code Scanning

The composite Action is the safest way to create, upload, and retain SARIF:

- uses: santhreal/keyhog@v0
  with:
    format: sarif
    upload-sarif: 'true'
    fail-on-findings: 'true'

Grant security-events: write as shown in the GitHub Action guide. The Action uploads before it enforces findings, keeps a workflow artifact, and makes only a fork pull request's restricted-token upload advisory. Trusted upload failures fail the job.

For another SARIF consumer, write the file directly:

keyhog scan . --format sarif --output keyhog.sarif

The command exits 1 on findings and 10 on a verified-live finding. Arrange report publication in an always-run or post step, then restore the exact scan status. KeyHog tags findings with CWE-798 and OWASP A07:2021.

Slack / Discord / webhook alerts

Post a one-line summary on every finding:

#!/usr/bin/env bash
set -euo pipefail
set +e
findings_json="$(keyhog scan . --format json-envelope --min-confidence 0.4)"
scan_status=$?
set -e
case "$scan_status" in
  0|1) ;;
  *) echo "keyhog scan did not complete (exit $scan_status)" >&2; exit "$scan_status" ;;
esac
count="$(echo "$findings_json" | jq '.findings | length')"
if [ "$count" -gt 0 ]; then
  curl -X POST -H 'Content-type: application/json' \
    --data "{\"text\":\"⚠ keyhog: $count secret(s) detected in $(basename "$PWD")\"}" \
    "$SLACK_WEBHOOK_URL"
  exit 1
fi
exit "$scan_status"

For Discord, replace text with content. For PagerDuty, use the events/v2/enqueue endpoint with severity critical for --severity critical findings.

Allowlists and baselines

When you have known-but-unfixable findings (rotated test keys, public demo creds, fixtures), use a baseline:

# Once
keyhog scan . --create-baseline .keyhog-baseline.json

# Forever after
keyhog scan . --baseline .keyhog-baseline.json

Baseline JSON is strict: unknown root or entry fields fail closed instead of silently changing suppression policy. The legacy v1 entry status field is accepted only for compatibility and is never serialized or used as a policy decision. Review baseline edits like code and regenerate them with --create-baseline when the identity set is intentionally changed.

For per-file/per-line allowlists, the moving parts live in two separate files. Scan execution policy has one canonical [scan] owner; unknown tables and retired flat spellings fail closed:

.keyhog.toml at the repo root:

[scan]
severity       = "high"
min_confidence = 0.4
threads        = 8
exclude        = ["vendor/**", "node_modules/**", "**/*.lock"]

.keyhogignore (or .keyhogignore.toml) alongside it - gitignore- style path globs plus detector:<id> and hash:<sha256> entries:

# silence all hits from this detector
detector:http-basic-auth

# gitignore-style path globs
vendor/**
node_modules/**
**/*.lock

See the .keyhogignore.toml reference for the full schema.

Exit codes

Use the canonical exit-code reference for the full numeric contract. In CI, findings and verified-live credentials block the change; configuration, system, backend, incomplete-coverage, panic, and interruption outcomes also fail the job because the requested security control did not complete. Never normalize every nonzero result to “findings found.”


Choose a scan policy for scale

# Lightweight staged-content check; independent of host autoroute state
keyhog scan --fast --git-staged --backend cpu

# Deep release/security gate; uses calibrated automatic routing
keyhog scan . --deep --severity high

# High-precision policy for a large tree where false-positive review dominates
keyhog scan /large/tree --precision --severity high

# Force GPU for a diagnostic/benchmark run
keyhog scan . --backend gpu-wgpu

# Write the versioned JSONL stream to a file
keyhog scan . --format jsonl-envelope --output findings.jsonl

--fast, --deep, and --precision intentionally resolve different detection policies and can produce different findings. Hardware and automatic backend selection must not. Measure the chosen policy on the real corpus and let persisted calibration choose among every measured-correct backend for that exact host and workload. See Configuration presets and Backends and routing before changing policy or forcing an engine.

Troubleshooting

Symptom Likely cause Fix
Exit 12 with a selected-GPU diagnostic Required, explicit, or calibration GPU execution could not start or complete Run keyhog backend --self-test, repair the GPU stack, and recalibrate; normal automatic runtime faults instead produce a visible complete-after-recovery receipt when the stable bytes can be replayed
Findings count drops vs prior run Baseline, detector corpus, scan policy, or .keyhog.toml changed Compare the effective config, detector digest, baseline, and input scope from both runs
Pre-commit hook is slow Scanning the whole repo on every commit Use --git-staged not scan .
SARIF report is too large for the consumer The selected scope produced more findings than the consumer accepts Narrow the scanned source, use a reviewed baseline, or choose an explicit severity policy; do not hide an incomplete upload
Detection misses a known token Detector absent from the loaded corpus / --fast disabled decode recursion or entropy discovery Re-run with the embedded corpus and --deep; file an issue if it still misses