Skip to content

fix: RFC 3161 signature verification, telemetry privacy, scan-path safety - #30

Merged
kuzivaai merged 9 commits into
mainfrom
fix/timestamp-verification-and-telemetry-privacy
Jul 21, 2026
Merged

fix: RFC 3161 signature verification, telemetry privacy, scan-path safety#30
kuzivaai merged 9 commits into
mainfrom
fix/timestamp-verification-and-telemetry-privacy

Conversation

@kuzivaai

Copy link
Copy Markdown
Owner

Three independent defect classes, plus the documentation they falsified. All 8 CI gates verified green locally on the final tree.

RFC 3161 — verify the token, don't just trust it

regula verify previously checked only that the messageImprint matched; the token's own signature was never evaluated. It now verifies the PKCS#7 SignedData signature (RFC 5652 §5.4, signedAttrs re-encoded as SET OF), confirms the signed attributes bind it to that exact TSTInfo, and requires the critical id-kp-timeStamping EKU (RFC 3161 §2.3).

Reports the strongest status actually provenHASH_MATCHED / SIGNATURE_VERIFIED / CHAIN_VERIFIED — and never a bare VERIFIED. New --tsa-trust-anchor chains to a caller-supplied anchor, documented as a LIMITED check: no revocation (CRL/OCSP), no name constraints, no intermediate chain building.

Validated against a live FreeTSA token (ecdsa/sha512), which exercises a different algorithm path from the EC-P256/SHA-256 fixture. A self-built fixture can only confirm our own reading of the spec; an independent implementation's signature verifying is real evidence.

Also fixes a defect in that implementation: algorithm-dispatch failures and genuine verification failures shared one error channel, so an unimplemented algorithm (Ed25519, say) surfaced as INVALID and hard-failed a valid pack. Those now degrade to UNSUPPORTED and retain the hash-only verdict — which the spec in this repo already required. Provably-bad signatures still return INVALID.

Telemetry — four privacy defects

# Defect Resolution
1 _SENTRY_DSN hardcoded to a live endpoint since 43da24c (10 Apr 2026) while docs/TRUST.md §8.2 told buyers published builds ship an empty one Read from REGULA_SENTRY_DSN, defaults empty
2 include_local_variables unset → sentry-sdk default True Explicitly False; hostname redacted
3 REGULA_NO_TELEMETRY gated only the first-run prompt, not sending Enforced at send time
4 DO_NOT_TRACK not honoured Added, per consoledonottrack.com

On (1): verified against the published artifact, not just the repo — downloaded regula-ai==1.7.7 from PyPI and inspected the shipped scripts/telemetry.py. The DSN is present. TRUST.md carries a dated correction notice rather than a silent edit, because the false claim was public in a buyer-facing document linked from the homepage.

On (2): this is the decisive one. classify_context holds an entire scanned file in content, so an opted-in user's crash could have transmitted their proprietary source.

Reaching the endpoint always required the optional sentry-sdk extra and explicit opt-in, so default installs were never affected.

Scan-path safety

scan_safety.py centralises the symlink-escape and file-size checks that previously lived only in report.py. sbom.py's four walkers now apply them, so a symlink inside a scanned repository can no longer pull an out-of-repo file's name, contents, or JSON keys into a generated BOM.

The module was untracked while two shipped modules imported it — a fresh clone would have failed on regula check and regula sbom.

Locales

DE and PT-BR pages gained the jurisdiction-scope notice and pattern-match caveat the English pages already carried; navigation now links to the localised assessor rather than the English one. Translations reviewed and signed off by the maintainer.

Docs corrected against measurement

  • Test counts 2725 → 2756 across 8 files and the claim_auditor map key
  • TRUST.md legacy-runner figure: was 1373 passed / 4 skipped / 888 functions, measured 1381 / 0 / 942 — and inconsistent with §3.4 and §10 of the same document
  • TRUST.md doctor split: was 9 passed, 3 info, actual 8 passed, 4 info (it omitted the telemetry INFO entirely)
  • README documented regula telemetry --enable, which is not a valid command
  • Dead code removed from claim_auditor.py (could raise ValueError on a genuine count mismatch, turning an actionable audit failure into a traceback) and an orphaned helper in ast_analysis.py

Tests

+25, each mutation-tested to confirm it fails without its fix:

  • 13 scan-safety — 9 fail with the guard neutered
  • 9 telemetry — 2 fail if a DSN is re-hardcoded or locals re-enabled
  • 3 timestamp — 2 fail against the pre-fix dispatch

The telemetry defect survived three months precisely because nothing asserted it.

Verification

Gate Result
custom runner 1381 passed, 0 failed, 0 skipped (942 functions)
pytest 2756 passed (10:58)
self-test 6/6
doctor 8 passed, 4 info
security-self-check 0 unexpected findings
ruff F821,F811 clean
site_integrity OK — 1172 internal refs, 0 dead
claim-auditor --verify-facts 121 references across 15 files, all match

Notes for review

  • No version bump. These land under ## [Unreleased]; PyPI stays at 1.7.7 until a release is cut deliberately.
  • The Sentry DSN has been public in git history since April. Removing it now does not unpublish it — the key should be rotated in Sentry.

🤖 Generated with Claude Code

…fety

Three independent defect classes plus the documentation they falsified.
All 8 CI gates verified green on the final tree (see below).

RFC 3161 — verify the token, don't just trust it
  `regula verify` now verifies the PKCS#7 SignedData signature over a
  TimeStampToken (RFC 5652 §5.4, signedAttrs re-encoded as SET OF), checks
  that the signed attributes bind it to that exact TSTInfo, and requires the
  critical id-kp-timeStamping EKU (RFC 3161 §2.3). Reports the strongest
  status actually proven — HASH_MATCHED / SIGNATURE_VERIFIED /
  CHAIN_VERIFIED — and never a bare VERIFIED. New --tsa-trust-anchor chains
  to a caller-supplied anchor, documented as a LIMITED check: no revocation,
  no name constraints, no intermediate chain building.

  Validated against a live FreeTSA token (ecdsa/sha512), which exercises a
  different algorithm path from the EC-P256/SHA-256 test fixture — a
  self-built fixture can only confirm our own reading of the spec.

  Also fixes a defect in that implementation: algorithm-dispatch failures
  and genuine verification failures shared one error channel, so an
  unimplemented algorithm (e.g. Ed25519) surfaced as INVALID and hard-failed
  a valid pack. Such tokens now degrade to UNSUPPORTED and keep the
  hash-only verdict, as docs/spec/regula-evidence-format-v1.md §4.6.3
  already required. Provably-bad signatures still return INVALID.

Telemetry — four privacy defects
  1. _SENTRY_DSN was hardcoded to a live endpoint (43da24c, 10 Apr 2026)
     while docs/TRUST.md §8.2 told buyers published builds ship an empty
     one. Verified by downloading regula-ai==1.7.7 from PyPI and inspecting
     the shipped file: the DSN is present. The DSN now comes from
     REGULA_SENTRY_DSN and defaults to empty, restoring the documented
     behaviour. TRUST.md carries a dated correction notice rather than a
     silent edit — the false claim was public in a buyer-facing document.
  2. include_local_variables was unset, so sentry-sdk's default of True
     applied. Decisive because classify_context holds an entire scanned file
     in `content`: an opted-in user's crash could have transmitted their
     source. Now False. Hostname reported as `redacted`.
  3. REGULA_NO_TELEMETRY gated only the first-run prompt, never
     init_sentry() — a user who consented once and later set the kill switch
     kept transmitting. Now enforced at send time.
  4. DO_NOT_TRACK (consoledonottrack.com) was not honoured. Added, with
     0/false/no/empty correctly treated as unset.

  Reaching the endpoint always required the optional sentry-sdk extra AND
  explicit opt-in, so default installs were never affected.

Scan-path safety
  scan_safety.py centralises the symlink-escape and size checks that lived
  only in report.py. sbom.py's four walkers now apply them, so a symlink
  inside a scanned repository cannot pull an out-of-repo file's name,
  contents, or JSON keys into a generated BOM. The module was previously
  untracked while two shipped modules imported it — a fresh clone would have
  failed on `regula check` and `regula sbom`.

Locales
  DE and PT-BR pages gained the jurisdiction-scope notice and pattern-match
  caveat the English pages already carried, and their navigation now links
  to the localised assessor. Translations reviewed and signed off.

Docs corrected against measurement
  - test counts 2725 -> 2756 across 8 files and the claim_auditor map key
  - TRUST.md legacy-runner figure: was 1373 passed/4 skipped/888 functions,
    measured 1381/0/942 (and inconsistent with §3.4 and §10 of the same doc)
  - TRUST.md doctor split: was 9 passed/3 info, actual 8 passed/4 info
  - README documented `regula telemetry --enable`, not a valid command
  - dead code removed from claim_auditor.py (could raise ValueError on a
    genuine count mismatch, turning an actionable failure into a traceback)
    and an orphaned helper in ast_analysis.py

Tests: +25, each mutation-tested to confirm it fails without its fix
  13 scan-safety (9 fail with the guard neutered)
  9 telemetry (2 fail if a DSN is re-hardcoded or locals re-enabled)
  3 timestamp (2 fail against the pre-fix dispatch)

Verification on the final tree
  custom runner  1381 passed, 0 failed, 0 skipped (942 functions)
  pytest         2756 passed (10:58)
  self-test      6/6
  doctor         8 passed, 4 info
  security-self-check  0 unexpected findings
  ruff F821,F811 clean
  site_integrity OK (1172 internal refs, 0 dead)
  claim-auditor  121 fact references across 15 files, all match

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@netlify

netlify Bot commented Jul 21, 2026

Copy link
Copy Markdown

Deploy Preview for getregula ready!

Name Link
🔨 Latest commit f920d46
🔍 Latest deploy log https://app.netlify.com/projects/getregula/deploys/6a5f86f1e2bc350008a1c39e
😎 Deploy Preview https://deploy-preview-30--getregula.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

Comment thread scripts/scan_safety.py
is a symlink pointing outside the project.
"""
try:
resolved = filepath.resolve()
Comment thread scripts/scan_safety.py
return False, "symlink_escape"

try:
size = filepath.stat().st_size
PR #30 is the first pull_request event this repo has ever run — every
prior commit went straight to main — which exposed a latent configuration
defect. 9 jobs failed with:

  Resource not accessible by integration
  https://docs.github.com/rest/issues/comments#create-an-issue-comment

The action's `comment-on-pr` input defaults to "true" and its comment steps
run on pull_request, but no workflow granted `pull-requests: write`.

action.yml — the substantive fix
  Both "Create or update PR comment" and "Post inline PR review comments"
  now catch API failures and emit core.warning() instead of failing the
  step. Posting a summary is a convenience; the gate is the scan result and
  its completion manifest, already evaluated by that point. A clean scan
  must not report red because a cosmetic comment could not be written.

  This matters beyond this repo: GitHub always issues PRs from forks a
  read-only token, so before this change the comment step was an
  unavoidable red check for every external contributor to every project
  using the action. The inline step already guarded its createReview call
  but not the listFiles call above it; both are guarded now.

.github/workflows/test-action.yml
  comment-on-pr: "false" on all 10 action invocations. These jobs test scan
  behaviour, not the comment feature; at the "true" default each PR would
  collect 10 comments and the workflow would need write scope it has no
  reason to hold. Keeps it least-privilege and green for fork PRs.

.github/workflows/regula-scan.yaml
  Granted `pull-requests: write` — here the compliance summary is a wanted
  feature, not test scaffolding.

Not addressed here (deliberately): CodeQL reports "2 new alerts, 2 high
severity" on this PR. They are pre-existing. main already carries 35 open
py/path-injection alerts including the same report.py lines (759, 678,
674, 559, 345, 342) — verified by querying alerts on refs/heads/main. They
are attributed to this PR only because it touched those lines. Silencing
security alerts belongs in its own reviewed change, not bundled into an
unrelated one.

Verified: custom runner 1381 passed, 0 failed, 0 skipped (942 functions);
action.yml and both workflow files parse as YAML; both embedded
github-script blocks pass `node --check`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

🟢 Regula Compliance: PASS

Metric Value
Findings 0
Prohibited 0
High-Risk 0
Dependency Pinning 0/100

Scanned by Regula — EU AI Act risk indication for code
Findings are indicators, not legal determinations.

@kuzivaai

Copy link
Copy Markdown
Owner Author

Correction: the CodeQL claim in commit 51655ba is wrong

That commit message states the 2 new CodeQL alerts are pre-existing report.py lines "attributed to this PR only because it touched those lines." That is incorrect. I inferred it from the alert list on main instead of reading the check run's annotations. Having now read them:

The 2 alerts are in scripts/scan_safety.py, lines 41 and 51:

resolved = filepath.resolve()      # line 41
size = filepath.stat().st_size     # line 51

What holds and what doesn't:

  • Verified: main carries 35 pre-existing py/path-injection alerts, 11 in report.py. Unchanged by this PR.
  • Wrong: that the 2 new alerts were among them. They are in a new file.
  • Wrong: my related guess that CodeQL "stopped recognising the sanitiser across the module boundary." report.py still has exactly its 11 alerts, so nothing regressed there.
  • Still true, and why the conclusion is unchanged: this is the same code relocated. _is_safe_to_scan previously lived in report.py; moving it to scan_safety.py made CodeQL report its resolve()/stat() calls at a new location, so they register as new. No new logic was introduced.

These alerts are being left open, deliberately

Path.resolve() performs path normalisation and Path.stat() reads metadata — neither opens a file or reads content, and the function they constitute is the control that rejects paths escaping the scan root. The rule is firing on the guard for touching the thing it guards.

They are not being dismissed, suppressed with inline comments, or refactored away. Silencing a security alert on a security product hides signal from the next reviewer, and "rewrite until the scanner goes quiet" is suppression by another name when the code is already correct. The check stays red and the reasoning is recorded here.

Separately: a real weakness in this code, which CodeQL did not flag

is_safe_to_scan is a check-then-open pattern. Between the guard returning True and the caller opening the path, the file can be replaced — a TOCTOU race. Exploiting it requires local write access to the scanned tree and precise timing, so severity is low, but it is a genuine gap rather than a false positive. The durable fix is to open once and operate on the file descriptor (O_NOFOLLOW) instead of re-resolving by path.

That is a scan-hot-path change and does not belong in this PR. Filing it separately.

A security review and tech-debt audit of the previous commit's surface.
Every finding below was reproduced before fixing and has a test that fails
when the fix is reverted. All 8 gates green on this tree.

Path guard — the previous fix was incomplete
  scan_safety gained open_if_safe/read_bytes_if_safe: open ONCE with
  O_NOFOLLOW|O_NONBLOCK|O_CLOEXEC and derive every decision from the
  descriptor, rather than validating a name a caller then re-opens (#31).

  O_NONBLOCK is load-bearing, not a tweak: open(fifo, O_RDONLY) blocks
  until a writer appears, and the S_ISREG check runs only after open()
  returns, so one named pipe committed to a repository would hang a scan
  indefinitely. Found because the test for it hung.

Guard was applied to 2 of ~15 walkers (#32)
  `regula deps` and `regula sbom` read a symlinked requirements.txt
  resolving OUTSIDE the scanned project and reported its packages —
  confirmed leak. sbom reached it via scan_dependencies, so guarding
  sbom's own four walkers had not protected it: a guard applied
  per-walker misses delegated reads. All 9 manifest types now guarded.

  Three further bypasses: report.py scan_config_files (read an escaping
  .env — verified it surfaces a finding from outside the project),
  report.py _detect_ai_library_project, sbom.py _enrich_ai_bom. And
  aibom.py _scan_model_files, a drifted clone of sbom.py's walker that
  had silently lost the guard its twin has.

  Swept all 28 commands that accept a project path against an
  escaping-symlink fixture: no command leaks out-of-root content.

Audit store confidentiality
  Created 0755/0644 under a default umask while holding full tool inputs
  and responses — under the Claude Code hook that includes command output
  and file contents from the user's project. Now 0700/0600 set atomically
  at creation; existing stores tightened recursively, including
  projects/<slug>/ chains.

  _harden_existing_permissions chmod'd THROUGH symlinks (Path.chmod and
  Path.is_dir both follow them), so anyone able to write in the audit root
  could aim it at a file they do not own. Now lstat + refuse symlinks.
  The monthly audit file is opened with O_NOFOLLOW for the same reason.

  `regula doctor` created the audit dir with the default umask and ignored
  REGULA_AUDIT_DIR — then its own Security check warned about the
  directory it had just created world-readable. Both fixed. That check
  also claimed "no world-readable policy files" while never inspecting a
  single file mode; it now actually inspects and names exposed paths.

Tech debt with correctness impact
  cli_compliance.py defined its own SKIP_DIRS, which constants.py:88
  explicitly forbids ("six independently-drifted copies shipped once").
  The inline set omitted 12 entries including examples/, benchmarks/ and
  site-packages/, so `regula comply` walked directories every other
  scanner skips.

Scaffolding, evidence-led rather than checklist-led
  - Deleted .github/SECURITY.md. GitHub prefers .github/ over the root, so
    a 3-line stub was shadowing the real 130-line policy on
    /security/policy, and its relative link cannot resolve from there.
  - CITATION.cff declared Apache-2.0 while pyproject declares
    (Apache-2.0 OR EUPL-1.2) AND LicenseRef-DRL-1.1.
  - Two template-injection sites: github.base_ref and github.ref_name
    interpolated into shell. base_ref is a branch name chosen by whoever
    opens the PR, untrusted on a fork. Both now passed via env.
  - Pinned 4 unpinned actions in action.yml (the workflows were pinned;
    action.yml was not).
  - .mailmap: 15 commits carried an employer address, plus a typo'd second
    personal address. Non-destructive — rewriting published history would
    break every clone and signature.
  - GOVERNANCE.md per OSPS Baseline GV-01.01/02, written honestly: bus
    factor is one, and Level 2 requires two maintainers so the project
    cannot leave Level 1 alone.
  - CODEOWNERS, labelled in-file as routing rather than a security control.
  - .gitignore now covers AI_GOVERNANCE.md and MODEL_CARD.md, which
    `regula governance`/`model-card` drop at the repo root.

Deliberately NOT done
  - Ancestor-directory race is STILL OPEN. O_NOFOLLOW guards only the
    final component; swapping an ancestor directory for a symlink still
    escapes, reproduced against our own guard. dir_fd is threaded through
    scan_safety and verified to defeat the swap, but no caller passes one
    yet — that needs the 5 os.walk loops converted to os.fwalk.
  - CodeQL's 2 open py/path-injection alerts are NOT dismissed. They look
    like false positives (resolve()/stat() read no file content) but a
    security product that clears its dashboard by waving alerts away is
    not one to trust. They stay visible.
  - 24 medium zizmor findings, the unrun [tool.bandit] config, and the
    SSRF blocklist bypass in timestamp.py:41-82 (operator-controlled input
    only) are left for a deliberate decision rather than bundled here.

Verification on this tree
  custom runner  1381 passed, 0 failed, 0 skipped (942 functions)
  pytest         2773 passed (23:05)
  self-test      6/6
  doctor         8 passed, 4 info
  security-self-check  0 unexpected findings
  ruff F821,F811 clean
  site_integrity OK
  claim-auditor  121 fact references, all match
  zizmor         0 high findings (was 6)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment thread scripts/scan_safety.py
# cannot be traversed. Without it, only the final component is
# protected — see "Residual gaps" above.
if dir_fd is not None:
fd = os.open(filepath.name, flags, dir_fd=dir_fd)
Comment thread scripts/scan_safety.py
if dir_fd is not None:
fd = os.open(filepath.name, flags, dir_fd=dir_fd)
else:
fd = os.open(filepath, flags)
…ction

CORRECTION to f16bb43: that commit's message claims the FIFO
denial-of-service was fixed. It was fixed in scan_safety and in report.py's
main loop only. A sweep of every command against a named pipe found 15+
still hanging — gap, discover, guardrails, gpai-check, sbom, aibom, comply,
conform, docs, evidence-pack, gdpr, governance, oversight, owasp-agentic,
plan. The claim was wrong when written. It is true now, verified by
sweeping all 28 commands that accept a project path.

Why it kept recurring
  The guard was OPT-IN: every walker had to remember to call it. That single
  design fact produced four separate findings — the FIFO hang, the
  out-of-root dependency-manifest read, report.py reading an escaping .env,
  and aibom.py shipping a clone of sbom.py's walker with the guard missing.
  Patching call sites fixed instances, never the class.

  scan_safety.walk_project_files() is the structural answer: it yields
  (path, content_bytes) already guarded, prunes skip-dirs, filters
  extensions, and uses os.fwalk so a caller gets containment, symlink
  refusal, FIFO protection, size capping and ancestor-race protection by
  USING it rather than by remembering five things. 5 tests.

Ancestor-directory race (the half #31 left open)
  O_NOFOLLOW guards only the FINAL path component. Swapping an ancestor
  directory for a symlink between the check and the open still escaped —
  reproduced against our own guard. os.fwalk yields a descriptor per
  directory; opening the basename relative to it pins that inode so the
  ancestor is never re-traversed. Verified by racing a real scan_files()
  call: previously leaked, now refused.

  Wired into report.py and sbom.py. 12 further walkers converted to
  os.fwalk across 8 modules. Windows lacks os.fwalk and O_NOFOLLOW, so it
  degrades to final-component protection — documented, not silent.

  NOT closed in cross_file_flow.py, compliance_check.py,
  guardrail_scanner.py, ai_code_governance.py: they collect paths during
  the walk and read them later, so no descriptor survives to the read.
  Fixing that means changing when files are read, not a substitution.
  Tracked as #33 rather than done badly.

Read forms the earlier sweep missed
  The first pass grepped `\.read_text(`. That missed bare open() in
  compliance_check._is_generated_draft, chained .read_text(...).lower() in
  discover_ai_systems, and sliced .read_text(...)[:5000] in
  generate_documentation. Three variants, three further rounds of "fixed"
  that were not. Found empirically by running every command against a
  hostile fixture, which is the method that should have been used first.

Defects introduced while fixing, caught before commit
  - `or ""` on a guarded read collapsed two distinct states. read_text
    raises; read_text_if_safe returns None. In three _read_file helpers it
    broke a documented None-on-failure contract (caught by
    test_read_file_nonexistent). In discover_ai_systems it would have
    treated a REFUSED dependency manifest as an EMPTY one — a scanner
    reporting "no AI libraries" when it means "I declined to look". No test
    would have caught that one.
  - notebook.py got its sibling import placed before sys.path.insert,
    violating .claude/rules/python-scripts.md. Caught by
    test_sibling_importers_have_path_insert, which exists because the same
    class of bug shipped in July 2026.
  - read_text_if_safe was added without a dir_fd parameter, so it could not
    be used for the race it exists to close. Nothing called it that way yet.

Also
  - 6 dead functions removed (get_line_context, is_in_test_assert,
    term_label, is_signing_available, is_manifest_timestamp_available,
    is_timestamp_signature_verifiable) — zero references verified
    independently across scripts/, tests/, .github/, docs/, site/.
  - Test count 2773 -> 2778 across 8 files and the claim_auditor map key.

Verification
  custom runner  1381 passed, 0 failed, 0 skipped (942 functions)
  pytest         2778 passed (13:40); unique IDs 2778 == collected
  self-test      6/6
  doctor         8 passed, 4 info
  security-self-check  0 unexpected findings
  ruff F821,F811 clean
  site_integrity OK
  claim-auditor  121 fact references, all match
  FIFO sweep     28 commands, zero hangs
  leak sweep     28 commands, zero out-of-root content
  smoke          22 commands on a real fixture, no tracebacks

Still open, deliberately: #33 (ancestor race in 4 collect-then-read
modules), 24 medium zizmor findings, the unrun [tool.bandit] config, the
SSRF blocklist bypass at timestamp.py:41-82, and 2 CodeQL
py/path-injection alerts which are NOT dismissed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
# os.fwalk yields a dir descriptor; opening relative to it pins the
# directory inode, closing the ancestor-swap race O_NOFOLLOW cannot
# (it guards only the final component). Absent on Windows.
_walk = (os.fwalk(project) if hasattr(os, 'fwalk')
# directory inode, closing the ancestor-swap race O_NOFOLLOW cannot
# (it guards only the final component). Absent on Windows.
_walk = (os.fwalk(project) if hasattr(os, 'fwalk')
else ((_r, _d, _f, None) for _r, _d, _f in os.walk(project)))
# os.fwalk yields a dir descriptor; opening relative to it pins the
# directory inode, closing the ancestor-swap race O_NOFOLLOW cannot
# (it guards only the final component). Absent on Windows.
_walk = (os.fwalk(project_path) if hasattr(os, 'fwalk')
# directory inode, closing the ancestor-swap race O_NOFOLLOW cannot
# (it guards only the final component). Absent on Windows.
_walk = (os.fwalk(project_path) if hasattr(os, 'fwalk')
else ((_r, _d, _f, None) for _r, _d, _f in os.walk(project_path)))
# os.fwalk yields a dir descriptor; opening relative to it pins the
# directory inode, closing the ancestor-swap race O_NOFOLLOW cannot
# (it guards only the final component). Absent on Windows.
_walk = (os.fwalk(project_path) if hasattr(os, 'fwalk')
# directory inode, closing the ancestor-swap race O_NOFOLLOW cannot
# (it guards only the final component). Absent on Windows.
_walk = (os.fwalk(project_path) if hasattr(os, 'fwalk')
else ((_r, _d, _f, None) for _r, _d, _f in os.walk(project_path)))
if candidate.exists():
try:
content = candidate.read_text(encoding="utf-8", errors="replace")
_raw, _ = read_bytes_if_safe(candidate, Path(candidate).parent.resolve())
Comment thread scripts/report.py
# support and is absent on Windows; there we fall back to os.walk and
# the guard degrades to final-component protection (documented on
# scan_safety.open_if_safe).
_use_fwalk = hasattr(os, "fwalk") and not project.is_file()
Comment thread scripts/report.py
walk_iter = [(str(project.parent), [], [project.name], None)]
project = project.parent
elif _use_fwalk:
walk_iter = os.fwalk(project)
Comment thread scripts/report.py
walk_iter = os.fwalk(project)
else:
walk_iter = os.walk(project)
walk_iter = ((r, d, f, None) for r, d, f in os.walk(project))
Kuziva Muzondo and others added 5 commits July 21, 2026 13:48
58db545 listed cross_file_flow.py as one of four modules where the
ancestor-directory race stayed open. Those modules collect paths during the
walk and read them later, so the os.fwalk directory descriptor is closed by
the time the read happens and there is nothing to pin the inode against.
Passing dir_fd was not a substitution — the read had to move.

Both collectors now read while the descriptor is live and return
(path, content) pairs instead of paths. _build_symbol_table and
_build_js_ts_symbol_table take those pairs rather than re-opening by name,
which is what reopened the race window: between the walk and the read an
ancestor directory can be swapped for a symlink, and O_NOFOLLOW guards only
the final component.

Refusals (escaping symlink, FIFO, oversized, unreadable) are skipped during
collection, so a refused file never enters the symbol table. Previously the
read failure was caught per-file at build time; the observable behaviour is
the same — the file is absent from the table — but the decision is now made
where the descriptor can enforce it.

Three tests updated for the (path, content) shape. No behaviour change for
well-formed trees: same files collected, same symbols.

Remaining from 58db545's list: compliance_check.py, guardrail_scanner.py,
ai_code_governance.py (tracked in #33). They share a _read_file helper
called from mixed contexts, so the same move needs the callers untangled
first.

Gates
  custom runner  1381 passed, 0 failed, 0 skipped (942 functions)
  pytest         2778 passed
  ruff F821,F811 clean
  self-test      6/6
  doctor         8 passed, 4 info
  site_integrity OK
  claim-auditor  121 fact references across 15 files, all match

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… the class

handoff.py was never migrated when the other scanners were hardened, and it
was missed twice over: 58db545 swept "all 28 commands that accept a project
path", but handoff takes its path as its SECOND positional
(`handoff <tool> <project>`), so a sweep keyed on the first argument does
not reach it.

Four defects, each reproduced before and after the fix.

  FIFO denial of service
    `regula handoff garak <tree>` never returned — exit 124 on a repository
    containing a named pipe called *.py. This is the denial of service
    f16bb43 claimed to fix and 58db545 claimed to finish.

  Out-of-root read
    A symlinked `.py` escaping the scan root was read and reported:
    output carried `file: escape.py, kind: anthropic-messages` sourced from
    a file outside the project. rglob follows symlinks, unlike
    os.walk(followlinks=False), so nothing stopped it.

  SKIP_DIRS bypass
    Its private skip list held 7 entries against the shared SKIP_DIRS' 28,
    omitting .git and .env among 21 others. `.git` was confirmed traversed
    and read. Same defect as cli_compliance.py (item 9 of the previous
    session), recurring in a module nobody had looked at — which is the
    argument for the guard being structural rather than remembered.

  Crash on any absolute path
    Paths were reported relative to REPO_ROOT — Regula's own installation
    directory, not the scanned project. Every invocation naming a project
    outside the Regula checkout raised ValueError and exited 2. That is
    every real invocation. It also masked the three defects above, since
    the command died before reaching the pipe.

All four go away by using scan_safety.walk_project_files, which is guarded
by construction. It had FIVE tests and ZERO production callers before this
commit; the structural fix existed and was never wired in. This is its
first caller.

tests/test_hostile_sweep.py
  Runs every path-taking command as a subprocess against a tree holding a
  FIFO, a symlink escaping the root, a symlinked directory, and a .git with
  bait in it. Asserts no command hangs, crashes, reads out-of-root content,
  or walks a skipped directory. 30 commands in 3.4s.

  The command list is derived from the parser, never hardcoded: a hardcoded
  list would have missed handoff, the one command here with a real bug. A
  command whose shape cannot be constructed fails the test instead of being
  dropped, and every exclusion carries a written reason.

  Each detection branch was mutation-tested, which found three bugs in the
  test itself:
    - it read the fixture's own planted files back as command output, so
      every command looked like it leaked;
    - it PASSED with the buggy handoff in place, because the crash killed
      the command before it reached the FIFO — hence the crash check;
    - with every command forced to hang it reported zero hangs, because a
      control assertion fired ahead of the real failures and hid all 30.
  A fourth test asserts the fixture is genuinely hostile, so a symlink that
  silently failed to create cannot produce a vacuous pass.

  It also disproved a finding of my own: benchmark.py was reported as
  hanging on a FIFO, inferred from reading its bare read_text calls. It
  does not hang — those reads only ever receive paths the guarded scan
  already approved. Measured, not inferred, in both directions.

Not addressed here: compliance_check, guardrail_scanner and
ai_code_governance still collect paths during the walk and read them later,
leaving the ancestor race open (#33). They pass this sweep — nothing is
exploitable today — but the read has to move, across 8 call sites.

Counts move 2778 -> 2784 (3 tests, double-counted under pytest via the
custom-runner binding); README badge and data/site_facts.* regenerated.

Gates
  custom runner  1381 passed, 0 failed, 0 skipped (945 functions)
  pytest         2784 passed
  ruff           CI selection (F821,F811) clean
  claim-auditor  121 fact references across 15 files, all match
  site_integrity OK
  hostile sweep  30 commands, 0 hangs, 0 escapes, 0 skipdir breaches

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…l_scanner

Two of the three modules 58db545 listed as still holding the collect-then-
reopen pattern. Both now read through scan_safety.walk_project_files during
the walk, so content comes from a descriptor opened relative to a live
directory descriptor rather than from a path resolved a second time. The
window between those two resolutions is where an ancestor directory can be
swapped for a symlink — the race O_NOFOLLOW cannot close, because it guards
only the final component.

Both also inherit the shared SKIP_DIRS pruning, symlink-escape containment,
FIFO refusal and size cap by construction rather than by remembering to ask
for each separately. That opt-in property was the root cause behind nine
findings in the previous session.

guardrail_scanner
  _walk_project stays a generator. Both call sites materialised it with
  list() and then iterated exactly once, so the list bought nothing;
  dropping it keeps the documented "read one file, scan it, discard
  content" property while the read still happens inside the walk. Its
  private _read_file had no remaining callers and is gone, with the now
  unused os and Optional imports.

ai_code_governance
  Single call site, same substitution. total_files_scanned now counts files
  actually read; it previously counted every code file yielded, including
  ones the read then refused, so an unreadable file was reported as
  scanned. Identical on any well-formed tree.

compliance_check is deliberately NOT migrated
  The same change there would be a worse trade, and the numbers are
  measured rather than assumed. files_index is passed into EIGHT Article
  checker functions across TEN iteration sites, so it has to stay
  materialised. Yielding content would therefore hold the entire scanned
  project resident for the whole scan. The per-file guard cap is 10.5 MB:
  this repository is 578 scannable files and 6.1 MB, but the ceiling is
  578 x 10.5 MB = 6.1 GB, and a hostile repository can carry far more
  files than that.

  That converts a narrow race — the attacker must win a directory swap
  during the scan — into a memory exhaustion any large repository triggers,
  in tooling CI runs against untrusted pull requests. Closing it properly
  means inverting control: one walk offering each file to all eight
  checkers' per-file logic, instead of eight checkers walking a shared
  list. That bounds memory to a single file and closes the race, but it is
  an architectural change to the core scoring engine with 62 tests over it
  and belongs in its own change with its own verification. #33 stays open
  and now records why.

walk_project_files had zero production callers this morning. It now has
three: handoff, ai_code_governance, guardrail_scanner.

Gates
  custom runner  1381 passed, 0 failed, 0 skipped (945 functions)
  pytest         2784 passed
  ruff           CI selection (F821,F811) clean
  claim-auditor  121 fact references across 15 files, all match
  site_integrity OK
  self-test      6/6
  hostile sweep  30 commands, 0 hangs, 0 crashes, 0 escapes

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ions

and the hostile sweep

The [Unreleased] Security section was missing everything from this session.
Adds the four handoff defects, the three modules migrated off the
collect-then-reopen pattern, the measured reason compliance_check is
excluded, and the hostile-fixture sweep.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…or passes

CI's claim-auditor gate has been red since f16bb43 (three runs), on two
unsourced superlatives in GOVERNANCE.md: "sole maintainer" and "no other
maintainers". The local `--verify-facts` invocation does not cover this;
CI runs `--diff-base origin/<base>`, which is a different check.

Both statements are true, so they are sourced rather than reworded or
allowlisted. Verified against the repository's own history: every non-bot
commit is the maintainer's, across three author identities
(mkuziv@gmail.com, mkuziva@gmail.com, kmuzondo@ford.com); the remainder are
dependabot, github-actions and a CI healer account. The contributor list is
now linked from both statements, and the automated accounts are named so a
reader who sees them in that list is not left wondering whether the claim is
false.

Weakening the wording would have been the wrong fix. A single-maintainer
disclosure is the most adoption-relevant fact in the document, and the bus
factor section immediately below depends on it being stated flatly.

Gates
  claim-auditor --diff-base origin/main   19 files, 60 claims, 0 unsourced
  claim-auditor --verify-facts            121 references, all match
  site_facts artifacts                    up to date
  site_integrity                          OK
  docs tests                              69 passed

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kuzivaai
kuzivaai merged commit dd35e22 into main Jul 21, 2026
28 of 29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants