Skip to content

Run the full audit — on ripwire itself, or with ripwire on another large repository #152

Description

@joyful-ii-V-I

What this is

prompts/full-audit.md is the orchestrator prompt that drives a severity-ranked audit of ripwire
across six lenses: bugs and hostile inputs, measured performance at the scale rung, verb-to-moment
matching, token efficiency, an ecosystem scan of papers and repositories with real momentum, and the
honesty of the output. It has been rewritten from its 2026-08-08 version to fold in what the last
audit learned. Anyone can run it. Running ripwire against a large repository the maintainers do not
have is the most valuable version of it, because that tree is an instrument this project lacks.

The evidence

The 2026-09-10 audit and its follow-ups (#127#136) found real problems:

  • an O(C²) walk costing llvm-project a fifth of its cold parse;
  • a cache that evicted its own working root's data;
  • resolver declines that answered "no callers" while 22% of one corpus's call references were
    dropped without a count;
  • a vendored scanner that could abort on one deeply nested file.

That same audit was fooled three times by its own instruments — cache eviction faking
super-linearity, a cap sweep over a population of zero, and an ISA probe the compiler folded to
scalar. The rewritten prompt puts the control for each of those before any lens: measure the
instrument first.

Also new since the previous version: Lens 1 covers hostile inputs and counting every omission; Lens
2 adds the scale rung (one llvm-sized run per audit), the interleaving and placebo protocol, and
red-first isolation arms instead of timing budgets; Lens 3 adds the adversarial-prose-set lesson;
Lens 6 (honesty of the output) is new; and the process section adds three rules, including
regenerating generated artifacts on the merged tree after a clean merge, because the population may
still be wrong.

Size

Scales with scope. One lens on one repository is small — that is a genuinely useful
contribution on its own. All six lenses with a gated plan is large. The two most valuable coming
from outside the project are Lens 1's hostile inputs and Lens 2 on an unfamiliar large tree.

Prerequisites

A coding agent that can orchestrate sub-agents in worktrees, or the patience to run the lenses one
at a time. Plain and ASan builds of ripwire. Optionally but valuably: a large repository and a
sampling profiler.

Where to start

prompts/full-audit.md is self-contained. Like every prompt in prompts/, it ends by writing a
severity-ranked plan and stopping
— a maintainer agrees the plan before any code is written.

File one issue per finding, each with the argv, the corpus and its commit, expected versus actual
output, the control run, and the severity.

Comment here to claim it, naming the lens and the tree.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions