fix(reachability): seed native/cross-language entry points, fix Zig reachability crash, stop silent zero-seed blackout - #75
Closed
gadievron wants to merge 2 commits into
Conversation
…eachability crash, stop silent zero-seed blackout
Six filed defects converge on the shared EntryPointDetector + reachability seeding
path; one is a Zig-local crash.
Central detector (utilities/agentic_enhancer/entry_point_detector.py):
- Add 'main' / 'http_handler' / 'middleware' to ENTRY_POINT_TYPES. The C/Go/Zig
parsers emit these program/web entry unit_types, but the set only knew the
Python/Express web vocabulary, so a compiled binary seeded zero entry points
(total reachability blackout).
- Add _unit_type() dual-key read (snake 'unit_type' + camel 'unitType') for
Check-1, Check-4, details and statistics. The per-parser reachable path
normalizes under camelCase 'unitType' (parsers/{c,php,ruby}/test_pipeline.py:257)
while the detector read snake-only, so Check-1/Check-4 were dead on that path
even for valid entry types. (The earlier-filed file_path/filePath locus is
phantom; the operative residual is exactly this dual-read gap.)
Zig classifier (parsers/zig/function_extractor.py):
- _classify_function: main -> 'main' (was generic 'function'), matching C/Go so a
Zig binary's entry point is now seedable.
Zig reachability filter (parsers/zig/test_pipeline.py):
- Rewrite apply_reachability_filter to the real EntryPointDetector(functions,
call_graph).detect_entry_points() / ReachabilityAnalyzer(functions,
reverse_call_graph, entry_points).get_all_reachable() contract. The old code
called a non-existent API; imports succeed (sys.path), so except ImportError
never fired and the wrong-arity TypeError crashed every Zig parse at
--processing-level reachable. Derived against Zig's own snake-case data shape
(no token-copy of the C normalizer).
Empty-seed safety-net (core/parser_adapter.py):
- A zero entry-point seed previously emptied the dataset silently (100% reduction
reported as SUCCESS) — the dominant failure for non-web library/stdlib targets.
Degrade to pass-through (units preserved, filtering NOT applied) + record a loud
warning in the filter metadata so the blackout can never be silent. The broader
generic-library seeding heuristic is architectural and out of scope.
Tests (RED->GREEN): tests/test_entry_point_detector_native_seeds.py (6),
tests/parsers/zig/test_zig_main_classification.py (3),
tests/parsers/zig/test_zig_reachability_api.py (2, reproduced the exact
TypeError), tests/test_reachability_empty_seed.py (2). 12 failed pre-fix (1
guard green at base by design) -> 13 passed after the fix. Full suite 189 passed /
63 skipped / 0 failed. go test/vet/build clean in parsers/go/go_parser (types.go
unchanged, gofmt-clean). ruff clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
gadievron
requested review from
ar7casper,
dgeyshis,
shahar-davidson,
sounil and
yotamleo
as code owners
May 28, 2026 23:53
…lean install The zig parser modules top-level import tree_sitter_zig, but the dependency was absent from requirements.txt and pyproject.toml, so a clean install raised ModuleNotFoundError: No module named 'tree_sitter_zig' and every zig test errored on collection. Add tree-sitter-zig pinned to the version the zig tests pass with. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced Jun 12, 2026
gadievron
added a commit
that referenced
this pull request
Jun 29, 2026
The #75 zero-seed net only fires at EXACTLY 0 entry points. A library (e.g. tree-sitter) trips a handful of INCIDENTAL seeds (code merely containing an input-reading pattern), yielding a 96.6% reduction (712 -> 24, all wasm) that looks like a successful filter while the real public-API core was dropped. Add blackout_warning() in utilities/agentic_enhancer (shared by all 7 filter sites). Advisory ONLY -- never changes which units are kept. Warns on total blackout (0 kept) OR >=90% pruned with NO structural entry point (route/main/ CLI/handler) -- i.e. all seeds are incidental input-pattern matches. Suppressed under library_mode (high reduction is then the intended precise result). Suggests --library-mode in the message. Wired into core/parser_adapter.py + all 6 subprocess pipelines (c/js/go/ruby/php/zig). Calibration: silent on Arkime (63%/54% reductions, real route/main seeds), fires on tree-sitter. Tests: 7 new covering both triggers + structural-seed and library-mode suppression.
Collaborator
Author
|
Superseded by #134 (the 2026-06-24 release), which independently re-landed this parser work — the added code and tests are verified present on origin/master (line-by-line subset check). Closing to unclog the review queue; please reopen if a specific hunk is found missing on master. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Six filed defects converge on the shared EntryPointDetector + reachability seeding
path; one is a Zig-local crash.
Central detector (utilities/agentic_enhancer/entry_point_detector.py):
parsers emit these program/web entry unit_types, but the set only knew the
Python/Express web vocabulary, so a compiled binary seeded zero entry points
(total reachability blackout).
Check-1, Check-4, details and statistics. The per-parser reachable path
normalizes under camelCase 'unitType' (parsers/{c,php,ruby}/test_pipeline.py:257)
while the detector read snake-only, so Check-1/Check-4 were dead on that path
even for valid entry types. (The earlier-filed file_path/filePath locus is
phantom; the operative residual is exactly this dual-read gap.)
Zig classifier (parsers/zig/function_extractor.py):
Zig binary's entry point is now seedable.
Zig reachability filter (parsers/zig/test_pipeline.py):
call_graph).detect_entry_points() / ReachabilityAnalyzer(functions,
reverse_call_graph, entry_points).get_all_reachable() contract. The old code
called a non-existent API; imports succeed (sys.path), so except ImportError
never fired and the wrong-arity TypeError crashed every Zig parse at
--processing-level reachable. Derived against Zig's own snake-case data shape
(no token-copy of the C normalizer).
Empty-seed safety-net (core/parser_adapter.py):
reported as SUCCESS) — the dominant failure for non-web library/stdlib targets.
Degrade to pass-through (units preserved, filtering NOT applied) + record a loud
warning in the filter metadata so the blackout can never be silent. The broader
generic-library seeding heuristic is architectural and out of scope.
Tests (RED->GREEN): tests/test_entry_point_detector_native_seeds.py (6),
tests/parsers/zig/test_zig_main_classification.py (3),
tests/parsers/zig/test_zig_reachability_api.py (2, reproduced the exact
TypeError), tests/test_reachability_empty_seed.py (2). 12 failed pre-fix (1
guard green at base by design) -> 13 passed after the fix. Full suite 189 passed /
63 skipped / 0 failed. go test/vet/build clean in parsers/go/go_parser (types.go
unchanged, gofmt-clean). ruff clean.
Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com