feat(derive): mint EXTERNAL_MODULE leaves for external JS imports (REG-624) - #449
Conversation
…G-624)
Close the graph dead-end where npm / scoped / unregistered-builtin JS imports
had no outgoing IMPORTS_FROM edge (REG-624: 48% of all IMPORT nodes). The
js_builtins derive pack previously minted an EXTERNAL_MODULE only for the
hardcoded registered Node builtins; every other external dependency import was
a leaf-less dead-end, so effects-db lookups (traceEffects.ts EXTERNAL_MODULE
arm) and "what external deps does this module use?" queries never resolved.
js_builtins_nodes.dl now mints an EXTERNAL_MODULE (source "external", file
"<external>") for every project-EXTERNAL import and js_builtins_edges.dl links
the IMPORT to it via an additive IMPORTS_FROM (resolvedVia "external").
Scope (an import is external iff ALL hold):
- specifier is not relative ("." prefix) and not absolute ("/" prefix) — an
unresolved RELATIVE import must stay a violation of the
imports-resolve-to-module guarantee (REG-1166), never be masked;
- not a registered Node builtin (those keep the richer nodejs-builtin leaf);
- not already resolved to a MODULE by an earlier pack (js_module_imports /
js_import_bindings run first and commit IMPORTS_FROM) — drops workspace
packages, which are internal.
A leading "node:" is normalized away so node:zlib and zlib share one leaf.
source = "external"; richer npm/crate/hackage tagging and subpath→package
grouping (lodash/merge → lodash) are deferred follow-ups. Rust crate and
Haskell import leaves remain out of scope (REG-687).
Verified: cargo test --lib (1408 passed, 0 failed) incl. two new derive tests
covering the npm/scoped/subpath/node:-normalization positives and the
relative/absolute/builtin/workspace-resolved/.rs negatives.
https://claude.ai/code/session_0129N2LmeRmrJQbTTbxXLPRJ
|
🚫 QA precheck REJECT (attempt 1, base=main). QA gate |
|
Both gate signals are expected for this change — neither is a fixable defect, so no code change:
Scope is Rust-only (3 derive files + 1 orchestrator description string); zero TS touched. Diff: +260/−5 across 4 files. Ready for owner review. Generated by Claude Code |
|
🚫 QA precheck REJECT (attempt 2, base=main). QA gate |
|
🚫 QA precheck REJECT (attempt 3, base=main). QA gate |
|
🚫 QA precheck REJECT (attempt 4, base=main). QA gate |
|
🚫 QA precheck REJECT (attempt 5, base=main). QA gate |
Closes part of REG-624 — Unresolved imports for external dependencies — need leaf nodes (JS slice).
Problem (BEFORE)
533 / 1118 IMPORT nodes (48%) had no outgoing
IMPORTS_FROMedge — they were graph dead-ends. Thejs_builtinsderive pack only minted anEXTERNAL_MODULEfor a hardcoded set of registered Node builtins (fs,path, …). Every npm package / scoped package / unregistered-builtin JS import (axios,react,@scope/pkg,node:zlib) was left leaf-less, so:traceEffects.ts:321(theEXTERNAL_MODULE → effects-dblookup arm) never fired for npm packages — the entireeffects-db/packages/*.yamlcorpus (axios, execa, glob, …) was unreachable;Evidence the pack was builtin-only:
js_builtins_nodes.dlmintedext_module_nodesolely from thebuiltin_spec(...)ground facts (node builtins).SPEC
Invariant restored: every project-EXTERNAL JS IMPORT has an
IMPORTS_FROMedge to anEXTERNAL_MODULEleaf (no external dead-ends), without masking genuinely-broken internal imports.Given a JS/TS file imports a bare specifier that is not a project module and not a registered builtin
When the
js_builtins_nodes/js_builtins_edgesderive packs runThen an
EXTERNAL_MODULEleaf (source="external",file="<external>") is minted and the IMPORT gets an additiveIMPORTS_FROM(resolvedVia="external") to it.Scope — an import is external iff ALL hold:
.prefix) and not absolute (no/prefix) — an unresolved relative import MUST stay a violation of theimports-resolve-to-moduleguarantee (REG-1166), never be masked by a fake leaf;builtin_spec) — those keep the richernodejs-builtinleaf; excluded soEXTERNAL_MODULE:fsis never re-minted with a conflictingsource;js_module_imports/js_import_bindingsrun beforejs_builtins_*(perSTDLIB_PACKSorder) and commit theirIMPORTS_FROMas EDB, so this drops workspace packages (@scope/util→ a real project MODULE), which are internal.A leading
node:is normalized away (strip_prefix) sonode:zlibandzlibcollapse to one leaf (byte-identical sid ⇒ set-dedup).AFTER (evidence)
cargo test --libinpackages/rfdb-server: 1408 passed, 0 failed. New/updated derive tests:js_builtins_nodes_mints_external_leaf_for_npm_imports— positives:axios,@grafema/util,lodash/merge,node:zlib→zlib(+ barezlibshare one leaf); negatives that mint nothing:./local,/abs, registeredfs, workspace@scope/internal(hasIMPORTS_FROM→MODULE),.rsIMPORT.js_builtins_edges_links_external_imports_to_leaves— external IMPORT →<external>leaf withresolvedVia="external"; relative + already-resolved emit no edge.js_builtins_nodes_mints_legacy_sids(4→5 node heads) andjs_builtins_edges_joins_minted_endpoints(1→2 IMPORTS_FROM heads).cargo checkongrafema-orchestrator(pack description string): exit 0.Adversarial review
Round A — correctness. Empty specifier excluded (
neq(Spec,"")injs_imp). Bare"node:"mints nothing (neq(M,"")). Scoped@scope/pkgand subpathlodash/mergemint distinct leaves (verbatim names — package grouping is a follow-up).node:zlib+zlibdedup to one leaf. Duplicate imports across files → one leaf, one edge per IMPORT. Bug found & fixed mid-review: inline 2-arg\+ builtin_spec(Spec, _)did not exclude builtins (engine negation needs a single-column projection) →fsleaked into the external head; fixed by projectingis_builtin_spec(Spec)(the\+ dotted(C)discipline). Second bug found & fixed: the edge pack both producedIMPORTS_FROMand negatedresolved_imp(which readsedge(_,_,"IMPORTS_FROM")) →NegationInCycle; resolved by droppingresolved_impfrom the edge pack — leaf-existence (minted only by the node pack for unresolved externals) is itself the not-resolved gate.Round B — structural. Prelude (
is_builtin_spec/ext_other_imp/ext_other_name) is duplicated across the two.dlfiles — the established per-pack include pattern (builtin_spec/js_imp/bi_bindingare already duplicated; documented in both headers). The node→edge two-pack coupling is the same contract as the existingext_module_node↔bi_import_edge.Round C — class & invariants. No analysis-pipeline invariant violated: new nodes are leaves (
file="<external>"→ no MODULE), edges are additive.depends(runs after, maps endpoint→file→MODULE) derives no spuriousDEPENDS_ON(no MODULE at<external>); full derive suite green confirms. Sibling latent — the same dead-end exists for Rust crates (rust_importspack) and Haskell imports: out of scope here, already tracked by REG-687.Blast radius / follow-ups
.dl+ cargo tests + one orchestrator description string); zero TS touched. The TS unit suite is orthogonal and requirespnpm buildartifacts not present in this env.rfdb-serverbinaries are rebuilt (release step).sourcetagging + subpath→package grouping (lodash/merge→lodash); Rust/Haskell external leaves (REG-687).https://claude.ai/code/session_0129N2LmeRmrJQbTTbxXLPRJ
Generated by Claude Code