Skip to content

feat(derive): mint EXTERNAL_MODULE leaves for external JS imports (REG-624) - #449

Merged
Disentinel merged 1 commit into
mainfrom
claude/inspiring-knuth-20f9re
Jun 17, 2026
Merged

feat(derive): mint EXTERNAL_MODULE leaves for external JS imports (REG-624)#449
Disentinel merged 1 commit into
mainfrom
claude/inspiring-knuth-20f9re

Conversation

@Disentinel

Copy link
Copy Markdown
Owner

Closes part of REG-624Unresolved imports for external dependencies — need leaf nodes (JS slice).

Problem (BEFORE)

533 / 1118 IMPORT nodes (48%) had no outgoing IMPORTS_FROM edge — they were graph dead-ends. The js_builtins derive pack only minted an EXTERNAL_MODULE for a hardcoded set of registered Node builtins (fs, path, …). Every npm package / scoped package / unregistered-builtin JS import (axios, react, @scope/pkg, node:zlib) was left leaf-less, so:

  • traceEffects.ts:321 (the EXTERNAL_MODULE → effects-db lookup arm) never fired for npm packages — the entire effects-db/packages/*.yaml corpus (axios, execa, glob, …) was unreachable;
  • "what external deps does this module use?" had no edge to traverse.

Evidence the pack was builtin-only: js_builtins_nodes.dl minted ext_module_node solely from the builtin_spec(...) ground facts (node builtins).

SPEC

Invariant restored: every project-EXTERNAL JS IMPORT has an IMPORTS_FROM edge to an EXTERNAL_MODULE leaf (no external dead-ends), without masking genuinely-broken internal imports.

Given a JS/TS file imports a bare specifier that is not a project module and not a registered builtin
When the js_builtins_nodes / js_builtins_edges derive packs run
Then an EXTERNAL_MODULE leaf (source="external", file="<external>") is minted and the IMPORT gets an additive IMPORTS_FROM (resolvedVia="external") to it.

Scope — an import is external iff ALL hold:

  1. specifier is not relative (no . prefix) and not absolute (no / prefix) — an unresolved relative import MUST stay a violation of the imports-resolve-to-module guarantee (REG-1166), never be masked by a fake leaf;
  2. not a registered builtin (builtin_spec) — those keep the richer nodejs-builtin leaf; excluded so EXTERNAL_MODULE:fs is never re-minted with a conflicting source;
  3. not already resolved to a MODULE by an earlier pack — js_module_imports / js_import_bindings run before js_builtins_* (per STDLIB_PACKS order) and commit their IMPORTS_FROM as EDB, so this drops workspace packages (@scope/util → a real project MODULE), which are internal.

A leading node: is normalized away (strip_prefix) so node:zlib and zlib collapse to one leaf (byte-identical sid ⇒ set-dedup).

AFTER (evidence)

cargo test --lib in packages/rfdb-server: 1408 passed, 0 failed. New/updated derive tests:

  • js_builtins_nodes_mints_external_leaf_for_npm_imports — positives: axios, @grafema/util, lodash/merge, node:zlibzlib(+ bare zlib share one leaf); negatives that mint nothing: ./local, /abs, registered fs, workspace @scope/internal (has IMPORTS_FROM→MODULE), .rs IMPORT.
  • js_builtins_edges_links_external_imports_to_leaves — external IMPORT → <external> leaf with resolvedVia="external"; relative + already-resolved emit no edge.
  • Updated js_builtins_nodes_mints_legacy_sids (4→5 node heads) and js_builtins_edges_joins_minted_endpoints (1→2 IMPORTS_FROM heads).
running 4 tests
test ...::js_builtins_edges_links_external_imports_to_leaves ... ok
test ...::js_builtins_edges_joins_minted_endpoints ... ok
test ...::js_builtins_nodes_mints_external_leaf_for_npm_imports ... ok
test ...::js_builtins_nodes_mints_legacy_sids ... ok

cargo check on grafema-orchestrator (pack description string): exit 0.

Adversarial review

Round A — correctness. Empty specifier excluded (neq(Spec,"") in js_imp). Bare "node:" mints nothing (neq(M,"")). Scoped @scope/pkg and subpath lodash/merge mint distinct leaves (verbatim names — package grouping is a follow-up). node:zlib+zlib dedup to one leaf. Duplicate imports across files → one leaf, one edge per IMPORT. Bug found & fixed mid-review: inline 2-arg \+ builtin_spec(Spec, _) did not exclude builtins (engine negation needs a single-column projection) → fs leaked into the external head; fixed by projecting is_builtin_spec(Spec) (the \+ dotted(C) discipline). Second bug found & fixed: the edge pack both produced IMPORTS_FROM and negated resolved_imp (which reads edge(_,_,"IMPORTS_FROM")) → NegationInCycle; resolved by dropping resolved_imp from the edge pack — leaf-existence (minted only by the node pack for unresolved externals) is itself the not-resolved gate.

Round B — structural. Prelude (is_builtin_spec/ext_other_imp/ext_other_name) is duplicated across the two .dl files — the established per-pack include pattern (builtin_spec/js_imp/bi_binding are already duplicated; documented in both headers). The node→edge two-pack coupling is the same contract as the existing ext_module_nodebi_import_edge.

Round C — class & invariants. No analysis-pipeline invariant violated: new nodes are leaves (file="<external>" → no MODULE), edges are additive. depends (runs after, maps endpoint→file→MODULE) derives no spurious DEPENDS_ON (no MODULE at <external>); full derive suite green confirms. Sibling latent — the same dead-end exists for Rust crates (rust_imports pack) and Haskell imports: out of scope here, already tracked by REG-687.

Blast radius / follow-ups

  • Rust-only change (.dl + cargo tests + one orchestrator description string); zero TS touched. The TS unit suite is orthogonal and requires pnpm build artifacts not present in this env.
  • The change lands in graph output only once the prebuilt rfdb-server binaries are rebuilt (release step).
  • Follow-ups (not scope-crept): npm/crate/hackage source tagging + subpath→package grouping (lodash/mergelodash); Rust/Haskell external leaves (REG-687).

⚠️ Not auto-merged — owner decides.

https://claude.ai/code/session_0129N2LmeRmrJQbTTbxXLPRJ


Generated by Claude Code

…G-624)

Close the graph dead-end where npm / scoped / unregistered-builtin JS imports
had no outgoing IMPORTS_FROM edge (REG-624: 48% of all IMPORT nodes). The
js_builtins derive pack previously minted an EXTERNAL_MODULE only for the
hardcoded registered Node builtins; every other external dependency import was
a leaf-less dead-end, so effects-db lookups (traceEffects.ts EXTERNAL_MODULE
arm) and "what external deps does this module use?" queries never resolved.

js_builtins_nodes.dl now mints an EXTERNAL_MODULE (source "external", file
"<external>") for every project-EXTERNAL import and js_builtins_edges.dl links
the IMPORT to it via an additive IMPORTS_FROM (resolvedVia "external").

Scope (an import is external iff ALL hold):
  - specifier is not relative ("." prefix) and not absolute ("/" prefix) — an
    unresolved RELATIVE import must stay a violation of the
    imports-resolve-to-module guarantee (REG-1166), never be masked;
  - not a registered Node builtin (those keep the richer nodejs-builtin leaf);
  - not already resolved to a MODULE by an earlier pack (js_module_imports /
    js_import_bindings run first and commit IMPORTS_FROM) — drops workspace
    packages, which are internal.
A leading "node:" is normalized away so node:zlib and zlib share one leaf.

source = "external"; richer npm/crate/hackage tagging and subpath→package
grouping (lodash/merge → lodash) are deferred follow-ups. Rust crate and
Haskell import leaves remain out of scope (REG-687).

Verified: cargo test --lib (1408 passed, 0 failed) incl. two new derive tests
covering the npm/scoped/subpath/node:-normalization positives and the
relative/absolute/builtin/workspace-resolved/.rs negatives.

https://claude.ai/code/session_0129N2LmeRmrJQbTTbxXLPRJ
@Disentinel

Copy link
Copy Markdown
Owner Author

🚫 QA precheck REJECT (attempt 1, base=main).

REJECT[zone]: diff touches engine zone (tier 3, Vadim/feat/datalog):
packages/rfdb-server/src/derive/stdlib.rs
packages/rfdb-server/src/derive/stdlib/js_builtins_edges.dl
packages/rfdb-server/src/derive/stdlib/js_builtins_nodes.dl
FLAG[rust]: .rs touched but no cargo on VM — compile gated by CI (REG-1177), QA agent must scrutinize

QA gate

Copy link
Copy Markdown
Owner Author

Both gate signals are expected for this change — neither is a fixable defect, so no code change:

REJECT[zone] (engine/datalog, tier-3) — correct & unavoidable. REG-624's fix is implemented on the modern in-engine derive path (the Wave-2b js_builtins_* packs that replaced the gated Haskell Builtins.hs resolver). The EXTERNAL_MODULE leaves + IMPORTS_FROM edges can only be minted in rfdb-server/src/derive/stdlib/*.dl (+ the cargo tests in stdlib.rs); there is no out-of-zone way to do this. So this is correctly a tier-3 change for @Disentinel to review/merge — which matches the PR's "not auto-merged, owner decides" stance. I am not force-merging.

FLAG[rust] (no cargo on QA VM → compile gated by CI) — covered, with evidence. Compilation is confirmed two ways:

  • Local: cargo test --lib in packages/rfdb-server1408 passed, 0 failed (incl. the 2 new derive tests + 2 updated head-count assertions); cargo check on grafema-orchestrator → exit 0.
  • CI (per REG-1177): the CI workflow run for 164726c = success (and the PR #449 QA workflow = success; RFDB Benchmarks = skipped/path-filtered).

Scope is Rust-only (3 derive files + 1 orchestrator description string); zero TS touched. Diff: +260/−5 across 4 files. Ready for owner review.


Generated by Claude Code

@Disentinel

Copy link
Copy Markdown
Owner Author

🚫 QA precheck REJECT (attempt 2, base=main).

REJECT[zone]: diff touches engine zone (tier 3, Vadim/feat/datalog):
packages/rfdb-server/src/derive/stdlib.rs
packages/rfdb-server/src/derive/stdlib/js_builtins_edges.dl
packages/rfdb-server/src/derive/stdlib/js_builtins_nodes.dl
FLAG[rust]: .rs touched but no cargo on VM — compile gated by CI (REG-1177), QA agent must scrutinize

QA gate

@Disentinel

Copy link
Copy Markdown
Owner Author

🚫 QA precheck REJECT (attempt 3, base=main).

REJECT[zone]: diff touches engine zone (tier 3, Vadim/feat/datalog):
packages/rfdb-server/src/derive/stdlib.rs
packages/rfdb-server/src/derive/stdlib/js_builtins_edges.dl
packages/rfdb-server/src/derive/stdlib/js_builtins_nodes.dl
FLAG[rust]: .rs touched but no cargo on VM — compile gated by CI (REG-1177), QA agent must scrutinize

QA gate

@Grafema-AI-Ops

Copy link
Copy Markdown
Collaborator

🚫 QA precheck REJECT (attempt 4, base=main).

REJECT[zone]: diff touches engine zone (tier 3, Vadim/feat/datalog):
packages/rfdb-server/src/derive/stdlib.rs
packages/rfdb-server/src/derive/stdlib/js_builtins_edges.dl
packages/rfdb-server/src/derive/stdlib/js_builtins_nodes.dl
FLAG[rust]: .rs touched but no cargo on VM — compile gated by CI (REG-1177), QA agent must scrutinize

QA gate

@Grafema-AI-Ops

Copy link
Copy Markdown
Collaborator

🚫 QA precheck REJECT (attempt 5, base=main).

REJECT[zone]: diff touches engine zone (tier 3, Vadim/feat/datalog):
packages/rfdb-server/src/derive/stdlib.rs
packages/rfdb-server/src/derive/stdlib/js_builtins_edges.dl
packages/rfdb-server/src/derive/stdlib/js_builtins_nodes.dl
FLAG[rust]: .rs touched but no cargo on VM — compile gated by CI (REG-1177), QA agent must scrutinize

QA gate

@Disentinel
Disentinel merged commit c777ef0 into main Jun 17, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants