Skip to content
CI

fix(db): fold seven raw-SQL identity tables into renameRepositoryIdentity #14731

Sign in to view logs

fix(db): fold seven raw-SQL identity tables into renameRepositoryIdentity

fix(db): fold seven raw-SQL identity tables into renameRepositoryIdentity #14731

Workflow file for this run

.github/workflows/ci.yml at 4f90078
name: CI
on:
pull_request:
# Explicit list because the default (opened/synchronize/reopened) omits ready_for_review -- once the
# heavy jobs below start skipping draft PRs, marking a PR ready must itself trigger a real CI run, not
# wait for the next push.
types: [opened, synchronize, reopened, ready_for_review]
push:
branches:
- main
permissions:
contents: read
concurrency:
# Keep pull requests in one ref-scoped group so newer commits cancel superseded PR runs.
# Add github.sha for push runs so distinct main commits do not cancel each other's validation.
group: ci-${{ github.ref }}-${{ github.event_name == 'push' && github.sha || 'pr' }}
# NB: keep this a literal boolean. An expression here (cancel-in-progress: ${{ ... }}) made GitHub
# fail the workflow at startup (startup_failure), so `validate` never reported.
cancel-in-progress: true
jobs:
# Detect which areas a PR touches so the heavy jobs can skip when irrelevant.
# On push to main everything runs regardless (keeps the coverage baseline solid).
changes:
name: changes
# Just actions/checkout + git diff --check + dorny/paths-filter -- no build/test work, and it runs FIRST
# to decide whether validate-code needs to run at all, so gating it behind the same contended self-hosted
# queue it's meant to protect was circular (#2507).
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
backend: ${{ steps.filter.outputs.backend }}
backendConfig: ${{ steps.filter.outputs.backendConfig }}
ui: ${{ steps.filter.outputs.ui }}
uiContract: ${{ steps.filter.outputs.uiContract }}
mcp: ${{ steps.filter.outputs.mcp }}
mcpCliHarness: ${{ steps.filter.outputs.mcpCliHarness }}
engine: ${{ steps.filter.outputs.engine }}
discoveryIndex: ${{ steps.filter.outputs.discoveryIndex }}
miner: ${{ steps.filter.outputs.miner }}
minerTestHarness: ${{ steps.filter.outputs.minerTestHarness }}
rees: ${{ steps.filter.outputs.rees }}
controlPlane: ${{ steps.filter.outputs.controlPlane }}
observability: ${{ steps.filter.outputs.observability }}
steps:
# Debounce rapid re-pushes to an open PR: every job below (validate-code, all 3 validate-tests
# shards, validate-tests-merge, security) has `needs: changes` in its dependency chain, so none of
# them get queued for a runner until this job finishes -- and this job lives in the SAME PR-ref-scoped
# concurrency group as everything else (top of file), so a newer push cancels this job outright while
# it's asleep, well before it ever reaches "Filter changed paths". That means a rapid burst of pushes
# only ever lets the LAST push's `changes` run actually complete and unblock the expensive fan-out --
# every earlier push in the burst pays for one cheap, cancelled `changes` job instead of the full
# ~7-job-slot fan-out starting, queueing, and then getting killed mid-flight. No custom "is this SHA
# still current" check is needed: cancel-in-progress already does that for free the moment a newer
# push lands. Skipped on push (github.event_name == 'push', i.e. a merge to main) -- that's a
# once-per-merge event, not iterative amendment, and main's coverage signal shouldn't be delayed.
- name: Debounce rapid pushes
if: ${{ github.event_name == 'pull_request' }}
run: sleep 45
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
# `git diff --check` with no args compares the working tree against the index -- on this checkout's
# default shallow, single-commit fetch they're always identical immediately after checkout, so this
# unconditionally passed regardless of what the PR's actual diff contained (a false-green: the step
# promised a whitespace check that never fired). Fetch just the one comparison SHA (still shallow,
# cheap) and diff against it directly instead. On push (github.event.before can be the null SHA for
# a branch's first push -- not reachable for `main` itself, but kept defensive) or when the fetch
# target isn't resolvable, skip rather than fail the whole pipeline over a best-effort check.
- name: Check whitespace
run: |
BASE_SHA="${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || github.event.before }}"
if [ -z "$BASE_SHA" ] || [ "$BASE_SHA" = "0000000000000000000000000000000000000000" ]; then
echo "No comparison SHA available (e.g. branch's first push) -- skipping."
exit 0
fi
git fetch --depth=1 origin "$BASE_SHA"
git diff --check "$BASE_SHA" HEAD
- name: Filter changed paths
id: filter
uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 # v4
with:
filters: |
backend:
- 'src/**'
- 'test/**'
- 'vitest*.config.ts'
- 'tsconfig*.json'
- 'package.json'
- 'package-lock.json'
- 'scripts/**'
- 'migrations/**'
- '.github/workflows/**'
# .github/actions/** (composite actions, e.g. setup-workspace) is workflow-adjacent
# infrastructure the same way .github/workflows/** above is -- without this, a PR editing
# ONLY a composite action file (touching nothing else backend matches) would skip both
# "Lint workflows" and "Lint composite actions" below entirely, with zero CI signal on a
# broken edit until the next push-triggered full run.
- '.github/actions/**'
- 'wrangler.jsonc'
- 'worker-configuration.d.ts'
- '.loopover.yml'
- '.nvmrc'
- '.release-please-manifest.json'
- 'release-please-config.json'
# The subset of `backend` above that ISN'T plain src/**+test/** TypeScript -- config, migrations,
# and workflow/tooling files with no import-graph edge a test file could ever have to them. Exists
# solely to gate scoped test selection (#ci-scoped-test-selection, validate-tests below): that
# verification covered only plain .ts source-file diffs replayed against real historical commits,
# never a migration or a package.json/tsconfig/workflow change, and `--changed`'s dependency
# tracing has no mechanism to connect a non-JS/TS file to the tests that would need to catch a
# regression in it (a migration's coverage comes from a migration-runner test that reads the
# migrations/ directory at runtime, not from a static `import` of the changed .sql file). A PR
# matching this stays on the full unscoped suite even if it also touches src/**.
backendConfig:
- 'vitest*.config.ts'
- 'tsconfig*.json'
- 'package.json'
- 'package-lock.json'
- 'scripts/**'
- 'migrations/**'
- '.github/workflows/**'
- '.github/actions/**'
- 'wrangler.jsonc'
- 'worker-configuration.d.ts'
- '.loopover.yml'
- '.nvmrc'
- '.release-please-manifest.json'
- 'release-please-config.json'
observability:
- 'grafana/dashboards/**'
- 'prometheus/rules/**'
# The UI's own app/extension code -- triggers the FULL toolchain (lint/typecheck/test/build).
# A dependency bump (package.json/package-lock.json) stays here too since it can break the UI
# build or types in ways only that full toolchain would catch.
ui:
- 'apps/loopover-ui/**'
- 'apps/loopover-miner-ui/**'
- 'apps/loopover-extension/**'
- 'apps/loopover-miner-extension/**'
- 'packages/loopover-ui-kit/**'
- 'scripts/build-extension.ts'
- 'scripts/build-miner-extension.ts'
- 'package.json'
- 'package-lock.json'
# Backend changes that can drift the OpenAPI contract the UI type-checks against, but say
# nothing about the UI app's OWN code -- only the lightweight drift check needs to run, not
# ui:lint/typecheck/test/build (#ci-scope). If a backend PR's change IS UI-relevant, regenerating
# `apps/loopover-ui/public/openapi.json` per CONTRIBUTING.md lands a change under `ui` above,
# which correctly re-triggers the full toolchain.
uiContract:
- 'src/**'
- 'scripts/write-ui-openapi.ts'
mcp:
# Self-contained package: build is `node --check` on its own files
# and the pack check only inspects the tarball. Root src/ cannot
# affect it, so it is intentionally NOT a trigger here.
- 'packages/loopover-mcp/**'
- 'scripts/check-mcp-package.ts'
- 'package-lock.json'
engine:
- 'packages/loopover-engine/**'
- 'package-lock.json'
miner:
- 'packages/loopover-miner/**'
- 'scripts/check-miner-package.ts'
- 'package-lock.json'
# No dedicated build/pack script (unlike mcp/engine/miner above) -- discovery-index is a normal
# vitest-covered workspace package (packages/discovery-index/src/**/*.ts is in vitest.config.ts's
# coverage.include), so this filter only needs to feed validate-code's overall gate + validate-tests
# below, not a standalone job.
discoveryIndex:
- 'packages/discovery-index/**'
- 'package-lock.json'
# 6 of the 7 MCP CLI-cluster test files, and ONLY these 6, are truly self-contained w.r.t. root
# src/**: verified by direct-import inspection that test/unit/mcp-cli-*.test.ts (5 files) and
# their shared test/unit/support/mcp-cli-harness.ts import nothing but node:* builtins + vitest,
# and mcp-discovery.test.ts spawns packages/loopover-mcp/bin/loopover-mcp.ts (gitignored
# compiled output; Node's own type-stripping runs the .ts directly, no build needed) as a real
# subprocess via StdioClientTransport -- none of the 6 ever load root src/ in-process. This mirrors
# the `mcp` filter's own established trust boundary above (a self-contained package build). They are
# NOT self-contained w.r.t. packages/loopover-engine/**, though: loopover-mcp now has a real
# dependency on @loopover/engine (isTestFile/isCodeFile), which the bin subprocess and
# local-branch.js load at runtime -- hence that path below, alongside the mcp package's own.
# test/unit/mcp-output-schemas.test.ts is DELIBERATELY NOT in this filter: unlike the other 6, it
# imports src/mcp/server.ts in-process, and server.ts alone directly imports ~40 other src/ modules
# (src/github/app.ts, src/signals/slop.ts, src/settings/autonomy.ts, src/orb/analytics.ts, and
# most of src/services/** + src/signals/**), so its real dependency surface is practically all of
# `backend`. An earlier version of this filter hand-picked a handful of "the src files that affect
# it" for that file too -- that guess missed dozens of server.ts's actual direct imports, which was
# a reachable CI blind spot (a backend change outside the guessed list could regress MCP server
# behavior with zero test rerun to catch it). Enumerating that surface by hand is not safely
# possible, so mcp-output-schemas.test.ts is simply never excluded -- it always runs whenever
# `backend` does, at the cost of keeping its slice of the suite's runtime unconditional.
mcpCliHarness:
- 'vitest*.config.ts'
- 'tsconfig*.json'
- 'package.json'
- 'package-lock.json'
- 'scripts/**'
- 'migrations/**'
- '.github/workflows/**'
- 'packages/loopover-mcp/**'
- 'packages/loopover-engine/**'
- 'test/helpers/**'
- 'test/unit/mcp-cli-*.test.ts'
- 'test/unit/mcp-discovery.test.ts'
- 'test/unit/support/mcp-cli-harness.ts'
rees:
- 'review-enrichment/**'
- '.github/workflows/ci.yml'
controlPlane:
- 'control-plane/**'
- '.github/workflows/ci.yml'
# Both miner-package test files are self-contained w.r.t. root src/**, the same trust boundary as
# mcpCliHarness above: check-miner-package.test.ts only spawns scripts/check-miner-package.ts as a
# real subprocess (node:child_process + vitest, nothing else), and miner-calibration-types.test.ts
# only imports the scaffolded packages/loopover-miner/lib/calibration.ts -- neither ever loads
# root src/ in-process. Mirrors mcpCliHarness's filter shape for the same reason (tooling/config
# changes that could plausibly affect how these tests run, plus the miner package + the test files
# themselves), swapping in the miner-specific package and test files.
minerTestHarness:
- 'vitest*.config.ts'
- 'tsconfig*.json'
- 'package.json'
- 'package-lock.json'
- 'scripts/**'
- 'migrations/**'
- '.github/workflows/**'
- 'packages/loopover-miner/**'
- 'test/unit/check-miner-package.test.ts'
- 'test/unit/miner-calibration-types.test.ts'
# Path-aware validation. Keep this as one job so a PR uses one dependency
# install for the fast checks (drift/typecheck/build/UI toolchain). The
# expensive full-suite coverage run is sharded out to validate-tests below
# (#ci-shard-coverage) -- it dominated wall-clock time (~9-10 of ~11 minutes
# on a typical backend PR) and doesn't benefit from sharing an install with
# these much-faster steps. Run on GitHub-hosted runners while the
# self-hosted review stack is CPU constrained.
validate-code:
name: validate-code
needs: changes
# draft != true also gates push runs correctly: github.event.pull_request is unset there, so the
# property access evaluates to null, and null != true is true.
if: ${{ github.event_name == 'push' || (github.event.pull_request.draft != true && (needs.changes.outputs.backend == 'true' || needs.changes.outputs.mcp == 'true' || needs.changes.outputs.engine == 'true' || needs.changes.outputs.discoveryIndex == 'true' || needs.changes.outputs.miner == 'true' || needs.changes.outputs.rees == 'true' || needs.changes.outputs.controlPlane == 'true' || needs.changes.outputs.ui == 'true' || needs.changes.outputs.observability == 'true')) }}
runs-on: ubuntu-latest
timeout-minutes: 45
env:
VITE_LOOPOVER_API_ORIGIN: https://api.loopover.ai
steps:
# Shallow (default depth): this job no longer uploads to Codecov -- that moved to validate-tests
# (#ci-shard-coverage) -- so it has no reason to fetch full history anymore.
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
# See .github/actions/setup-workspace for what this covers (npmrc, Node, node_modules cache) --
# the checkout above stays a separate, explicit step since a local `uses: ./path` action needs
# the repo already on disk to even find its own action.yml.
- name: Setup workspace
uses: ./.github/actions/setup-workspace
# apps/loopover-ui/.source (fumadocs-mdx codegen) is regenerated by "Generate docs content
# collections" below, gated on push||ui==true -- nothing between here and there reads it (docs-drift,
# env-reference, manifest/engine-parity/branding-drift checks all read raw source files directly, and
# the root Typecheck step's tsconfig.json excludes apps/loopover-ui entirely), so it doesn't need to
# be regenerated unconditionally this early too (a prior version of this step duplicated that one,
# added independently and unconditionally before either author noticed the other already existed).
- name: Lint workflows
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' }}
run: npm run actionlint
# actionlint (above) does not cover .github/actions/**/action.yml at all -- confirmed a genuine,
# long-standing upstream limitation (rhysd/actionlint#46 and #401, open since 2021), not something
# fixable by a different invocation: even the raw actionlint binary, run directly with no wrapper,
# treats any file it's given as a workflow and errors on runs/inputs/outputs as unexpected
# top-level keys. This validates composite action files against GitHub's own official
# action-metadata JSON Schema instead -- real structural validation, not the full expression-
# context linting actionlint does for workflows, which genuinely doesn't exist anywhere for
# action.yml files.
- name: Lint composite actions
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' }}
run: npm run lint:composite-actions
- name: Check migrations
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' }}
run: npm run db:migrations:check
# Guards against a DIFFERENT drift class than db:migrations:check's own column-collision check (#2551,
# which only catches two migration FILES independently adding the same column): src/db/schema.ts's
# DECLARED shape silently drifting from what migrations/ ACTUALLY produces when replayed (#2565).
- name: Schema-vs-migrations drift check
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' }}
run: npm run db:schema-drift:check
# Guards against the same staleness-drift class as db:migrations:check/ui:openapi:check: two PRs that
# each independently add a wrangler.jsonc binding can both pass CI in isolation, then merge sequentially
# and leave a stale committed worker-configuration.d.ts with zero prior gate signal (#2557).
- name: cf-typegen drift check
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' }}
run: npm run cf-typegen:check
# Same drift class as cf-typegen/schema-drift above, just a different generated artifact: this file's
# firstReference line numbers point into src/selfhost/ai.ts, so any backend change that shifts those
# lines (not just one that adds/removes an env var) can make it stale. Was previously enforced ONLY by
# the local `npm run test:ci` aggregate script, never by this workflow -- went stale twice in one day
# (2026-07-04) with zero CI signal until someone happened to run the full local script. Gated on `ui`
# too, not just `backend`: the generated file itself lives under apps/loopover-ui/**, so a PR that
# hand-edits it (or a docs page docs:drift-check below cross-checks) without touching backend source
# would otherwise never re-run this check.
- name: Selfhost env-reference drift check
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' || needs.changes.outputs.ui == 'true' }}
run: npm run selfhost:env-reference:check
# Miner/AMS twin of the selfhost check above -- same generated-artifact-drift class, same
# local-only-until-now gap (an `env.SOMETHING` read added under packages/loopover-miner/**
# with no CI signal that packages/loopover-miner/docs/env-reference.md and
# apps/loopover-ui/src/lib/ams-env-reference.ts had gone stale). Confirmed harmful live: this
# exact gap let a stale env-reference block an unrelated MCP npm release, since the release
# workflow was the only place this check ever ran. Gated on `ui` too, same reason as its sibling:
# the generated UI-side file lives under apps/loopover-ui/**. Also gated on `engine`:
# DEFAULT_SOURCE_ROOTS in generate-env-reference.mjs explicitly includes
# packages/loopover-engine/src/miner (the coding-agent driver's env vars live there, not under
# packages/loopover-miner itself), so an engine-only PR touching that directory used to skip this
# check entirely.
- name: Miner env-reference drift check
if: ${{ github.event_name == 'push' || needs.changes.outputs.miner == 'true' || needs.changes.outputs.ui == 'true' || needs.changes.outputs.engine == 'true' }}
run: npm run miner:env-reference:check
# Same generated-artifact-drift class, extracted from src/github/commands.ts's command catalogs (#3046).
# Also local-only until now; same backend-or-ui gating rationale as the step above.
- name: Command reference drift check
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' || needs.changes.outputs.ui == 'true' }}
run: npm run command-reference:check
# Cross-checks docs pages against source-of-truth enumerable surfaces (feature flags, commands, gate
# modes) -- see the script's own header comment. Also local-only until now; gated on `ui` as well as
# `backend` since the docs pages it validates are themselves UI-side content that can drift on their
# own (a docs-only edit that renames/drops a documented flag/command/gate-mode) with no backend change
# to trigger a re-check otherwise. Also gated on `engine`, unlike its two siblings above (`Selfhost
# env-reference drift check`/`Command reference drift check`): this script's readFileSync reads
# packages/loopover-engine/src/focus-manifest.ts directly to cross-check FocusManifest fields against
# .loopover.yml.example, a dependency the other two generated-artifact-drift checks don't share, so
# deliberately NOT kept uniform with them here (test/unit/ci-generated-artifact-drift-checks.test.ts
# asserts backend||ui is a SUBSET of every condition, not that all three are byte-identical).
- name: Docs drift check
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' || needs.changes.outputs.ui == 'true' || needs.changes.outputs.engine == 'true' }}
run: npm run docs:drift-check
# Cross-checks the bundled fallback YAML in src/config/loopover-repo-focus-manifest.ts against the
# real root .loopover.yml -- see the script's own header comment. Same local-only-until-now gap as
# the drift checks above: nothing in this workflow previously ran it, so the two could silently
# diverge with zero CI signal. .loopover.yml itself is now part of the `backend` path filter above
# specifically so an edit to ONLY that file still re-triggers this job.
- name: Manifest drift check
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' }}
run: npm run manifest:drift-check
# Mechanical drift tripwire for the hand-duplicated src/{review,settings,signals} <-> loopover-engine
# twin files, plus a version-skew check on the installed @loopover/engine (#4260). Same
# local-only-until-now gap as the drift checks above. Also gated on `miner`: the reverse-direction
# version pin this script checks, packages/loopover-miner/expected-engine.version, lives under the
# miner package, not engine or backend -- a PR editing only that one pin file used to skip its own
# consistency check entirely.
- name: Engine-parity drift check
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' || needs.changes.outputs.engine == 'true' || needs.changes.outputs.miner == 'true' }}
run: npm run engine-parity:drift-check
# Guards against the "gittensory" branding creeping back into runtime source after the LoopOver
# rebrand -- see the script's own header comment (#6786 is the concrete incident this was written
# for). Scoped broadly (src/** plus every workspace package's bin/lib/src/scripts dirs), so it's
# gated broadly to match; same local-only-until-now gap as the drift checks above. Also gated on
# `discoveryIndex`: BRANDING_DRIFT_PATHSPECS includes "packages/*/src/**/*.ts", which matches
# packages/discovery-index/src/**.
- name: Branding drift check
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' || needs.changes.outputs.mcp == 'true' || needs.changes.outputs.engine == 'true' || needs.changes.outputs.miner == 'true' || needs.changes.outputs.ui == 'true' || needs.changes.outputs.discoveryIndex == 'true' }}
run: npm run branding-drift:check
# Every engines.node in this repo used to be open-ended (">=22.0.0", no upper bound), so npm never
# warned or blocked a Node 23+ install locally -- exactly how the Node 26 jsdom/localStorage gap
# (nodejs/node#60303) went undetected in apps/loopover-miner-ui (#7592/#7597), then again in
# apps/loopover-ui + packages/loopover-ui-kit, with zero CI signal either time (CI itself was never
# at risk: setup-workspace pins node-version-file: .nvmrc). This is the same local-only-until-now gap
# as the drift checks above. Gated the same broad way as branding-drift-check: every workspace group
# that can declare its own engines.node (mcp/engine/miner/ui/discoveryIndex) plus backend, which now
# also covers a `.nvmrc`-only bump.
- name: Engines/.nvmrc sync check
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' || needs.changes.outputs.mcp == 'true' || needs.changes.outputs.engine == 'true' || needs.changes.outputs.miner == 'true' || needs.changes.outputs.ui == 'true' || needs.changes.outputs.discoveryIndex == 'true' }}
run: npm run engines-nvmrc:check
# .release-please-manifest.json is release-please's own state file -- it normally keeps this in
# sync itself when its generated Release PR merges, but the documented human-override path for
# a stuck release tag (publish-mcp.yml's own header comment) bumps package.json directly with
# no way to touch release-please's state. That gap went uncaught with zero CI signal, then
# surfaced days later as release-please re-proposing an already-published version as a brand-new
# Release PR, which fails on publish (#7086/#7087). `.release-please-manifest.json` and
# `release-please-config.json` are now part of the `backend` path filter above so an edit to
# either alone still re-triggers this job.
- name: Release-please manifest sync check
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' || needs.changes.outputs.mcp == 'true' || needs.changes.outputs.engine == 'true' || needs.changes.outputs.miner == 'true' || needs.changes.outputs.ui == 'true' }}
run: npm run release-manifest:sync:check
# Not gated on `backend`: read the full script -- it only ever reads grafana/dashboards/*.json and
# prometheus/rules/alerts.yml, both fully covered by the `observability` filter above, with zero
# reference to any src/, test/, or other backend path. The `backend` clause had no structural
# justification and made this run on every backend PR (the highest-volume PR shape in the repo) for
# nothing.
- name: Validate observability configs
if: ${{ github.event_name == 'push' || needs.changes.outputs.observability == 'true' }}
env:
NODE_OPTIONS: ""
run: node --experimental-strip-types scripts/validate-observability-configs.ts
# Runs ahead of Typecheck AND "Test with coverage" (#ci-engine-build-order): src/mcp/find-opportunities.ts
# (root backend, since #2281/#3985) imports packages/loopover-miner/lib/opportunity-fanout.js --
# a .js-suffixed specifier esbuild/Vite resolve straight to the real (gitignored, uncompiled)
# .ts source, same as @loopover/engine's own src/**, so THIS half needs no build at all -- which
# itself imports @loopover/engine THE PACKAGE, though, via a bare `@loopover/engine` specifier
# elsewhere in the chain, and THAT resolves through node_modules to dist/. That package's dist/ is
# gitignored and only exists after this build step, so ANY backend typecheck or test run needs it built
# first, not just a PR that touches packages/loopover-engine/** directly (this step's original, too-
# narrow trigger). Originally ran after coverage (broke test:coverage on PRs that didn't touch the
# engine) and after Typecheck (broke a real, non-vi.mock `import type` from the package specifier with
# TS2307 in CI while passing locally against a stale leftover dist/ -- first hit by PR #5082). Now ahead
# of both.
#
# Turborepo (turbo.json) owns the actual cache correctness here (content-hashes @loopover/engine's own
# inputs independently of whatever this restore serves) -- the .turbo/cache restore below is purely an
# accelerator, same accumulating-cache shape as "Restore TypeScript incremental build cache" below (a
# run_id-suffixed save key + prefix restore-keys fallback, not the hit-or-miss node_modules pattern),
# since turbo's cache directory grows additively rather than being wholesale-replaced per lockfile hash.
# This is also what lets each validate-tests shard (which independently needs this same package built)
# skip rebuilding it from scratch when a warm cache from this job or an earlier run is available.
#
# Gate includes mcp/miner (not just backend/engine/ui) even though "Build engine package" below stays
# scoped to backend/engine/ui: "Build MCP"/"Build miner CLI" further down invoke turbo too, and since
# both packages declare @loopover/engine as a package.json dependency, turbo's own ^build resolution
# builds engine as their transitive dependency regardless of whether this explicit step ran. Without
# this wider gate, an mcp- or miner-only PR would still get a correct build (turbo builds it inline
# with no cache to hit) -- just a cold one, and with no save step to persist it either.
#
# The matching "Save Turborepo cache" step is NOT alongside this one -- it's moved all the way down
# to after "UI build" (the last step in this job that invokes turbo), since actions/cache/save takes
# one synchronous snapshot at the moment it runs and every later turbo-routed step in between (Build
# MCP, Build miner CLI, Build UI-kit package, UI lint/typecheck, Extension lint/typecheck, UI build)
# would otherwise write to .turbo/cache AFTER the snapshot already happened, so none of their cache
# entries would ever persist cross-run. Confirmed this was a real gap (not just theoretical) by
# reading the actual step order before moving it.
#
# Key uses a "turbo-code-" prefix, distinct from validate-tests' "turbo-tests-" prefix below, even
# though both jobs cache the same .turbo/cache path and both build @loopover/engine -- they used to
# share one "turbo-" prefix, which meant all 7 jobs in a single run (this one plus 6 validate-tests
# shards) raced for the SAME cache key. actions/cache/save no-ops on a duplicate key within a run (see
# the comment on validate-tests' own Save step below), and a validate-tests shard reaches its Save
# step after only ~6 steps versus this job's ~20+, so a shard almost always won the race -- meaning
# the richer cache this job builds (engine + mcp + miner + ui-kit + ui + extension) was very likely
# silently discarded in favor of a shard's much leaner engine-only save, defeating most of the point
# of the Save-repositioning fix above. Splitting the prefixes gives each job's Restore/Save pair its
# own independent lineage instead of racing.
- name: Restore Turborepo cache
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' || needs.changes.outputs.engine == 'true' || needs.changes.outputs.ui == 'true' || needs.changes.outputs.mcp == 'true' || needs.changes.outputs.miner == 'true' }}
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: .turbo/cache
key: turbo-code-${{ hashFiles('package-lock.json') }}-${{ github.run_id }}
restore-keys: |
turbo-code-${{ hashFiles('package-lock.json') }}-
- name: Build engine package
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' || needs.changes.outputs.engine == 'true' || needs.changes.outputs.ui == 'true' }}
run: npx turbo run build --filter=@loopover/engine
# .tsbuildinfo mutates every run (tsc's own incremental state), unlike node_modules above which is
# immutable per lockfile -- so this needs the run_id-suffixed-key + restore-keys-prefix pattern (always
# creates a new cache entry to save into, restore falls back to the most recent matching prefix) rather
# than the hit-or-miss pattern node_modules uses. Measured ~13.5s cold vs ~3.6s warm locally; tsc
# verifies each file's content hash before trusting cached state, so a fresh checkout's reset mtimes
# can't cause a false cache hit that would mask a real type error.
- name: Restore TypeScript incremental build cache
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' }}
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: .tsbuildinfo
key: tsbuildinfo-${{ hashFiles('tsconfig.json', 'package-lock.json') }}-${{ github.run_id }}
restore-keys: |
tsbuildinfo-${{ hashFiles('tsconfig.json', 'package-lock.json') }}-
# Deliberately NOT routed through Turborepo's //#typecheck task, unlike every other build/lint/
# typecheck step in this job -- investigated and rejected, not just skipped. test/** (tsconfig.json's
# own "include") reaches via relative imports into six other workspace packages/apps (see turbo.json's
# //#typecheck comment for the full audit), so a turbo.json inputs list correct enough to avoid a
# stale-cache-masks-a-real-error bug here would need to track that sprawling, growing cross-package
# surface by hand. tsc's own --incremental engine (below) already tracks the exact real dependency
# graph natively and precisely, with zero drift risk, and already delivers the actual measured
# speedup (~13.5s cold vs ~3.6s warm) -- wrapping it in turbo would layer a cruder, hand-maintained
# approximation on top of a mechanism that's already strictly more correct for this specific task.
# Gated on engine/mcp/miner too, not just backend: root src/** genuinely imports across those three
# package boundaries (~57 files import @loopover/engine directly; src/mcp/find-opportunities.ts
# imports packages/loopover-miner/lib/opportunity-fanout.js and opportunity-ranker.js;
# src/services/mcp-compatibility.ts imports packages/loopover-mcp/package.json), so a type-only
# regression introduced in any of those three packages, on a PR that touches only that package, used
# to skip this check entirely -- not caught until the push-triggered run on main, after this repo's
# auto-merge gate had already merged it. tsc's own --incremental engine (unaffected by this widening;
# see its own comment two steps up) already tracks the real graph regardless of which trigger fired.
- name: Typecheck
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' || needs.changes.outputs.engine == 'true' || needs.changes.outputs.mcp == 'true' || needs.changes.outputs.miner == 'true' }}
run: npm run typecheck
- name: Save TypeScript incremental build cache
if: ${{ !cancelled() && (github.event_name == 'push' || needs.changes.outputs.backend == 'true') }}
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: .tsbuildinfo
key: tsbuildinfo-${{ hashFiles('tsconfig.json', 'package-lock.json') }}-${{ github.run_id }}
# Same engine/mcp/miner gap as Typecheck above, and more consequential here: test/workers/worker-
# runtime.test.ts (vitest.config.ts excludes it from the main "Test with coverage" run below, so
# nothing else in CI runs it) imports src/index and src/api/routes, which themselves import
# find-opportunities.ts (miner) and mcp-compatibility.ts (mcp) -- this is the only test in all of CI
# that boots the real Cloudflare Workers runtime end-to-end, and it used to never run at all on an
# engine/mcp/miner-only PR.
- name: Worker runtime tests
if: ${{ github.event_name == 'push' || needs.changes.outputs.backend == 'true' || needs.changes.outputs.engine == 'true' || needs.changes.outputs.mcp == 'true' || needs.changes.outputs.miner == 'true' }}
run: npm run test:workers
# loopover-mcp now depends on @loopover/engine for real (isTestFile/isCodeFile), so a
# PR that only touches packages/loopover-engine/** must also rebuild + pack-check the mcp package,
# not just the mcp filter's own (deliberately narrower) path list.
#
# Routed through Turborepo (turbo.json's @loopover/mcp#build, dependsOn ["^build"]) rather than the
# old npm run build:mcp -- turbo resolves @loopover/engine as its topological dependency (mcp's own
# package.json depends on @loopover/engine) and reuses the "Restore Turborepo cache" above, so an
# engine-only PR's already-built engine is never rebuilt a second time here.
# packages/loopover-mcp ships .ts source only -- no compiled .js/.d.ts is committed (gitignored;
# see vitest.config.ts's resolveMinerMcpTsSourcePlugin for how tests run against the .ts directly
# with no build step). This "Build MCP" step still exists because the PACKAGE ITSELF has to ship
# runnable .js in its published npm tarball (npm install -g gives you plain Node, no TS toolchain),
# and "MCP package check" right below needs that real build output on disk to validate.
- name: Build MCP
if: ${{ github.event_name == 'push' || needs.changes.outputs.mcp == 'true' || needs.changes.outputs.engine == 'true' }}
run: npx turbo run build --filter=@loopover/mcp
- name: MCP package check
if: ${{ github.event_name == 'push' || needs.changes.outputs.mcp == 'true' || needs.changes.outputs.engine == 'true' }}
run: npm run test:mcp-pack
# Same "ships .ts only, this build is for the published npm tarball + the pack-check below, not for
# tests" story as MCP's own "Build MCP" step above. Invokes turbo.json's @loopover/miner#build:tsc
# (cache: false -- historically to avoid a cache restore stomping this package's hand-written .js
# files while the #7290 migration was still in-flight; now that #7317 closed it out, every bin/lib
# file is compiler-owned and that original risk no longer applies, but re-enabling caching here is a
# separate change this PR doesn't make) and @loopover/miner#build:verify (real caching win: skips
# re-`node --check`-ing all 121 bin/lib files when neither changed) DIRECTLY as their own task names,
# deliberately NOT via the aggregate `@loopover/miner#build` task (which exists for standalone/local
# `npm run build` callers) -- that aggregate's own script is `npm run build:tsc && npm run
# build:verify`, i.e. it re-invokes both of these exact scripts a second time even when turbo already
# ran them as cached prerequisite tasks (declaring a task as a dependsOn prerequisite doesn't replace
# a parent task's own script body). Calling the aggregate here would silently double both the tsc
# compile and the syntax check on every run, negating the caching win this comment describes --
# confirmed by reproducing the double execution locally before switching to this direct form. This
# also finally activates the build:tsc/build:verify split from Phase 0, which no ci.yml step
# exercised until now.
- name: Build miner CLI
if: ${{ github.event_name == 'push' || needs.changes.outputs.miner == 'true' || needs.changes.outputs.engine == 'true' }}
run: npx turbo run build:tsc build:verify --filter=@loopover/miner
# loopover-miner depends on @loopover/engine for real too (same relationship as loopover-mcp above
# -- packages/loopover-miner/package.json lists it as a dependency), so this and "Build miner CLI"
# above now also trigger on an engine-only PR, matching "Build MCP"/"MCP package check"'s existing
# engine coupling. Pre-existing asymmetry, not introduced by this change -- fixed here while already
# touching these exact lines.
- name: Miner package check
if: ${{ github.event_name == 'push' || needs.changes.outputs.miner == 'true' || needs.changes.outputs.engine == 'true' }}
run: npm run test:miner-pack
# scripts/check-miner-deployment-docs.ts reads packages/loopover-engine/src/miner/**/*.ts
# (ENGINE_MINER_DIR) to build the registered-commands list checked against DEPLOYMENT.md, so an
# engine-only PR touching that directory used to skip this audit entirely.
- name: Miner deployment docs audit
if: ${{ github.event_name == 'push' || needs.changes.outputs.miner == 'true' || needs.changes.outputs.engine == 'true' }}
run: npm run test:miner-deployment-docs-audit
# review-enrichment is not an npm workspace member (its own package-lock.json), so it needs its own
# cache entry -- same restore/save-after-success pattern and fork/trusted key split as the root
# install above, for the same reasons.
- name: Restore review-enrichment node_modules cache
id: rees-node-modules-cache
if: ${{ github.event_name == 'push' || needs.changes.outputs.rees == 'true' }}
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: review-enrichment/node_modules
# Same Node-version and manifest guards as the root cache key above -- REES runs on the same
# pinned .nvmrc and has its own package.json lifecycle/install validation.
key: npm-rees-${{ (github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == true) && 'fork' || 'trusted' }}-${{ hashFiles('.nvmrc') }}-${{ hashFiles('review-enrichment/package.json', 'review-enrichment/package-lock.json') }}
- name: REES install
if: ${{ (github.event_name == 'push' || needs.changes.outputs.rees == 'true') && steps.rees-node-modules-cache.outputs.cache-hit != 'true' }}
run: npm run rees:install
- name: Save review-enrichment node_modules cache
if: ${{ (github.event_name == 'push' || needs.changes.outputs.rees == 'true') && steps.rees-node-modules-cache.outputs.cache-hit != 'true' }}
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: review-enrichment/node_modules
key: ${{ steps.rees-node-modules-cache.outputs.cache-primary-key }}
- name: REES build, source-map validation, and tests
if: ${{ github.event_name == 'push' || needs.changes.outputs.rees == 'true' }}
run: npm --prefix review-enrichment test
# review-enrichment's node:test suite runs against the built dist/, so its real coverage was invisible
# to Codecov (which only ever saw vitest's src/** lcov). `npm run rees:coverage` re-runs that suite under
# c8 (v8 coverage) and remaps the dist hits back to review-enrichment/src/** through the build's source
# maps, producing an lcov Codecov can gate. The authoritative pass/fail for REES stays the uninstrumented
# step above: c8's instrumentation inflates that suite's timing-sensitive linear-time/ReDoS-guard
# assertions past their (uninstrumented) budgets, so a non-zero child exit here is expected -- this
# harvest only needs to emit lcov, and a genuinely broken suite is already caught above (and would show
# as ~0% patch anyway). The verify step below fails CI if no report lands.
- name: REES coverage
if: ${{ github.event_name == 'push' || needs.changes.outputs.rees == 'true' }}
run: npm run rees:coverage || true
- name: Verify REES coverage report exists
if: ${{ github.event_name == 'push' || needs.changes.outputs.rees == 'true' }}
run: |
if [ ! -s review-enrichment/coverage/lcov.info ]; then
echo "::error title=REES coverage::review-enrichment/coverage/lcov.info is missing or empty"
exit 1
fi
# Separate `rees` flag so codecov/patch actually gates review-enrichment/** changes -- before this the
# package was in no coverage report at all, so a REES-only PR got a vacuous patch check. Same trusted vs.
# fork-tokenless split and override_* handling as the shard uploads in validate-tests below.
- name: Upload REES coverage to Codecov
if: ${{ (github.event_name == 'push' || needs.changes.outputs.rees == 'true') && github.event.pull_request.head.repo.fork != true }}
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./review-enrichment/coverage/lcov.info
flags: rees
disable_search: true
override_branch: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.ref || github.ref_name }}
override_commit: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
override_pr: ${{ github.event_name == 'pull_request' && github.event.pull_request.number || '' }}
fail_ci_if_error: true
- name: Upload REES coverage to Codecov (fork PR tokenless)
if: ${{ (github.event_name == 'push' || needs.changes.outputs.rees == 'true') && github.event.pull_request.head.repo.fork == true }}
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
files: ./review-enrichment/coverage/lcov.info
flags: rees
disable_search: true
override_branch: ${{ github.event.pull_request.head.repo.owner.login }}:${{ github.event.pull_request.head.ref }}
override_commit: ${{ github.event.pull_request.head.sha }}
override_pr: ${{ github.event.pull_request.number }}
fail_ci_if_error: true
# control-plane is not an npm workspace member (its own package-lock.json), so it needs its own
# cache entry -- same restore/save-after-success pattern and fork/trusted key split as REES above.
- name: Restore control-plane node_modules cache
id: control-plane-node-modules-cache
if: ${{ github.event_name == 'push' || needs.changes.outputs.controlPlane == 'true' }}
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: control-plane/node_modules
key: npm-control-plane-${{ (github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == true) && 'fork' || 'trusted' }}-${{ hashFiles('.nvmrc') }}-${{ hashFiles('control-plane/package.json', 'control-plane/package-lock.json') }}
- name: Control-plane install
if: ${{ (github.event_name == 'push' || needs.changes.outputs.controlPlane == 'true') && steps.control-plane-node-modules-cache.outputs.cache-hit != 'true' }}
run: npm run control-plane:install
- name: Save control-plane node_modules cache
if: ${{ (github.event_name == 'push' || needs.changes.outputs.controlPlane == 'true') && steps.control-plane-node-modules-cache.outputs.cache-hit != 'true' }}
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: control-plane/node_modules
key: ${{ steps.control-plane-node-modules-cache.outputs.cache-primary-key }}
- name: Control-plane build and tests
if: ${{ github.event_name == 'push' || needs.changes.outputs.controlPlane == 'true' }}
run: npm run control-plane:test
- name: Control-plane coverage
if: ${{ github.event_name == 'push' || needs.changes.outputs.controlPlane == 'true' }}
run: npm run control-plane:coverage || true
- name: Verify control-plane coverage report exists
if: ${{ github.event_name == 'push' || needs.changes.outputs.controlPlane == 'true' }}
run: |
if [ ! -s control-plane/coverage/lcov.info ]; then
echo "::error title=Control-plane coverage::control-plane/coverage/lcov.info is missing or empty"
exit 1
fi
- name: Upload control-plane coverage to Codecov
if: ${{ (github.event_name == 'push' || needs.changes.outputs.controlPlane == 'true') && github.event.pull_request.head.repo.fork != true }}
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./control-plane/coverage/lcov.info
flags: control-plane
disable_search: true
override_branch: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.ref || github.ref_name }}
override_commit: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
override_pr: ${{ github.event_name == 'pull_request' && github.event.pull_request.number || '' }}
fail_ci_if_error: true
- name: Upload control-plane coverage to Codecov (fork PR tokenless)
if: ${{ (github.event_name == 'push' || needs.changes.outputs.controlPlane == 'true') && github.event.pull_request.head.repo.fork == true }}
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
files: ./control-plane/coverage/lcov.info
flags: control-plane
disable_search: true
override_branch: ${{ github.event.pull_request.head.repo.owner.login }}:${{ github.event.pull_request.head.ref }}
override_commit: ${{ github.event.pull_request.head.sha }}
override_pr: ${{ github.event.pull_request.number }}
fail_ci_if_error: true
- name: OpenAPI drift check
if: ${{ github.event_name == 'push' || needs.changes.outputs.ui == 'true' || needs.changes.outputs.uiContract == 'true' }}
run: npm run ui:openapi:check
# #2556: RepositorySettingsSchema (hand-authored Zod) can silently drift from the actual
# RepositorySettings TS type -- the OpenAPI drift check above only verifies the generated spec
# matches the Zod schema, never that the schema matches the type the API actually serializes.
- name: OpenAPI settings-parity check
if: ${{ github.event_name == 'push' || needs.changes.outputs.ui == 'true' || needs.changes.outputs.uiContract == 'true' }}
run: npm run ui:openapi:settings-parity
# Checks apps/loopover-ui/src' known-latest MCP version string against the published package, so
# its dependency is `ui` (the file it scans) + `mcp` (the package it checks against) -- NOT the
# OpenAPI contract, which this script never reads.
- name: UI/MCP version audit
if: ${{ github.event_name == 'push' || needs.changes.outputs.ui == 'true' || needs.changes.outputs.mcp == 'true' }}
run: npm run ui:version-audit
# loopover-ui and loopover-miner-ui both import compiled output from this shared package
# (packages/loopover-ui-kit/dist), which is gitignored like every other workspace package's
# dist -- so it does not exist on a fresh checkout until something builds it. Same class of gap
# as "Build engine package" above; runs before UI lint/typecheck/tests/build all in this same
# job, so building it here once covers all four.
#
# `!cancelled()` on this step and every UI step below it (through "UI tests (ui)"): this job runs
# a long, unrelated chain of drift checks before reaching here (migrations, schema, env-reference,
# docs, manifest, engine-parity, branding, .nvmrc, release-manifest, observability, the MCP
# version audit just above, ...) -- any ONE of those failing without continue-on-error otherwise
# skips every step after it by GitHub Actions' own default, INCLUDING this build. "UI tests
# (ui-miner)"/"UI tests (miner-extension)" already had this guard (see their own comment below);
# this build step and everything through "UI tests (ui)" didn't, so a same-day MCP release
# tripping the version-audit step above skipped ui-kit's build entirely, and ui-miner's tests
# (which DID still run, guarded) then failed with a confusing "Failed to resolve import
# '@loopover/ui-kit/components/...'" that looked like a real code bug instead of a skipped
# upstream build -- confirmed live, PR #8182. The job as a whole still fails if any step fails;
# this only stops one unrelated failure from hiding/masking every UI result behind it.
- name: Build UI-kit package
if: ${{ !cancelled() && (github.event_name == 'push' || needs.changes.outputs.ui == 'true') }}
run: npx turbo run build --filter=@loopover/ui-kit
# apps/loopover-ui's fumadocs-mdx virtual modules (collections/browser, collections/server, imported by
# docs-client-loader.tsx/docs-source.ts) are gitignored generated output, normally produced by npm ci's
# own `postinstall` hook -- but "Install dependencies" above is SKIPPED on a node_modules cache hit (the
# cached path never includes this output), so a warm-cache run reaches UI lint/typecheck/tests/build with
# no collections output at all and fails with "Cannot find module 'collections/browser'". Same class of
# gap as "Build UI-kit package" above (a required, gitignored artifact absent on a fresh/cached
# checkout); runs unconditionally here so cache hit or miss makes no difference.
- name: Generate docs content collections
if: ${{ !cancelled() && (github.event_name == 'push' || needs.changes.outputs.ui == 'true') }}
run: npm run postinstall --workspace @loopover/ui
# Routed through Turborepo -- "Build UI-kit package" above already built ui-kit's dist once for all of
# lint/typecheck/tests/build in this same job (GitHub Actions steps run strictly sequentially), so
# these turbo invocations reuse it without needing @loopover/ui#lint/@loopover/ui-miner#lint to
# declare their own ^build dependsOn (they intentionally don't -- eslint needs no compiled dist).
# Two --filter flags means "either package" (union), not intersection; both packages' tasks run
# even if one fails, so a failing @loopover/ui lint no longer hides an @loopover/ui-miner lint error
# behind it the way the old `&&` chain did -- both surface in one pass. Deliberately NOT changing the
# package.json ui:lint/ui:typecheck scripts themselves, since ui-deploy.yml and CONTRIBUTING.md's
# documented local gate sequence both call those directly and rely on their own ui:kit:build prefix.
- name: UI lint
if: ${{ !cancelled() && (github.event_name == 'push' || needs.changes.outputs.ui == 'true') }}
run: npx turbo run lint --filter=@loopover/ui --filter=@loopover/ui-miner
- name: UI typecheck
if: ${{ !cancelled() && (github.event_name == 'push' || needs.changes.outputs.ui == 'true') }}
run: npx turbo run typecheck --filter=@loopover/ui --filter=@loopover/ui-miner
# Split into 3 independent steps (was one `&&`-chained step) -- turbo.json deliberately has no `test`
# task for any package (per the plan doc: test orchestration stays out of Turborepo entirely, so
# Codecov's codecov/patch keeps one centralized source of truth), so this can't use the same
# union-`--filter` fix already applied to "UI lint"/"UI typecheck" two steps above. Plain `&&` meant a
# failing @loopover/ui test silently prevented @loopover/ui-miner's and @loopover/miner-extension's
# tests from ever running in that invocation -- same class of failure-masking bug those two steps'
# own comment already describes, just still present here. `!cancelled()` on all 3 (not `always()`)
# matches the pattern already used by "Save Turborepo cache"/"Save TypeScript incremental build
# cache" elsewhere in this file: still run after an earlier step's failure, just not after the job
# was cancelled outright. The job as a whole still fails if any of the three fails.
- name: UI tests (ui)
if: ${{ !cancelled() && (github.event_name == 'push' || needs.changes.outputs.ui == 'true') }}
run: npm --workspace @loopover/ui run test
- name: UI tests (ui-miner)
if: ${{ !cancelled() && (github.event_name == 'push' || needs.changes.outputs.ui == 'true') }}
run: npm --workspace @loopover/ui-miner run test
- name: UI tests (miner-extension)
if: ${{ !cancelled() && (github.event_name == 'push' || needs.changes.outputs.ui == 'true') }}
run: npm --workspace @loopover/miner-extension run test
# Same rationale and --filter union semantics as "UI lint"/"UI typecheck" above -- and the same
# `!cancelled()` need as "Build UI-kit package" through "UI tests (ui)" above: an earlier
# unrelated step failing (a drift check, the MCP version audit) would otherwise skip these too,
# silently hiding real Extension lint/typecheck results behind it.
- name: Extension lint
if: ${{ !cancelled() && (github.event_name == 'push' || needs.changes.outputs.ui == 'true') }}
run: npx turbo run lint --filter=@loopover/extension --filter=@loopover/miner-extension
- name: Extension typecheck
if: ${{ !cancelled() && (github.event_name == 'push' || needs.changes.outputs.ui == 'true') }}
run: npx turbo run typecheck --filter=@loopover/extension --filter=@loopover/miner-extension
# `npm run ui:build` also regenerates apps/loopover-ui/public/openapi.json (needed for a
# standalone build), but this step's trigger condition is a strict subset of "OpenAPI drift
# check" above (push || ui==true, vs. push || ui==true || uiContract==true), so whenever this
# step runs, that check already ran and passed in this same job -- the committed spec is
# already byte-identical to what regenerating it here would produce. apps/loopover-ui's own
# `build` script (vite build) doesn't touch openapi.json either, so routing through Turborepo
# preserves that skip: @loopover/ui#build's dependsOn (turbo.json) is
# ["^build", "@loopover/extension#build", "@loopover/miner-extension#build"] -- the exact same
# extension + miner-extension build pair this step ran explicitly before, now resolved by turbo's
# own graph instead of hand-chained `&&`, plus ^build (ui-kit, already a same-job cache hit from
# "Build UI-kit package" above). Verified locally: deleting the committed-gitignored extension zip
# and running this exact invocation regenerates it correctly (the cross-package outputs edge is
# honored, not just replayed from cache).
- name: UI build
if: ${{ !cancelled() && (github.event_name == 'push' || needs.changes.outputs.ui == 'true') }}
run: npx turbo run build --filter=@loopover/ui
# Paired with "Restore Turborepo cache" above (same key, same gate condition) -- deliberately
# positioned here, after every turbo-routed step in this job (Build engine package, Build MCP, Build
# miner CLI, Build UI-kit package, UI lint/typecheck, Extension lint/typecheck, UI build), rather than
# immediately after "Build engine package" the way it was originally: actions/cache/save takes one
# synchronous snapshot at the moment it runs, so anything written to .turbo/cache by a later step
# would never be captured if this ran earlier. `!cancelled()` (not `always()`) still saves whatever
# was successfully built even if an earlier step in this job failed, as long as the job wasn't
# cancelled outright.
- name: Save Turborepo cache
if: ${{ !cancelled() && (github.event_name == 'push' || needs.changes.outputs.backend == 'true' || needs.changes.outputs.engine == 'true' || needs.changes.outputs.ui == 'true' || needs.changes.outputs.mcp == 'true' || needs.changes.outputs.miner == 'true') }}
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: .turbo/cache
key: turbo-code-${{ hashFiles('package-lock.json') }}-${{ github.run_id }}
# Bundle-size regression tracking for the deployed marketing/product site (loopover-ui only --
# loopover-miner-ui is a self-hosted operator dashboard with no build step in this job at all, see
# "UI build" above). Fork-excluded (no secrets.CODECOV_TOKEN on a fork PR, same boundary as the
# coverage-upload steps below) and non-blocking (continue-on-error): unlike coverage, nothing gates
# merges on bundle size, so an upload hiccup must never fail CI. Uses @codecov/bundle-analyzer's
# standalone CLI against the built dist/client directly rather than a bundler-plugin hook --
# @codecov/vite-plugin's latest release doesn't support this repo's Vite 8 yet (peer range caps at
# 6.x; this repo's Vite 8 runs on rolldown, not classic rollup, so @codecov/rollup-plugin doesn't fit
# either) -- revisit wiring the plugin in once upstream catches up.
- name: Upload UI bundle stats to Codecov
if: ${{ (github.event_name == 'push' || needs.changes.outputs.ui == 'true') && github.event.pull_request.head.repo.fork != true }}
continue-on-error: true
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
run: npm run bundle-analysis --workspace @loopover/ui
# The full-suite coverage run, sharded (#ci-shard-coverage). This alone was ~9-10 of the ~11 minutes a
# typical backend PR spent in validate-code, because vitest schedules whole test FILES atomically to
# workers -- one unsharded job with --maxWorkers=4 (a standard runner's real CPU budget) has no way to
# use more than 4-way parallelism no matter how many files exist. Splitting the vitest invocation itself
# across N parallel GitHub-hosted jobs (vitest's own --shard=<i>/<n>, not a hand-rolled file list) gives
# N*4-way parallelism instead, at zero extra $ on this public repo (standard runners are free regardless
# of job count -- more jobs consumes more of the shared free pool, not more billable minutes).
# Each shard uploads its own lcov.info + junit report to Codecov; multiple uploads for the same
# commit/PR are additive (Codecov merges them into one patch-coverage view), which is the standard
# pattern for matrix-split test suites. vitest.config.ts already had COVERAGE_NO_THRESHOLDS support
# wired in (a shard only exercises part of the tree, so the global 90% backstop threshold would always
# false-fail per-shard) -- this job is what finally turns it on; previously nothing ever set it.
validate-tests:
name: validate-tests
needs: changes
# Also runs on REES/mcp/engine/miner/discoveryIndex-only changes: Codecov holds codecov/patch until
# `after_n_builds` (3) coverage uploads land (codecov.yml), and a PR scoped to just one of those
# self-contained packages doesn't otherwise touch `backend`, so without the shards it would upload no
# coverage report at all and codecov/patch would never post -- not even a vacuous pass, no check at
# all, and the required `validate` aggregate treats a skipped job as success (#7318-#7321: 3
# packages/loopover-miner/lib-only PRs merged this way with validate-tests never running). Running the
# shards guarantees the 3 uploads so Codecov posts a real patch verdict covering whichever package
# actually changed (review-enrichment/** via the `rees` flag from validate-code; packages/loopover-mcp,
# packages/loopover-engine, packages/loopover-miner, and packages/discovery-index via their entries in
# vitest.config.ts's coverage.include, exercised by the root test/** suite these shards already run).
# Draft PRs still skip the whole fan-out (#6448), so these triggers only apply to ready PRs.
if: ${{ github.event_name == 'push' || (github.event.pull_request.draft != true && (needs.changes.outputs.backend == 'true' || needs.changes.outputs.rees == 'true' || needs.changes.outputs.controlPlane == 'true' || needs.changes.outputs.mcp == 'true' || needs.changes.outputs.engine == 'true' || needs.changes.outputs.miner == 'true' || needs.changes.outputs.discoveryIndex == 'true')) }}
runs-on: ubuntu-latest
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3]
env:
VITE_LOOPOVER_API_ORIGIN: https://api.loopover.ai
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
# Full history so Codecov can resolve the merge base for patch coverage.
fetch-depth: 0
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
# See .github/actions/setup-workspace (npmrc, Node, node_modules cache -- same cache key formula
# as validate-code, so a lockfile-unchanged PR gets a cache hit here too and skips npm ci
# entirely; concurrent jobs racing to save the same key is safe, actions/cache/save no-ops if
# another job already wrote it).
- name: Setup workspace
uses: ./.github/actions/setup-workspace
# Any backend test run needs the engine package's dist/ built first -- see the identical step's
# comment in validate-code (#ci-engine-build-order) for why. Same accumulating .turbo/cache
# restore/save pattern as validate-code's pair, but a DIFFERENT key prefix ("turbo-tests-" vs.
# validate-code's "turbo-code-"): these two jobs used to share one plain "turbo-" prefix, which meant
# all 7 jobs in a single run (validate-code plus these 6 shards) raced for the same cache slot, and
# actions/cache/save no-ops on a duplicate key within a run -- see the shard-vs-shard version of that
# same race below. Because a shard reaches its own Save after only ~6 steps versus validate-code's
# ~20+, a shard almost always won that race, meaning validate-code's much richer cache (engine + mcp +
# miner + ui-kit + ui + extension) was very likely discarded every run in favor of a shard's
# engine-only one. Splitting the prefixes gives each job's own lineage instead of racing; within THIS
# prefix, the 6 shards below still race each other for one slot, which is fine since they all save the
# exact same content (engine + mcp + miner builds all run in this job) -- whichever shard wins, the
# content is equivalent, unlike the cross-job case this split just fixed.
- name: Restore Turborepo cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: .turbo/cache
key: turbo-tests-${{ hashFiles('package-lock.json') }}-${{ github.run_id }}
restore-keys: |
turbo-tests-${{ hashFiles('package-lock.json') }}-
- name: Build engine package
run: npx turbo run build --filter=@loopover/engine
# packages/loopover-mcp and packages/loopover-miner both ship .ts source only (no compiled .js
# committed -- gitignored, same as @loopover/engine's dist/), but this job's own SKIP_MCP_CLI_HARNESS/
# SKIP_MINER_TEST_HARNESS env below (see "Test with coverage") can both be false, which spawns the
# REAL built CLI binary (packages/loopover-mcp/bin/loopover-mcp.js / packages/loopover-miner/bin/
# loopover-miner.js) as a subprocess -- AND, separately, every mcp-cli-*/miner-mcp-* test that instead
# runs the .ts entry directly via `node --experimental-strip-types` (test/unit/support/mcp-cli-harness.ts
# et al.) still needs this: that flag only strips types from the directly-executed entry file, it does
# NOT make plain Node resolve a `.js`-suffixed relative import to a sibling .ts the way Vite/esbuild do
# (confirmed live: `node --experimental-strip-types packages/loopover-mcp/bin/loopover-mcp.ts doctor`
# with no compiled lib/ present fails ERR_MODULE_NOT_FOUND on lib/local-branch.js, the CLI's own first
# local import) -- so bin/lib's compiled .js has to physically exist on disk either way. Before the
# compiled output stopped being committed, this job's checkout already had it for free; now it needs
# building here too, same as validate-code's own "Build MCP"/"Build miner CLI" steps.
#
# --force on both: turbo.json's @loopover/mcp#build task now declares real `outputs` (see its own
# comment) so a cache HIT restores the actual files instead of only the log, but this job's 3 parallel
# shards still race on ONE shared turbo-tests- cache key/slot (below), and a hit-from-a-sibling-shard
# is exactly the kind of cross-job interaction that's expensive to keep re-verifying under. --force
# makes this job's own correctness independent of that entirely: it always executes for real here,
# never trusts a hit (from any source, this run or a prior one) for the two packages hundreds of
# downstream tests treat as load-bearing. The packages are small (a few seconds each, confirmed
# locally) -- this is a deliberate, small, fixed cost traded for eliminating an entire class of
# cache-hit-but-file-missing failures, not a workaround. Unconditional (no `if:`), matching "Build
# engine package" immediately above; --force doesn't skip the write side, so this run's own build
# still contributes a correct, restorable cache entry for anything else that reads the turbo-tests-
# prefix (this job's own next run, or a future validate-tests-only invocation).
- name: Build MCP
run: npx turbo run build --filter=@loopover/mcp --force
- name: Build miner CLI
run: npx turbo run build:tsc build:verify --filter=@loopover/miner --force
# Fail fast and unambiguously, right here, if either binary is somehow still missing after the two
# --force builds above -- instead of dozens of confusing per-test MODULE_NOT_FOUND/ENOENT failures far
# downstream in "Test with coverage" that all look unrelated to each other and to this step. This
# should be structurally impossible now (both builds above are forced, never cache-skipped), but it's
# a one-line, load-bearing tripwire for a class of bug that has already cost real debugging time twice.
- name: Verify MCP/miner CLI binaries were actually built
run: |
set -euo pipefail
for f in packages/loopover-mcp/bin/loopover-mcp.js packages/loopover-miner/bin/loopover-miner.js; do
if [ ! -f "$f" ]; then
echo "::error::$f is missing after Build MCP/Build miner CLI both ran with --force. Every mcp-cli-*/miner-* test below will fail with a confusing MODULE_NOT_FOUND/ENOENT instead of this message -- something deeper changed (e.g. the build script itself stopped writing to this path); this is not a cache/timing issue, --force rules that out."
exit 1
fi
done
- name: Save Turborepo cache
if: ${{ !cancelled() }}
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: .turbo/cache
key: turbo-tests-${{ hashFiles('package-lock.json') }}-${{ github.run_id }}
- name: Prepare test reports dir
run: mkdir -p reports/junit
# Restore-only -- this job never saves it, test-timing-refresh.yml is the sole writer, on its own
# schedule (pulling from Codecov's Test Analytics API, which already pools per-test durations
# across every push-to-main run). key never actually matches (no save ever uses this literal
# string); it exists only so the step always falls through to the restore-keys prefix match,
# picking whatever the most recent refresh wrote. A cache MISS here is always safe: see
# scripts/compute-test-shards.ts's fallback -- it splits evenly across shards when no timing
# data is available for a file (or none at all), the same balance vitest's own --shard already
# gives today, so this can only make shard balance better than today's baseline, never worse.
- name: Restore test timing cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: test-timing.json
key: test-timing-unused
restore-keys: |
test-timing-
- name: Test with coverage (shard ${{ matrix.shard }}/3)
id: coverage
env:
# Disables vitest.config.ts's own global 90% threshold check for THIS per-shard invocation -- a
# single shard only exercises part of the tree, so it would always false-fail. The global-threshold
# "catastrophe net" this disables is NOT dropped from CI, though: validate-tests-merge (below) merges
# all shards' coverage via vitest's own --mergeReports and re-checks the threshold against that
# merged (whole-suite) total, without COVERAGE_NO_THRESHOLDS set.
COVERAGE_NO_THRESHOLDS: "true"
# Same self-contained-test-file narrowing as the pre-sharding job had -- see the `mcpCliHarness`/
# `minerTestHarness` filter comments in the `changes` job above for the full rationale.
SKIP_MCP_CLI_HARNESS: ${{ github.event_name == 'pull_request' && needs.changes.outputs.mcpCliHarness != 'true' }}
SKIP_MINER_TEST_HARNESS: ${{ github.event_name == 'pull_request' && needs.changes.outputs.minerTestHarness != 'true' }}
# Scoped test selection (#ci-scoped-test-selection): a PR touching miner/mcp/discoveryIndex/backend --
# never rees or engine -- runs vitest's own --changed against origin/main instead of the full
# ~2,900+-test suite, since the same 6-shard fan-out this job exists for (see header comment) is
# otherwise massive overkill for a diff touching a handful of files. engine is deliberately EXCLUDED
# from this fast path: it's the one package other code imports through a built, gitignored dist/ that
# --changed's import-graph analysis can't safely trace across a node_modules/package.json-exports
# boundary -- verified by reproducing a real missed regression (editing packages/loopover-engine/
# src/ directly, --changed failed to select the miner-side wrapper-integration tests that actually
# exercise it). rees stays excluded too (unreviewed, no evidence gathered either way -- narrower
# scope of caution costs little since it's a small slice of PRs).
#
# backend (src/**) was added to this fast path after direct verification, not by assumption: 5 real
# historical single/few-file src/**-only commits (spanning orb, review, api, db, selfhost -- see
# loopover#7590-era CI investigation) were replayed with the target commit actually checked out and
# --changed run against its parent. Every one correctly included its own direct dependent test
# file(s); zero false negatives found. Selection size scaled sensibly with how foundational the file
# was (a leaf module: ~0.1% of the suite; the shared DB repositories layer: ~31%) -- always less than
# the unscoped 100%, never more. That verification covered plain .ts source-file diffs only -- never
# a migration, package.json/tsconfig/vitest.config, or workflow/tooling change, none of which have
# any import-graph edge a test file could trace to them. `backendConfig` (the `changes` job above)
# captures exactly that non-traceable subset of `backend` and is required false here so a PR
# touching one still gets the full unscoped suite even if it also touches src/**. The one
# deliberately-unfiltered wide-fan-out file,
# test/unit/mcp-output-schemas.test.ts (imports src/mcp/server.ts, which alone directly imports ~40
# other src/ modules -- see the `mcpCliHarness` filter comment in the `changes` job above), is NOT
# given special-casing here: unlike the static path-filter that comment describes, --changed is a
# live, same-source-tree import-graph trace, which this batch of verification confirmed handles
# direct in-repo imports correctly (only the engine dist/ package-boundary case above is unsafe).
# Confirmed live: combining --changed with an explicit positional filename NARROWS the selection
# (intersection) rather than unioning it, so that composition was deliberately NOT used to force this
# file in -- doing so would have silently dropped every other selected file instead.
#
# miner/mcp/discoveryIndex don't have engine's exposure either: nothing else in the repo imports them
# as libraries, so their tests are reached via direct same-package imports, verified to trace
# correctly. Every push to main still runs the full unscoped suite unconditionally (this job's own
# top-level `if:` above is unchanged), so a scoped miss is caught within minutes of merge, not never.
# Kill switch: set the repo variable SCOPED_TEST_SELECTION_ENABLED to "false" to force the full suite
# for every PR (e.g. if a gap is ever found) without a revert PR.
SCOPED_TEST_SELECTION: ${{ github.event_name == 'pull_request' && vars.SCOPED_TEST_SELECTION_ENABLED != 'false' && needs.changes.outputs.rees != 'true' && needs.changes.outputs.controlPlane != 'true' && needs.changes.outputs.engine != 'true' && needs.changes.outputs.backendConfig != 'true' && (needs.changes.outputs.backend == 'true' || needs.changes.outputs.miner == 'true' || needs.changes.outputs.mcp == 'true' || needs.changes.outputs.discoveryIndex == 'true') }}
run: |
EXCLUDE_ARGS=()
if [ "$SKIP_MCP_CLI_HARNESS" = "true" ]; then
echo "Skipping the self-contained MCP CLI-harness tests (no packages/loopover-mcp-relevant paths changed)."
EXCLUDE_ARGS+=(
--exclude "test/unit/mcp-cli-*.test.ts"
--exclude "test/unit/mcp-discovery.test.ts"
)
fi
if [ "$SKIP_MINER_TEST_HARNESS" = "true" ]; then
echo "Skipping the self-contained miner-package tests (no packages/loopover-miner-relevant paths changed)."
EXCLUDE_ARGS+=(
--exclude "test/unit/check-miner-package.test.ts"
--exclude "test/unit/miner-calibration-types.test.ts"
)
fi
SCOPE_ARGS=()
if [ "$SCOPED_TEST_SELECTION" = "true" ]; then
echo "Scoped test selection: no rees/engine paths changed -- running vitest --changed=origin/main instead of the full suite."
# REGRESSION (2026-07-21): this used to be `git fetch --depth=1 origin main`, reasoning it was
# "a cheap, idempotent guarantee" on top of this job's checkout already using fetch-depth: 0
# (full history for every ref, including origin/main -- confirmed sufficient on its own). It
# was NOT idempotent: a --depth=1 fetch of a ref that already has full history locally
# re-shallows THAT ref specifically, cutting its reachable history at the single fetched
# commit. vitest's --changed=origin/main below needs `git merge-base <head>...origin/main` to
# resolve, and once origin/main was re-shallowed to one commit with no shared history
# reachable from a PR branch forked further back, merge-base returned nothing -- vitest
# silently fell back to "no changed files" and every shard printed "No test files found,
# exiting with code 0", failing every scoped PR's "Verify coverage report exists" step,
# contributor and maintainer alike. Dropping --depth entirely keeps this genuinely idempotent
# (a no-op fast-forward on top of history the checkout step already fetched in full) while
# still acting as a real defensive top-up for any future scenario where that assumption stops
# holding, instead of betting everything on the checkout step with no fallback at all.
git fetch origin main
# --coverage.all=false: without it, v8's default "report every coverage.include-matched file
# at 0% even if untouched by this run" would make this shard's upload claim the rest of the
# ~2,900-file suite is 0% covered, polluting Codecov's project trend on every scoped PR. Real
# coverage for files actually exercised is unaffected either way.
SCOPE_ARGS+=(--changed=origin/main --coverage.all=false)
# A scoped PR touching zero files any test imports (a pure docs/comment-only diff under a
# scoped path, e.g. packages/loopover-miner/docs/**) is a real, valid outcome here -- vitest
# itself prints "No test files found, exiting with code 0" and exits clean, writing no
# coverage/lcov.info at all (nothing was instrumented). Confirmed live: PR #8165, a pure
# packages/loopover-miner/docs + apps/loopover-ui/content/docs mdx change, matched zero test
# files and failed "Verify coverage report exists" even though nothing was actually broken.
# Capture vitest's own stdout (the same "No test files found" string this file's own comment
# above already documents relying on) rather than re-deriving the zero-files case from
# scratch, and record it as a step output so "Verify coverage report exists" below can tell
# this apart from a real crash that should still fail loudly.
set -o pipefail
npm run test:coverage -- --maxWorkers=4 --shard=${{ matrix.shard }}/3 --reporter=default --reporter=blob --reporter=junit --outputFile.blob=blob-report/report-${{ matrix.shard }}.blob --outputFile.junit=reports/junit/vitest.xml "${EXCLUDE_ARGS[@]}" "${SCOPE_ARGS[@]}" 2>&1 | tee vitest-scoped-output.log
if grep -q "No test files found" vitest-scoped-output.log; then
echo "no_tests_matched=true" >> "$GITHUB_OUTPUT"
elif [ ! -s coverage/lcov.info ]; then
# #8194 (the week's third scoped-selection edge case, PR #8192's shards): the selection
# matched and PASSED real tests -- set -o pipefail above means this line is only reachable
# on vitest exit 0 -- but none of them exercise anything under coverage.include (src/**):
# a test-only/docs-drift diff. --coverage.all=false then writes an empty or absent lcov,
# which is NOT a failed run; the junit report written above is the proof the suite ran.
# Without this output, "Verify coverage report exists" fails the green run. The success
# signal here is the test run itself, never the coverage file's size.
echo "scoped run passed but exercised no src/** files -- skipping coverage artifacts for this shard"
echo "no_src_coverage=true" >> "$GITHUB_OUTPUT"
fi
else
# Duration-aware sharding (#ci-duration-aware-sharding), full-suite case only: vitest's own
# --shard splits by file COUNT alone, no duration awareness -- confirmed via real per-shard CI
# timing data to produce a consistent ~20-30% gap between the slowest and fastest of the 6
# shards, every run sampled. Deliberately not applied to the scoped-selection branch above:
# that file set isn't known until vitest resolves --changed itself, so a precomputed
# assignment can't cover it without duplicating vitest's own dependency-graph resolution here.
# compute-test-shards.ts enforces its own hard invariant (the union of all 3 shards' files
# must exactly equal the discovered file set, no file missing or duplicated) and refuses to
# write output at all if that's ever violated, so a bug here fails this step loudly rather
# than silently dropping a test file from CI.
node --experimental-strip-types scripts/compute-test-shards.ts --shards=3 --timing=test-timing.json --output=shard-assignment.json
# #7767: capture via a checked assignment so a throw in the node one-liner (bad matrix.shard
# value, corrupt/missing shard-assignment.json) actually aborts this step -- the old
# `mapfile -t SHARD_FILES < <(node -e ...)` ran node in a subshell whose non-zero exit was
# swallowed (mapfile itself returns 0), leaving SHARD_FILES empty and silently making vitest
# run the entire suite instead of its slice. Same fix idiom as compose_file_args (#7765).
if ! SHARD_FILES_RAW="$(node -e "console.log(JSON.parse(require('fs').readFileSync('shard-assignment.json','utf8'))['${{ matrix.shard }}'].join('\n'))")"; then
echo "::error::failed to resolve shard file list for shard ${{ matrix.shard }}"
exit 1
fi
mapfile -t SHARD_FILES <<< "$SHARD_FILES_RAW"
npm run test:coverage -- --maxWorkers=4 "${SHARD_FILES[@]}" --reporter=default --reporter=blob --reporter=junit --outputFile.blob=blob-report/report-${{ matrix.shard }}.blob --outputFile.junit=reports/junit/vitest.xml "${EXCLUDE_ARGS[@]}"
fi
- name: Test failure guidance
if: ${{ failure() && steps.coverage.conclusion == 'failure' }}
run: |
echo "::error title=Tests::The backend test coverage suite failed (shard ${{ matrix.shard }}/3)."
echo "Coverage itself is gated by Codecov on changed lines (codecov/patch), computed from all shards' merged lcov."
echo "Reproduce locally with: 'npm run test:coverage' (unsharded, runs the whole suite)."
- name: Verify coverage report exists
# Skipped when the scoped-selection branch above matched zero test files (steps.coverage.outputs.
# no_tests_matched) -- a legitimately test-free diff (e.g. docs-only under a scoped path) writes no
# coverage/lcov.info at all, and that is not a failure. Also skipped when the selection matched
# tests that PASSED but exercised nothing under src/** (no_src_coverage, #8194) -- same legitimacy,
# different detector: tests matched, so no_tests_matched can't see it. A real failure (tests ran
# and either failed or the coverage step crashed before writing output) still fails this job via
# `success()` below, same as before.
if: ${{ success() && steps.coverage.outputs.no_tests_matched != 'true' && steps.coverage.outputs.no_src_coverage != 'true' }}
run: |
if [ ! -s coverage/lcov.info ]; then
echo "::error title=Coverage::coverage/lcov.info is missing or empty"
exit 1
fi
# Consumed by validate-tests-merge to re-check the global coverage threshold against all shards
# combined -- see this job's own header comment.
- name: Upload coverage blob report
if: ${{ success() && steps.coverage.outputs.no_tests_matched != 'true' && steps.coverage.outputs.no_src_coverage != 'true' }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: coverage-blob-shard-${{ matrix.shard }}
path: blob-report/report-${{ matrix.shard }}.blob
retention-days: 1
# Direct upload for trusted contexts (push + same-repo PRs). Multiple shards' uploads for the same
# commit/PR are additive in Codecov -- see this job's own header comment.
- name: Upload coverage to Codecov
# Same no_tests_matched skip as the verify step above (#8167 follow-up): with zero matched tests
# there is no lcov to upload, and fail_ci_if_error would otherwise turn that non-event red.
if: ${{ success() && steps.coverage.outputs.no_tests_matched != 'true' && steps.coverage.outputs.no_src_coverage != 'true' && github.event.pull_request.head.repo.fork != true }}
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./coverage/lcov.info
flags: shard-${{ matrix.shard }}
disable_search: true
override_branch: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.ref || github.ref_name }}
override_commit: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
override_pr: ${{ github.event_name == 'pull_request' && github.event.pull_request.number || '' }}
# Coverage upload is part of the hard gate: upload or service errors
# should fail CI instead of allowing a PR to merge without patch data.
fail_ci_if_error: true
# Fork PRs run without secrets -- see the identical step's comment in the old single-job validate-code
# (preserved in git history) for the full tokenless-upload rationale (branch-prefix requirement, sha
# override, etc). Condition dropped the push-or-backend check since this job's own `if:` already
# covers it.
- name: Upload coverage to Codecov (fork PR tokenless)
if: ${{ success() && steps.coverage.outputs.no_tests_matched != 'true' && steps.coverage.outputs.no_src_coverage != 'true' && github.event.pull_request.head.repo.fork == true }}
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
files: ./coverage/lcov.info
flags: shard-${{ matrix.shard }}
disable_search: true
override_branch: ${{ github.event.pull_request.head.repo.owner.login }}:${{ github.event.pull_request.head.ref }}
override_commit: ${{ github.event.pull_request.head.sha }}
override_pr: ${{ github.event.pull_request.number }}
fail_ci_if_error: true
# Test results are useful Codecov annotations, not the coverage gate. Keep
# their upload non-blocking so a JUnit ingestion hiccup does not fail CI
# after the tests and hard coverage upload have already passed.
- name: Upload Vitest results to Codecov
if: ${{ !cancelled() && steps.coverage.outputs.no_tests_matched != 'true' && steps.coverage.outputs.no_src_coverage != 'true' && github.event.pull_request.head.repo.fork != true }}
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./reports/junit/vitest.xml
report_type: test_results
flags: shard-${{ matrix.shard }}
disable_search: true
override_branch: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.ref || github.ref_name }}
override_commit: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
override_pr: ${{ github.event_name == 'pull_request' && github.event.pull_request.number || '' }}
fail_ci_if_error: false
- name: Upload Vitest results to Codecov (fork PR tokenless)
if: ${{ !cancelled() && steps.coverage.outputs.no_tests_matched != 'true' && steps.coverage.outputs.no_src_coverage != 'true' && github.event.pull_request.head.repo.fork == true }}
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
files: ./reports/junit/vitest.xml
report_type: test_results
flags: shard-${{ matrix.shard }}
disable_search: true
override_branch: ${{ github.event.pull_request.head.repo.owner.login }}:${{ github.event.pull_request.head.ref }}
override_commit: ${{ github.event.pull_request.head.sha }}
override_pr: ${{ github.event.pull_request.number }}
fail_ci_if_error: false
# Re-checks vitest.config.ts's global 90% coverage threshold against all shards MERGED -- each shard
# above deliberately disables that check for itself (COVERAGE_NO_THRESHOLDS=true), since a single shard's
# partial view would always false-fail it. This is the "loose catastrophe net" (e.g. a deleted test file)
# vitest.config.ts's own comment describes; Codecov's patch gate (changed-lines only) doesn't cover a
# whole-repo regression outside the diff, so this is what actually restores that backstop for CI, using
# vitest's own --mergeReports against each shard's uploaded blob report.
#
# That "merged = whole-suite" assumption breaks when scoped test selection (#ci-scoped-test-selection,
# see the shard job's own SCOPED_TEST_SELECTION comment above) was active for this run: all 3 shards then
# ran the SAME narrow `--changed=origin/main` subset (not a partition of the ~2,900-test full suite), so
# merging them still only reconstructs that narrow slice's coverage -- correctly high for the files it
# touches, but the threshold judges the WHOLE include set, so it false-fails even a fully-tested scoped
# PR (confirmed live: a 3-file packages/loopover-mcp-only PR with 100% coverage on its own two touched
# test files still reported 0%/80% and failed). This job disables the threshold in that same case, via
# the identical SCOPED_TEST_SELECTION condition -- Codecov's patch gate already enforces real per-line
# coverage on a scoped PR's actual diff regardless, so nothing is lost by skipping the whole-suite
# backstop specifically when it can't see the whole suite.
validate-tests-merge:
name: validate-tests-merge
needs: [changes, validate-tests]
# Same trigger as validate-tests: whenever shards ran (REES/mcp/engine/miner/discoveryIndex-only PRs
# included), merge their reports too.
if: ${{ github.event_name == 'push' || (github.event.pull_request.draft != true && (needs.changes.outputs.backend == 'true' || needs.changes.outputs.rees == 'true' || needs.changes.outputs.controlPlane == 'true' || needs.changes.outputs.mcp == 'true' || needs.changes.outputs.engine == 'true' || needs.changes.outputs.miner == 'true' || needs.changes.outputs.discoveryIndex == 'true')) }}
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
with:
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
# Same cache key as validate-code/validate-tests -- a cache hit here is the common case since those
# jobs run concurrently and one of them usually wins the race to populate it first. save-cache:
# false -- this job only ever reads that cache, it never writes to it (see
# .github/actions/setup-workspace).
- name: Setup workspace
uses: ./.github/actions/setup-workspace
with:
save-cache: "false"
- name: Download all shards' blob reports
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: coverage-blob-shard-*
path: all-blob-reports
merge-multiple: true
- name: Merge shard coverage and check the global threshold
env:
# Mirrors validate-tests' own SCOPED_TEST_SELECTION condition exactly (this job already has
# `needs.changes` available). vitest.config.ts checks this var for TRUTHINESS, not `=== 'true'`,
# so the false branch must be an empty string (falsy), not the literal string "false" (which JS
# treats as truthy) -- that's why this is a `&& 'true' || ''` expression, not a bare boolean.
COVERAGE_NO_THRESHOLDS: ${{ (github.event_name == 'pull_request' && vars.SCOPED_TEST_SELECTION_ENABLED != 'false' && needs.changes.outputs.rees != 'true' && needs.changes.outputs.controlPlane != 'true' && needs.changes.outputs.engine != 'true' && needs.changes.outputs.backendConfig != 'true' && (needs.changes.outputs.backend == 'true' || needs.changes.outputs.miner == 'true' || needs.changes.outputs.mcp == 'true' || needs.changes.outputs.discoveryIndex == 'true')) && 'true' || '' }}
run: |
# Unlike the per-shard --changed run (validate-tests above), --mergeReports exits 1 (not 0) when
# every merged shard's report represents zero matched test files -- confirmed live: PR #8168, a
# pure docs-only scoped PR where all 3 shards legitimately matched nothing, printed vitest's own
# "No test files found, exiting with code 1" and failed here even with COVERAGE_NO_THRESHOLDS
# already disabling the threshold check that comment block above describes. Only treat that exact
# case as success, and only when COVERAGE_NO_THRESHOLDS is set (i.e. this run is the scoped case
# to begin with) -- in the unscoped full-suite case, "no test files found" while merging would be
# a real, surprising break worth failing loudly on, not a legitimate outcome to paper over.
set -o pipefail
if npx vitest run --coverage --mergeReports=all-blob-reports 2>&1 | tee vitest-merge-output.log; then
exit 0
fi
if [ -n "$COVERAGE_NO_THRESHOLDS" ] && grep -q "No test files found" vitest-merge-output.log; then
echo "Merged report matched zero test files, which is expected for this scoped, test-free PR -- not a failure."
exit 0
fi
exit 1
# Diff-scoped security gate: fails only on vulnerabilities this PR introduces.
# Ambient advisories in untouched deps are handled by Renovate + the scheduled
# audit workflow, so one upstream CVE never blocks unrelated PRs.
security:
name: security
if: ${{ github.event_name == 'pull_request' && github.event.pull_request.draft != true }}
# actions/dependency-review-action needs no self-hosted toolchain/cache (checkout + a lockfile diff), so
# this always runs on ubuntu-latest -- keeping it self-hosted for same-repo PRs only competed for the
# scarce self-hosted pool with validate-code, the one job that actually benefits from it (#2501).
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
pull-requests: write
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7
- name: Dependency review
uses: actions/dependency-review-action@a1d282b36b6f3519aa1f3fc636f609c47dddb294 # v5.0.0
with:
fail-on-severity: moderate
comment-summary-in-pr: on-failure
# Single required status check. Branch protection points at "validate"; this
# aggregates the path-aware jobs and the PR-only dependency review gate so that
# requirement keeps working unchanged.
# Path-filtered jobs report "skipped", which is treated as success.
validate:
name: validate
needs: [changes, validate-code, validate-tests, validate-tests-merge, security]
if: ${{ always() }}
# Pure result-aggregation (reads needs.*.result, echoes pass/fail) -- no build/test work, so it never
# needed the self-hosted pool's cached toolchain (#2507).
runs-on: ubuntu-latest
timeout-minutes: 2
steps:
- name: All required jobs passed
if: ${{ !(contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled')) }}
run: echo "All required CI jobs passed (path-filtered jobs reported skipped, which is OK)."
- name: A required job failed
if: ${{ contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') }}
run: |
echo "::error title=CI::A required CI job failed or was cancelled."
echo "Job results: ${{ join(needs.*.result, ', ') }}"
exit 1