feat(miner): wire the opt-in discovery-plane client into discover and attempt (#7220)

* feat(discovery-index): add the soft-claim coordination endpoint

Adds POST /v1/discovery-index/soft-claim as a second endpoint on the
discovery-index service (#7218), accepting the payload shape
discovery-soft-claim.ts's buildSoftClaimRequest already produces
client-side. Lets opted-in miner instances avoid starting duplicate
work on the same discovered opportunity: a claim is accepted if the
key is free, or reported already-held with its age (and its TTL
refreshed) if not; a release frees the key immediately.

Storage reuses the existing TtlCache (a third instance, since it
holds different data than the query-result/policy caches) rather
than a new mechanism. The shipped SoftClaimRequest wire type carries
no caller identity at all (buildSoftClaimRequest hardcodes note and
instanceId to null), so "refresh rather than double-count on repeat
calls" can only mean any repeat claim on a still-active key refreshes
it -- there is no identity on the wire to distinguish callers. The
parser never reads note/instanceId/any other field, so nothing
forbidden can leak into the stored record or response regardless of
what a caller sends.

Closes #7166

* feat(miner): wire the opt-in discovery-plane client into discover and attempt

Adds packages/loopover-miner/lib/discovery-index-client.js, gated fully
behind LOOPOVER_MINER_DISCOVERY_PLANE (default off, zero behavior
change when unset). Wires two integration points:

- discover-cli.js's runDiscover supplements the local fan-out with
  hosted discovery-index results for the same scope, deduped against
  local candidates (local always wins on a duplicate).
- attempt-cli.js submits a soft-claim right after the local
  claim-ledger claim succeeds (before work begins), and a paired
  release on every terminal outcome, using discovery-soft-claim.ts's
  buildSoftClaimRequest directly against the claim-ledger's own
  record shape -- no translation needed, the shapes already match by
  design. The opt-in check happens at each attempt-cli.js call site,
  not left to the client function's own internal check alone, so a
  disabled plane costs zero calls and a test-injected override can't
  bypass the gate.

A second, independent opt-in (LOOPOVER_MINER_DISCOVERY_TELEMETRY)
emits a low-cardinality anonymized operational event via the
package's own logger -- no hosted telemetry collector exists for the
discovery plane yet, so this is a structured local log line for now.

Regenerates env-reference.md/ams-env-reference.ts for the four new
env vars and updates DEPLOYMENT.md's discovery-plane section, which
was marked a placeholder pending this wiring.

Closes #7168

Author: JSONbored

apps/loopover-ui/src/lib/ams-env-reference.ts

@@ -51,6 +51,26 @@ export const AMS_ENV_REFERENCE_ROWS: MinerEnvReferenceRow[] = [
     firstReference: "lib/deny-hook-synthesis.js",
     defaultValue: "",
   },
+  {
+    name: "LOOPOVER_MINER_DISCOVERY_INDEX_URL",
+    firstReference: "lib/discovery-index-client.js",
+    defaultValue: "",
+  },
+  {
+    name: "LOOPOVER_MINER_DISCOVERY_PLANE",
+    firstReference: "lib/discovery-index-client.js",
+    defaultValue: "",
+  },
+  {
+    name: "LOOPOVER_MINER_DISCOVERY_SHARED_SECRET",
+    firstReference: "lib/discovery-index-client.js",
+    defaultValue: "",
+  },
+  {
+    name: "LOOPOVER_MINER_DISCOVERY_TELEMETRY",
+    firstReference: "lib/discovery-index-client.js",
+    defaultValue: "",
+  },
   {
     name: "LOOPOVER_MINER_EVENT_LEDGER_DB",
     firstReference: "lib/event-ledger.js",
@@ -194,6 +214,10 @@ export const AMS_ENV_REFERENCE_MARKDOWN = [
   '| `LOOPOVER_MINER_CONFIG_DIR` | `lib/deny-hook-synthesis.js` | `""` |',
   "| `LOOPOVER_MINER_CONTRIBUTION_PROFILE_CACHE_DB` | `lib/contribution-profile-cache.js` | (none) |",
   '| `LOOPOVER_MINER_DENY_HOOK_SYNTHESIS_DB` | `lib/deny-hook-synthesis.js` | `""` |',
+  '| `LOOPOVER_MINER_DISCOVERY_INDEX_URL` | `lib/discovery-index-client.js` | `""` |',
+  '| `LOOPOVER_MINER_DISCOVERY_PLANE` | `lib/discovery-index-client.js` | `""` |',
+  '| `LOOPOVER_MINER_DISCOVERY_SHARED_SECRET` | `lib/discovery-index-client.js` | `""` |',
+  '| `LOOPOVER_MINER_DISCOVERY_TELEMETRY` | `lib/discovery-index-client.js` | `""` |',
   "| `LOOPOVER_MINER_EVENT_LEDGER_DB` | `lib/event-ledger.js` | (none) |",
   "| `LOOPOVER_MINER_GOVERNOR_LEDGER_DB` | `lib/governor-ledger.js` | (none) |",
   "| `LOOPOVER_MINER_GOVERNOR_STATE_DB` | `lib/governor-state.js` | (none) |",

packages/loopover-miner/DEPLOYMENT.md

@@ -207,4 +207,13 @@ and `LOOPOVER_MINER_CONFIG_DIR` are covered above under the fleet/state notes; t
 
 ## Optional hosted discovery plane (opt-in)
 
-The Phase 6 **hosted discovery-index** is **off by default** — unlike Orb fleet export (`ORB_AIR_GAP` is the only opt-out). Operators who want cross-fleet metadata queries or soft-claim coordination must opt in explicitly. See [`docs/discovery-plane-operator-guide.md`](docs/discovery-plane-operator-guide.md) ([#4309](https://github.com/JSONbored/loopover/issues/4309), placeholder until [#4300](https://github.com/JSONbored/loopover/issues/4300) / [#4301](https://github.com/JSONbored/loopover/issues/4301) / [#4302](https://github.com/JSONbored/loopover/issues/4302) ship).
+The Phase 6 **hosted discovery-index** is **off by default** — unlike Orb fleet export (`ORB_AIR_GAP` is the only opt-out). Operators who want cross-fleet metadata queries or soft-claim coordination must opt in explicitly, via `lib/discovery-index-client.js` ([#7168](https://github.com/JSONbored/loopover/issues/7168), wired against the hosted server from [#7164](https://github.com/JSONbored/loopover/issues/7164)/[#7166](https://github.com/JSONbored/loopover/issues/7166)):
+
+| Variable | Read by | Purpose |
+| --- | --- | --- |
+| `LOOPOVER_MINER_DISCOVERY_PLANE` | `lib/discovery-index-client.js` | Master opt-in (truthy string, off by default). No hosted discovery-index traffic or telemetry unless set. |
+| `LOOPOVER_MINER_DISCOVERY_INDEX_URL` | `lib/discovery-index-client.js` | Base URL of the hosted discovery-index service. Required for the plane to do anything once enabled. |
+| `LOOPOVER_MINER_DISCOVERY_SHARED_SECRET` | `lib/discovery-index-client.js` | Optional bearer secret for the hosted endpoint, if it requires one. |
+| `LOOPOVER_MINER_DISCOVERY_TELEMETRY` | `lib/discovery-index-client.js` | Second, independent opt-in for anonymized operational telemetry — can stay off while the plane itself is queried/claimed against. |
+
+See [`docs/discovery-plane-operator-guide.md`](docs/discovery-plane-operator-guide.md) for the full invariant list (metadata-only, no compensation signals, credentials stay local).

packages/loopover-miner/docs/env-reference.md

@@ -13,6 +13,10 @@ Generated by `npm run miner:env-reference`. Do not edit manually.
 | `LOOPOVER_MINER_CONFIG_DIR` | `lib/deny-hook-synthesis.js` | `""` |
 | `LOOPOVER_MINER_CONTRIBUTION_PROFILE_CACHE_DB` | `lib/contribution-profile-cache.js` | (none) |
 | `LOOPOVER_MINER_DENY_HOOK_SYNTHESIS_DB` | `lib/deny-hook-synthesis.js` | `""` |
+| `LOOPOVER_MINER_DISCOVERY_INDEX_URL` | `lib/discovery-index-client.js` | `""` |
+| `LOOPOVER_MINER_DISCOVERY_PLANE` | `lib/discovery-index-client.js` | `""` |
+| `LOOPOVER_MINER_DISCOVERY_SHARED_SECRET` | `lib/discovery-index-client.js` | `""` |
+| `LOOPOVER_MINER_DISCOVERY_TELEMETRY` | `lib/discovery-index-client.js` | `""` |
 | `LOOPOVER_MINER_EVENT_LEDGER_DB` | `lib/event-ledger.js` | (none) |
 | `LOOPOVER_MINER_GOVERNOR_LEDGER_DB` | `lib/governor-ledger.js` | (none) |
 | `LOOPOVER_MINER_GOVERNOR_STATE_DB` | `lib/governor-state.js` | (none) |

packages/loopover-miner/lib/attempt-cli.d.ts

@@ -15,6 +15,7 @@ import type { resolveMinerGoalSpec } from "./miner-goal-spec.js";
 import type { ClaimConflictResult, resolveClaimConflict } from "./claim-conflict-resolver.js";
 import type { recordOwnSubmission } from "./governor-state.js";
 import type { getAttemptHistory } from "./portfolio-queue.js";
+import type { submitSoftClaim } from "./discovery-index-client.js";
 
 type CommonAttemptResultFields = {
   repoFullName: string;
@@ -96,6 +97,9 @@ export type RunAttemptOptions = {
   resolveClaimConflict?: typeof resolveClaimConflict;
   recordOwnSubmission?: typeof recordOwnSubmission;
   getAttemptHistory?: typeof getAttemptHistory;
+  /** Hosted soft-claim coordination at work-start/work-end, when the plane is enabled (#7168). Defaults to
+   *  discovery-index-client.js's own submitSoftClaim. */
+  submitSoftClaim?: typeof submitSoftClaim;
   /** Invoked with the real structured result at every return point, in addition to (never instead of) the
    *  plain exit-code return -- the loop orchestrator's real hook into what actually happened. */
   onResult?: (result: AttemptCliResult) => void;

packages/loopover-miner/lib/attempt-cli.js

@@ -40,6 +40,7 @@ import { getAttemptHistory } from "./portfolio-queue.js";
 import { loadReputationHistory, recordOwnSubmission } from "./governor-state.js";
 import { runMinerAttempt } from "./attempt-runner.js";
 import { resolveGitHubToken } from "./github-token-resolution.js";
+import { isDiscoveryPlaneEnabled, submitSoftClaim } from "./discovery-index-client.js";
 
 const ATTEMPT_USAGE =
   "Usage: loopover-miner attempt <owner/repo> <issue#> --miner-login <login> [--base <branch>] [--live] [--dry-run] [--json]";
@@ -210,6 +211,7 @@ export async function runAttempt(args, options = {}) {
   let allocation = null;
   let worktreeResult = null;
   let claimedIssue = false;
+  let claimRecord = null;
 
   try {
     allocator = (options.openWorktreeAllocator ?? openWorktreeAllocator)();
@@ -517,8 +519,19 @@ export async function runAttempt(args, options = {}) {
       return 11;
     }
 
-    const claimRecord = claimResult.claim;
+    claimRecord = claimResult.claim;
     claimedIssue = true;
+    // Hosted soft-claim coordination (#7168), opt-in via LOOPOVER_MINER_DISCOVERY_PLANE -- gated HERE at the
+    // call site (not left to submitSoftClaim's own internal check alone) so a disabled plane costs zero calls,
+    // matching discover-cli.js's supplementWithDiscoveryIndex gating; a caller-injected options.submitSoftClaim
+    // (tests, or a future programmatic caller) can't accidentally bypass the opt-in this way either. Awaited
+    // (not fire-and-forget) so a sibling instance racing the same issue is genuinely less likely to start
+    // duplicate work in the window before this attempt's claim reaches the shared index -- the whole point of
+    // coordinating BEFORE work begins, not after.
+    if (isDiscoveryPlaneEnabled(env)) {
+      const submitClaim = options.submitSoftClaim ?? submitSoftClaim;
+      await submitClaim(claimRecord, { env });
+    }
 
     const runAttemptPipeline = options.runMinerAttempt ?? runMinerAttempt;
     let result;
@@ -701,6 +714,13 @@ export async function runAttempt(args, options = {}) {
     // unexpected throw) releases the soft-claim -- a claim that outlives its own attempt process would
     // wrongly tell a sibling miner this issue is still in flight.
     if (claimedIssue && claimLedger) claimLedger.releaseClaim(parsed.repoFullName, parsed.issueNumber);
+    // Paired hosted release (#7168): same call-site opt-in gate as the claim submission above. Only fires when
+    // the initial claim submission actually ran (claimRecord is only set once claimedIssue is), so a run that
+    // never reached the claim point (e.g. blocked_max_concurrent_claims) has nothing to release remotely.
+    if (claimedIssue && claimRecord && isDiscoveryPlaneEnabled(env)) {
+      const submitClaim = options.submitSoftClaim ?? submitSoftClaim;
+      await submitClaim({ ...claimRecord, status: "released" }, { env });
+    }
     if (allocation && allocator) allocator.release(attemptId);
     allocator?.close();
     claimLedger?.close();

packages/loopover-miner/lib/discover-cli.d.ts

@@ -15,6 +15,7 @@ import type { PolicyVerdictCacheStore } from "./policy-verdict-cache.js";
 import type { EnqueueRankedDiscoverySummary } from "./portfolio-discovery.js";
 import type { PortfolioQueueStore } from "./portfolio-queue.js";
 import type { RankedCandidatesStore } from "./ranked-candidates.js";
+import type { queryDiscoveryIndex } from "./discovery-index-client.js";
 
 export type ParsedDiscoverArgs =
   | {
@@ -62,6 +63,8 @@ export type DiscoverResult = {
 };
 
 export type RunDiscoverOptions = {
+  /** Read for the discovery-index opt-in gate (#7168) -- defaults to `process.env`. */
+  env?: Record<string, string | undefined>;
   githubToken?: string;
   apiBaseUrl?: string;
   /** Per-tenant credential env var name (#4784); defaults to GITHUB_TOKEN. Overridden by a `--token-env` flag. */
@@ -94,6 +97,9 @@ export type RunDiscoverOptions = {
     rankedIssues: RankedCandidateIssue[],
     options: { queueStore: PortfolioQueueStore },
   ) => EnqueueRankedDiscoverySummary;
+  /** Supplements the local fan-out with hosted discovery-index results for the same scope, when the plane is
+   *  enabled (#7168). Defaults to discovery-index-client.js's own queryDiscoveryIndex. */
+  queryDiscoveryIndex?: typeof queryDiscoveryIndex;
   /** Invoked with the real structured result at each success return point (dry-run and full-run), in addition
    *  to (never instead of) the plain exit-code return -- mirrors `RunAttemptOptions.onResult`. Never fires on a
    *  parse-error/unexpected-error `reportCliFailure` branch, matching runAttempt's own asymmetry (#6522). */

packages/loopover-miner/lib/discover-cli.js

@@ -15,6 +15,7 @@ import { extractContributionProfile } from "./contribution-profile-extract.js";
 import { initContributionProfileCache } from "./contribution-profile-cache.js";
 import { filterCandidatesByProfiles } from "./contribution-profile-filter.js";
 import { argsWantJson, describeCliError, reportCliFailure } from "./cli-error.js";
+import { isDiscoveryPlaneEnabled, queryDiscoveryIndex, recordDiscoveryTelemetry } from "./discovery-index-client.js";
 
 const DISCOVER_USAGE =
   "Usage: loopover-miner discover <owner/repo> [<owner/repo>...] | --search <query> [--dry-run] [--json] [--api-base-url <url>] [--token-env <VAR>]";
@@ -36,6 +37,35 @@ export function sanitizeDiscoverDisplayText(value) {
     .slice(0, MAX_DISCOVER_TITLE_DISPLAY_LENGTH);
 }
 
+function dedupeKey(repoFullName, issueNumber) {
+  return `${repoFullName.toLowerCase()}#${issueNumber}`;
+}
+
+/**
+ * Supplements `fanOut.issues` with hosted discovery-index results for the same scope (#7168) -- a complete
+ * no-op (returns `fanOut` unchanged) unless the plane is enabled, so a run with the flag unset behaves exactly
+ * as before this feature existed. Local results always win on a duplicate issue (the discovery-index candidate
+ * is dropped, not merged over it) -- this instance's own live fan-out is more current than a cached shared
+ * index entry. Discovery-index candidates lack `assignees` (not part of the public contract), so they're
+ * annotated with an empty array to match opportunity-fanout.js's own candidate shape; contribution-profile-
+ * filter.js's assignee-exclusion rule treats that identically to "no assignees on this issue".
+ */
+async function supplementWithDiscoveryIndex(fanOut, queryScope, options) {
+  const env = options.env ?? process.env;
+  if (!isDiscoveryPlaneEnabled(env)) return fanOut;
+  const queryIndex = options.queryDiscoveryIndex ?? queryDiscoveryIndex;
+  const response = await queryIndex(queryScope, { env });
+  recordDiscoveryTelemetry("discover_query", response.candidates.length > 0 ? "supplemented" : "empty", { env });
+  if (response.candidates.length === 0) return fanOut;
+
+  const seen = new Set(fanOut.issues.map((issue) => dedupeKey(issue.repoFullName, issue.issueNumber)));
+  const supplemented = response.candidates
+    .filter((candidate) => !seen.has(dedupeKey(candidate.repoFullName, candidate.issueNumber)))
+    .map((candidate) => ({ ...candidate, assignees: [] }));
+  if (supplemented.length === 0) return fanOut;
+  return { ...fanOut, issues: [...fanOut.issues, ...supplemented] };
+}
+
 function parseRepoTarget(value) {
   const trimmed = typeof value === "string" ? value.trim() : "";
   const [owner, repo, extra] = trimmed.split("/");
@@ -211,6 +241,12 @@ export async function runDiscover(args, options = {}) {
   // Eligibility filtering (#6798): resolve each candidate repo's ContributionProfile and drop candidates the
   // repo's own conventions would reject, BEFORE ranking. Safe by default -- see resolveContributionProfilesForDiscover.
   const resolveProfiles = options.resolveContributionProfiles ?? resolveContributionProfilesForDiscover;
+  // Same scope this run already asks GitHub about (#7168) -- the discovery-index supplement, when enabled,
+  // asks the shared hosted index about the identical targets/search rather than a different query entirely.
+  const discoveryQueryScope =
+    parsed.search !== null
+      ? { repos: [], orgs: [], searchTerms: [parsed.search] }
+      : { repos: parsed.targets.map((target) => `${target.owner}/${target.repo}`), orgs: [], searchTerms: [] };
 
   // #4847: fetch + rank are read-only GitHub GETs and pure local computation, so a dry run still does them for
   // real (that's the useful "what would this discover?" output) -- but it never opens any local store (portfolio
@@ -220,10 +256,11 @@ export async function runDiscover(args, options = {}) {
   if (parsed.dryRun) {
     const fanOutOptions = { apiBaseUrl, forge: options.forge, policyDocCache: null, policyVerdictCache: null };
     try {
-      const fanOut =
+      let fanOut =
         parsed.search !== null
           ? await searchTargets(parsed.search, githubToken, fanOutOptions)
           : await fetchTargets(parsed.targets, githubToken, fanOutOptions);
+      fanOut = await supplementWithDiscoveryIndex(fanOut, discoveryQueryScope, options);
       // #6798: same eligibility filter as the real path, so a dry run shows the exact candidate set a real run
       // would enqueue (and the same excluded set), rather than an unfiltered preview.
       const repoFullNames = [...new Set(fanOut.issues.map((issue) => issue.repoFullName))];
@@ -322,10 +359,11 @@ export async function runDiscover(args, options = {}) {
   const fanOutOptions = { apiBaseUrl, forge: options.forge, policyDocCache, policyVerdictCache };
 
   try {
-    const fanOut =
+    let fanOut =
       parsed.search !== null
         ? await searchTargets(parsed.search, githubToken, fanOutOptions)
         : await fetchTargets(parsed.targets, githubToken, fanOutOptions);
+    fanOut = await supplementWithDiscoveryIndex(fanOut, discoveryQueryScope, options);
 
     // Eligibility filter (#6798): drop candidates a target repo's own conventions would reject, before ranking.
     // A repo with no trustworthy eligibility profile keeps every candidate (filterCandidatesByProfiles' safe

packages/loopover-miner/lib/discovery-index-client.d.ts

@@ -0,0 +1,43 @@
+import type { DiscoveryIndexQuery, DiscoveryIndexResponse } from "@loopover/engine";
+
+export const DISCOVERY_PLANE_FLAG: string;
+export const DISCOVERY_INDEX_URL_FLAG: string;
+export const DISCOVERY_TELEMETRY_FLAG: string;
+
+export type DiscoveryIndexClientOptions = {
+  env?: Record<string, string | undefined>;
+  /** Always called as `fetchImpl(url, init)` with a plain string URL -- narrower than `typeof fetch` on
+   *  purpose, since that's the only shape this module ever actually calls it with. */
+  fetchImpl?: (url: string, init: RequestInit) => Promise<Response>;
+  requestTimeoutMs?: number;
+};
+
+/** The shape claim-ledger.js's rowToClaim (and claimIssueWithinCap(...).claim) already produces -- passed
+ *  straight into @loopover/engine's buildSoftClaimRequest with no translation. */
+export type SoftClaimLedgerRecord = {
+  repoFullName: string;
+  issueNumber: number;
+  claimedAt: string;
+  status: "active" | "released" | "expired";
+  note?: string | null;
+};
+
+export function isDiscoveryPlaneEnabled(env?: Record<string, string | undefined>): boolean;
+
+export function isDiscoveryTelemetryEnabled(env?: Record<string, string | undefined>): boolean;
+
+export function queryDiscoveryIndex(
+  query: Partial<DiscoveryIndexQuery>,
+  options?: DiscoveryIndexClientOptions,
+): Promise<DiscoveryIndexResponse>;
+
+export function submitSoftClaim(
+  claim: SoftClaimLedgerRecord,
+  options?: DiscoveryIndexClientOptions,
+): Promise<{ sent: boolean }>;
+
+export function recordDiscoveryTelemetry(
+  event: string,
+  outcome: string,
+  options?: { env?: Record<string, string | undefined> },
+): void;