|
| 1 | +# Hosted discovery plane — operator guide (opt-in) |
| 2 | + |
| 3 | +Operator-facing guide for the **optional** Phase 6 hosted discovery-index plane ([#4250](https://github.com/JSONbored/gittensory/issues/4250)). This is the client/miner half of that roadmap item: how a `gittensory-miner` instance opts in, what it may send, and what never leaves the operator's machine. |
| 4 | + |
| 5 | +> **Placeholder scope:** the concrete env var names, HTTP paths, and telemetry event types below are **provisional** until the sibling contract/telemetry/client issues land. When those ship, update this file to match the real symbols — do not treat the TBD names as stable API yet. |
| 6 | +> |
| 7 | +> | Issue | What it defines | |
| 8 | +> |-------|-----------------| |
| 9 | +> | [#4300](https://github.com/JSONbored/gittensory/issues/4300) | Public-data-only discovery-index API contract (request/response shapes) | |
| 10 | +> | [#4301](https://github.com/JSONbored/gittensory/issues/4301) | Anonymized telemetry event schema for the optional hosted plane | |
| 11 | +> | [#4302](https://github.com/JSONbored/gittensory/issues/4302) | Client-side soft-claim coordination request builder | |
| 12 | +> | [#4250](https://github.com/JSONbored/gittensory/issues/4250) | Hosted discovery-index server (maintainer-operated counterpart) | |
| 13 | +
|
| 14 | +Part of the Miner Wave 2 discovery plane ([#2353](https://github.com/JSONbored/gittensory/issues/2353) Phase 6). Distinct from Phase 1's **local-only** metadata fan-out documented in [`cross-repo-discovery-phase1.md`](cross-repo-discovery-phase1.md) — that path never phones home today. |
| 15 | + |
| 16 | +## Default posture: opt-in (not like Orb) |
| 17 | + |
| 18 | +Two telemetry/export surfaces exist in Gittensory, and they intentionally use **opposite defaults**: |
| 19 | + |
| 20 | +| Surface | Default | Operator action | Precedent | |
| 21 | +|---------|---------|-------------------|-----------| |
| 22 | +| **Orb fleet calibration** (`src/selfhost/orb-collector.ts`) | **ON** once a GitHub App is configured | Opt out only via `ORB_AIR_GAP=true` (air-gapped / send-nothing) | Review-stack self-host contract — export is always on unless air-gapped | |
| 23 | +| **Hosted discovery plane** (this guide) | **OFF** | Opt **in** explicitly before any hosted index query or plane telemetry | Hybrid, self-host-first miner deployment — participation in a shared hosted plane is never assumed | |
| 24 | + |
| 25 | +Do **not** copy Orb's wording for this plane. Orb's header comment is explicit: "Export is ALWAYS ON… there is no opt-out flag" aside from `ORB_AIR_GAP`. The discovery plane is the opposite: **no hosted traffic unless the operator turns it on.** |
| 26 | + |
| 27 | +## What the plane is for |
| 28 | + |
| 29 | +When enabled, a miner may query a **shared, metadata-only** discovery index instead of every fleet member independently fanning out GitHub search/listing calls against the same repos — mitigating cross-fleet rate-limit pressure (the same class of incident addressed for the review stack in [#1936](https://github.com/JSONbored/gittensory/issues/1936)). |
| 30 | + |
| 31 | +The plane: |
| 32 | + |
| 33 | +- Serves **public GitHub metadata only** (issue titles, labels, counts, timestamps, URLs — the same class of fields Phase 1 already uses locally). |
| 34 | +- May coordinate **soft claims** across the fleet (server-side dedup is [#4250](https://github.com/JSONbored/gittensory/issues/4250); client request shape is [#4302](https://github.com/JSONbored/gittensory/issues/4302)). |
| 35 | +- Never receives source trees, diffs, tokens, or write credentials. |
| 36 | + |
| 37 | +Local discovery (`opportunity-fanout` + `opportunity-ranker`) continues to work with **zero** hosted configuration. |
| 38 | + |
| 39 | +## Opt-in mechanism (names TBD — update when #4300/#4301 ship) |
| 40 | + |
| 41 | +Until the contract issue lands, treat these as **documentation placeholders** for the shape operators should expect: |
| 42 | + |
| 43 | +| Variable (provisional) | Default | Purpose | |
| 44 | +|------------------------|---------|---------| |
| 45 | +| `GITTENSORY_MINER_DISCOVERY_PLANE` | unset / `false` | Master opt-in. When not truthy (`1`, `true`, `yes`, `on`), the miner must not call the hosted index or emit discovery-plane telemetry. | |
| 46 | +| `GITTENSORY_MINER_DISCOVERY_INDEX_URL` | unset | Hosted index base URL. Required when the plane is enabled; ignored when opt-in is off. | |
| 47 | +| `GITTENSORY_MINER_DISCOVERY_TELEMETRY` | unset / `false` | Separate opt-in for anonymized operational telemetry ([#4301](https://github.com/JSONbored/gittensory/issues/4301)). Plane queries can stay on while telemetry stays off. | |
| 48 | + |
| 49 | +**Truthy-string convention** (when implemented): `/^(1|true|yes|on)$/i`, matching other `GITTENSORY_*` flags in this repo. |
| 50 | + |
| 51 | +**Operator checklist (enabled plane):** |
| 52 | + |
| 53 | +1. Set `GITTENSORY_MINER_DISCOVERY_PLANE=true` (exact name may change — see [#4300](https://github.com/JSONbored/gittensory/issues/4300)). |
| 54 | +2. Set `GITTENSORY_MINER_DISCOVERY_INDEX_URL` to the operator-trusted index endpoint ([#4250](https://github.com/JSONbored/gittensory/issues/4250)). |
| 55 | +3. Optionally set `GITTENSORY_MINER_DISCOVERY_TELEMETRY=true` if you want anonymized operational events for the hosted service — not required for index queries. |
| 56 | +4. Keep `GITHUB_TOKEN` (or equivalent) on the instance only; never configure tokens intended for the hosted plane to receive. |
| 57 | + |
| 58 | +With opt-in off (default), behavior is byte-identical to today: local SQLite ledgers, local fan-out, no hosted calls. |
| 59 | + |
| 60 | +## Contrast with local soft-claims today |
| 61 | + |
| 62 | +`packages/gittensory-miner/lib/claim-ledger.js` records soft claims **locally only** ("never uploads, syncs, or phones home"). Fleet-wide coordination before work starts is what [#4302](https://github.com/JSONbored/gittensory/issues/4302) + the hosted index ([#4250](https://github.com/JSONbored/gittensory/issues/4250)) add **on top of** that ledger — only after explicit opt-in. |
| 63 | + |
| 64 | +After-the-fact duplicate adjudication (`isDuplicateClusterWinnerByClaim` in `@jsonbored/gittensory-engine`) remains separate; it resolves collisions by observing what publicly landed first, not by preventing overlap up front. |
| 65 | + |
| 66 | +## Invariants |
| 67 | + |
| 68 | +Mirrors [`DEPLOYMENT.md`](../DEPLOYMENT.md) tone — concrete guarantees for operators: |
| 69 | + |
| 70 | +- **Default OFF** — no hosted discovery-index traffic and no discovery-plane telemetry unless the operator opts in. |
| 71 | +- **Metadata-only index queries** — responses are issue/listing metadata compatible with local `normalizeCandidate` shape; no source upload, no clone, no repo archive. |
| 72 | +- **Read-only client posture** — the miner uses GET/list/search semantics toward GitHub directly (Phase 1) and toward the hosted index when enabled; the plane does not grant the miner new GitHub write capability. |
| 73 | +- **Credentials stay local** — GitHub tokens, PATs, and actor-capable secrets are injected at runtime on the operator's machine or secret store; they are **never** included in index or telemetry payloads. |
| 74 | +- **No compensation signals in the plane** — raw reward values, wallet addresses, hotkeys, trust scores, or private rankings never cross this boundary (same public boundary as [`cross-repo-discovery-phase1.md`](cross-repo-discovery-phase1.md) Acceptance). |
| 75 | +- **Telemetry is a second opt-in** — even with the plane enabled, anonymized telemetry ([#4301](https://github.com/JSONbored/gittensory/issues/4301)) remains separately gated. |
| 76 | +- **Anonymized identifiers only** — when telemetry ships, repo/issue correlation uses HMAC-hashed identifiers keyed by a **per-instance dedicated secret** the collector never holds (same posture as `getOrCreateAnonSecret` / `hmacField` in `src/selfhost/orb-collector.ts` — key separation from GitHub App / webhook secrets). |
| 77 | +- **Low-cardinality reason buckets** — any free-text-adjacent telemetry fields use bucketed categories (Orb's `bucketReasonCode` pattern), not raw maintainer or model prose. |
| 78 | +- **Core miner still works offline** — claims, plans, queues, and local ledgers do not require the hosted plane; `gittensory-miner doctor` / `status` remain no-network commands. |
| 79 | + |
| 80 | +### Never included (client → hosted plane) |
| 81 | + |
| 82 | +Inventory style matches `src/selfhost/orb-collector.ts:15-17` ("No diffs, no code…") adapted for discovery-plane domain ([#4301](https://github.com/JSONbored/gittensory/issues/4301)): |
| 83 | + |
| 84 | +- Source file contents, patches, or diffs |
| 85 | +- Full issue/PR bodies or review comments |
| 86 | +- GitHub tokens, PATs, App private keys, or any actor-capable credential |
| 87 | +- Commit SHAs, branch names tied to unpublished work, or CI log excerpts |
| 88 | +- Operator login identities, emails, or hostnames usable as PII |
| 89 | +- Raw gate reasons, model transcripts, or free-text maintainer notes |
| 90 | +- Reward amounts, wallet addresses, hotkeys, trust scores, or private rankings |
| 91 | + |
| 92 | +### Never retained by the hosted service (server-side — [#4250](https://github.com/JSONbored/gittensory/issues/4250)) |
| 93 | + |
| 94 | +The server operating doc (maintainer-only) will restate the same boundary: **never holds source or actor-capable credentials.** This client guide does not define server retention policy; see #4250 deliverables. |
| 95 | + |
| 96 | +## Related docs |
| 97 | + |
| 98 | +- [`cross-repo-discovery-phase1.md`](cross-repo-discovery-phase1.md) — local, metadata-only Phase 1 discovery (no hosted plane). |
| 99 | +- [`miner-goal-spec.md`](miner-goal-spec.md) — per-repo `.gittensory-miner.yml` targeting policy. |
| 100 | +- [`../DEPLOYMENT.md`](../DEPLOYMENT.md) — laptop vs fleet deployment and core miner invariants. |
0 commit comments