Skip to content

chore(orb): verify brokered self-host relay and telemetry #1825

Description

@JSONbored

Parent: #1819

Context

Brokered Orb mode lets a self-hosted engine use the central Orb token broker and webhook relay instead of requiring every operator to manage a full GitHub App key path. The runtime also exports anonymized outcome telemetry for fleet calibration.

This path needs an end-to-end operational audit before the self-host stack is presented as officially ready.

Requirements

  • Verify token broker flow for installation token minting, caching, expiry, refresh, and broker outage behavior.
  • Verify webhook relay behavior in push and pull modes, including registration, drain, ack, retry, and stale-event handling.
  • Investigate and document expected handling for relay registration failures, especially when pull mode still drains events.
  • Verify outcome export behavior, anonymization defaults, air-gap mode, retry behavior, and failure visibility.
  • Confirm stable ingress requirements for operators who choose push relay mode.
  • Ensure logs/Sentry context identify relay/token/export failures without exposing tokens, enrollment secrets, raw webhook bodies, private repo config, or unhashed private identifiers when anonymization is enabled.

Deliverables

  • End-to-end audit notes for brokered token, webhook relay, and outcome export paths.
  • Reproducible smoke checklist for both pull-mode and push-mode relay where practical.
  • Clear recommendation for stable ingress versus pull-mode default for NAT/tailnet deployments.
  • Any focused fixes needed for relay registration/degraded telemetry behavior.
  • Website docs updates for GitHub App and Orb setup/troubleshooting.

Acceptance criteria

  • A brokered self-host can receive PR events, process reviews, publish GitHub output, and export outcome telemetry.
  • Relay failures are visible and actionable without falsely implying the engine is healthy when no events can arrive.
  • Pull-mode degraded registration does not page incorrectly when event draining is still functioning.
  • Air-gapped deployments can disable outbound outcome export intentionally.

Validation

  • Brokered token smoke test.
  • Relay drain/register smoke test.
  • Outcome export smoke or controlled test with expected anonymization.
  • Relevant unit tests for failure classification if code changes are needed.

Ownership

Maintainer-only. No gittensor:* labels.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

Status
Done

Relationships

None yet

Development

No branches or pull requests

Issue actions