Skip to content

feat(observability): expand self-host Sentry context and monitors #1824

Description

@JSONbored

Parent: #1819

Context

The self-host runtime has optional Sentry wiring, release resolution, source-map support for official/self-built releases, structured-log forwarding, and selected review failure capture. The next step is broader, consistent operational context and monitor coverage for critical review-system paths.

Sentry must remain opt-in. Community self-hosters should send events only to their own DSN if they choose to enable it.

Requirements

  • Keep SENTRY_DSN unset as a complete no-op with negligible overhead.
  • Add safe Sentry context/tags for critical self-host engine paths: webhook ingest, queue claim/process, GitHub token/broker calls, AI provider attempts, gate/check-run publish, comment publish, scheduled sweep, and backup/relay failures.
  • Add Sentry cron monitors or equivalent check-ins for recurring work where silent stoppage is worse than an exception.
  • Define alert classes and thresholds that are actionable without paging on one-off fail-open noise.
  • Scrub all secrets, tokens, auth headers, request bodies, raw diffs, prompts, review text, private repo config, and local auth paths.
  • Keep high-cardinality identifiers out of tags unless safely truncated or intentionally fingerprinted.

Deliverables

  • Sentry context/tag helper or documented taxonomy for self-host operational events.
  • Monitor coverage for the scheduled loop and the highest-value recurring jobs.
  • Alert/runbook documentation for dead-letter growth, check-run permission gaps, AI provider exhaustion, relay failures, backup failures, and scheduled monitor misses.
  • Tests for no-op behavior, scrubbing, and representative capture sites.
  • Website docs describing how self-hosters enable Sentry with their own DSN.

Acceptance criteria

  • A production Sentry issue points to the affected subsystem, repo/PR when applicable, release, environment, job type, and safe reason code.
  • No sensitive payloads or private config values are present in captured events.
  • Operators can distinguish transient degradation from persistent failure classes.
  • Sentry remains disabled by default for community self-hosters.

Validation

  • Unit tests for scrubbing/tag allowlists and no-DSN behavior.
  • Focused smoke test or controlled failure confirming context appears in Sentry when enabled.
  • Docs build and relevant self-host observability tests pass.

Ownership

Maintainer-only. No gittensor:* labels.

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:featureGittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.

    Projects

    Status
    Done

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions