Skip to content

observability(tracing): add sampled Sentry spans for review latency #1734

Description

@JSONbored

Part of #998. Related: #1001 and #1207.

Context

We need to answer operational questions like "why did this review take so long?" and "which stage is slow or failing?" without digging through scattered logs. The current Sentry setup is strong for errors; the next layer is sampled tracing across the review pipeline.

This issue is specifically for Sentry spans/traces. It should not replace the existing OpenTelemetry issue; it should either coexist cleanly or reuse the same span boundaries where practical.

Requirements

  • Keep tracing opt-in by sampling config. Default sample rate stays 0 unless explicitly configured.
  • Add spans around the critical review path: webhook ingest, queue claim/process, GitHub API reads/writes, REES call, AI provider attempts, gate planning, comment/check-run publishing, and DB/cache work where it materially affects latency.
  • Tag spans with safe dimensions: repo, PR number, job type, provider/model/effort, analyzer count, outcome, release, and environment.
  • Never attach prompts, diffs, review bodies, tokens, headers, or private configuration blobs to spans.
  • Keep trace cardinality controlled; avoid tags that contain full IDs or unbounded text unless hashed/truncated deliberately.

Deliverables

  • Shared span helper/wrapper so instrumentation is consistent and easy to audit.
  • Instrumentation for self-host and REES call paths where they intersect review latency.
  • Tests or focused smoke coverage proving tracing is no-op when sampling is off.
  • Documentation for enabling a low sample rate and using traces to debug slow reviews.

Acceptance criteria

  • With sampling enabled, one review produces a connected trace with the major stages visible.
  • With sampling disabled, behavior stays unchanged and no Sentry trace traffic is emitted.
  • Slow or failed stages are filterable without exposing sensitive payloads.

Metadata

Metadata

Assignees

Labels

maintainer-onlyWork to be completed solely by jsonbored - yields no gittensor points.

Projects

Status
Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions