@@ -36,7 +36,7 @@ function SelfHostingTroubleshooting() {
3636 < CodeBlock
3737 lang = "bash"
3838 code = { `docker compose ps
39- docker compose logs --tail=200 gittensory
39+ docker compose logs --tail=200 loopover
4040curl http://localhost:8787/ready
4141curl http://localhost:8787/metrics` }
4242 />
@@ -122,18 +122,17 @@ rees_analyzer_config_invalid`}
122122 </ p >
123123 < CodeBlock
124124 lang = "bash"
125- code = { `curl http://localhost:8787/metrics | grep gittensory_queue
126- docker compose logs gittensory | grep selfhost_job_dead` }
125+ code = { `curl http://localhost:8787/metrics | grep loopover_queue
126+ docker compose logs loopover | grep selfhost_job_dead` }
127127 />
128128
129129 < h2 > GitHub rate-limit responses or admission deferrals</ h2 >
130130 < p >
131131 Two independent signals cover this:{ " " }
132- < code > gittensory_github_rest_rate_limit_responses_total</ code > counts actual 403/429
133- responses from GitHub, and the{ " " }
134- < code > gittensory_jobs_rate_limit_admission_deferred_total</ code > /{ " " }
135- < code > gittensory_jobs_rate_limit_budget_deferred_total</ code > /{ " " }
136- < code > gittensory_jobs_rate_limited_by_type_total</ code > counters track jobs the queue itself
132+ < code > loopover_github_rest_rate_limit_responses_total</ code > counts actual 403/429 responses
133+ from GitHub, and the < code > loopover_jobs_rate_limit_admission_deferred_total</ code > /{ " " }
134+ < code > loopover_jobs_rate_limit_budget_deferred_total</ code > /{ " " }
135+ < code > loopover_jobs_rate_limited_by_type_total</ code > counters track jobs the queue itself
137136 held back < em > before</ em > making a request, to avoid tripping a limit. All three job-side
138137 counters carry the same three labels — < code > kind</ code > (< code > webhook</ code > or{ " " }
139138 < code > background</ code > ), < code > key_scope</ code > (< code > installation</ code > ,{ " " }
@@ -145,20 +144,20 @@ docker compose logs gittensory | grep selfhost_job_dead`}
145144 A short burst of deferrals is expected and self-resolving: the queue is deliberately trading
146145 a few seconds of delay to avoid a real 429. Treat it as a real problem only once it's
147146 < strong > sustained</ strong > — which is exactly what{ " " }
148- < code > GittensoryGitHubRateLimitResponses </ code > (real 403/429s observed) and{ " " }
149- < code > GittensoryQueueRateLimitDeferralsHigh </ code > (a sustained deferral rate, not a blip)
150- are tuned to alert on, rather than firing on every brief admission hold.
147+ < code > LoopoverGitHubRateLimitResponses </ code > (real 403/429s observed) and{ " " }
148+ < code > LoopoverQueueRateLimitDeferralsHigh </ code > (a sustained deferral rate, not a blip) are
149+ tuned to alert on, rather than firing on every brief admission hold.
151150 </ p >
152151 < CodeBlock
153152 lang = "promql"
154153 code = { `# Deferrals broken down by token pool and job type over the last 10m
155- sum by (key_scope, job_type) (rate(gittensory_jobs_rate_limit_admission_deferred_total [10m]))
154+ sum by (key_scope, job_type) (rate(loopover_jobs_rate_limit_admission_deferred_total [10m]))
156155
157156# Is one key_scope (e.g. a single installation token) the bottleneck?
158- topk(5, sum by (key_scope) (rate(gittensory_jobs_rate_limit_budget_deferred_total [10m])))
157+ topk(5, sum by (key_scope) (rate(loopover_jobs_rate_limit_budget_deferred_total [10m])))
159158
160159# Real rate-limit responses from GitHub itself (not just internal deferrals)
161- sum(rate(gittensory_github_rest_rate_limit_responses_total [10m]))` }
160+ sum(rate(loopover_github_rest_rate_limit_responses_total [10m]))` }
162161 />
163162 < p >
164163 If a single < code > key_scope=installation</ code > pool is consistently the bottleneck, the fix
@@ -168,33 +167,33 @@ sum(rate(gittensory_github_rest_rate_limit_responses_total[10m]))`}
168167
169168 < h2 > Low GitHub response-cache hit rate</ h2 >
170169 < p >
171- < code > gittensory_github_response_cache_total </ code > (REST) and{ " " }
172- < code > gittensory_github_graphql_cache_total </ code > (GraphQL) both carry a{ " " }
173- < code > result </ code > label — < code > hit </ code > , < code > miss </ code > , < code > set </ code > ,{ " " }
174- < code > coalesced </ code > , < code > bypassed</ code > , or < code > error</ code > — and a{ " " }
175- < code > class </ code > label identifying the endpoint family. A healthy cache should show most
176- traffic as < code > hit </ code > for endpoints that are read repeatedly in one review/maintenance
177- pass (PR reads, check-run lookups); a low hit rate on those specific classes, not the
178- overall average, is the useful signal.
170+ < code > loopover_github_response_cache_total </ code > (REST) and{ " " }
171+ < code > loopover_github_graphql_cache_total </ code > (GraphQL) both carry a < code > result </ code > { " " }
172+ label — < code > hit </ code > , < code > miss </ code > , < code > set </ code > , < code > coalesced </ code > ,{ " " }
173+ < code > bypassed</ code > , or < code > error</ code > — and a < code > class </ code > label identifying
174+ the endpoint family. A healthy cache should show most traffic as < code > hit </ code > for
175+ endpoints that are read repeatedly in one review/maintenance pass (PR reads, check-run
176+ lookups); a low hit rate on those specific classes, not the overall average, is the useful
177+ signal.
179178 </ p >
180179 < CodeBlock
181180 lang = "promql"
182181 code = { `# REST hit rate by endpoint class over the last 15m
183- sum by (class) (rate(gittensory_github_response_cache_total {result="hit"}[15m]))
182+ sum by (class) (rate(loopover_github_response_cache_total {result="hit"}[15m]))
184183/
185- sum by (class) (rate(gittensory_github_response_cache_total [15m]))
184+ sum by (class) (rate(loopover_github_response_cache_total [15m]))
186185
187186# GraphQL hit rate — same shape, separate metric
188- sum by (class) (rate(gittensory_github_graphql_cache_total {result="hit"}[15m]))
187+ sum by (class) (rate(loopover_github_graphql_cache_total {result="hit"}[15m]))
189188/
190- sum by (class) (rate(gittensory_github_graphql_cache_total [15m]))` }
189+ sum by (class) (rate(loopover_github_graphql_cache_total [15m]))` }
191190 />
192191
193192 < h2 > Qdrant / vector-store errors</ h2 >
194193 < p >
195- < code > gittensory_qdrant_errors_total </ code > carries an < code > op</ code > label (
194+ < code > loopover_qdrant_errors_total </ code > carries an < code > op</ code > label (
196195 < code > upsert</ code > , < code > query</ code > , or < code > delete</ code > ) so you can tell whether
197- indexing or retrieval is failing. < code > GittensoryQdrantErrorRateHigh </ code > fires on a
196+ indexing or retrieval is failing. < code > LoopoverQdrantErrorRateHigh </ code > fires on a
198197 sustained error ratio, not an isolated blip.
199198 </ p >
200199 < ul >
@@ -226,14 +225,14 @@ curl -X DELETE "$QDRANT_URL/collections/gittensory"`}
226225
227226 < h2 > Orb export or relay problems</ h2 >
228227 < p >
229- For brokered self-host deployments, < code > gittensory_orb_events_exported_total </ code > and{ " " }
230- < code > gittensory_orb_export_errors_total </ code > track the hourly outcome-export loop;{ " " }
231- < code > GittensoryOrbExportErrorRateHigh </ code > fires on a sustained error ratio there. The
228+ For brokered self-host deployments, < code > loopover_orb_events_exported_total </ code > and{ " " }
229+ < code > loopover_orb_export_errors_total </ code > track the hourly outcome-export loop;{ " " }
230+ < code > LoopoverOrbExportErrorRateHigh </ code > fires on a sustained error ratio there. The
232231 pull-mode relay loop (for installations receiving events outbound from Orb) reports through{ " " }
233- < code > gittensory_orb_relay_drains_total </ code > (< code > result=events</ code > when it drained
234- something, < code > result=empty</ code > otherwise) and{ " " }
235- < code > gittensory_orb_webhook_total </ code > (< code > event</ code > + < code > result</ code > labels)
236- for what happened to each relayed event once enqueued locally.
232+ < code > loopover_orb_relay_drains_total </ code > (< code > result=events</ code > when it drained
233+ something, < code > result=empty</ code > otherwise) and < code > loopover_orb_webhook_total </ code > { " " }
234+ (< code > event</ code > + < code > result</ code > labels) for what happened to each relayed event
235+ once enqueued locally.
237236 </ p >
238237 < p >
239238 If exports are failing but the relay itself looks healthy, the export loop's Sentry
@@ -257,7 +256,7 @@ curl -X DELETE "$QDRANT_URL/collections/gittensory"`}
257256 </ li >
258257 < li >
259258 < strong > Pull mode</ strong > — logged at < code > warn</ code > and non-fatal: the drain loop (
260- < code > gittensory_orb_relay_drains_total </ code > ) keeps retrying on its own schedule
259+ < code > loopover_orb_relay_drains_total </ code > ) keeps retrying on its own schedule
261260 regardless, so a transient failure here recovers on its own once the broker is reachable
262261 again. A registration failure that never clears across many retries still points at{ " " }
263262 < code > ORB_ENROLLMENT_SECRET</ code > being wrong, revoked, or not yet provisioned
@@ -274,9 +273,9 @@ curl -X DELETE "$QDRANT_URL/collections/gittensory"`}
274273 < p >
275274 Each AI provider (self-host < code > AI_PROVIDER</ code > entries) has its own circuit breaker:
276275 after 3 consecutive failures it stops attempting real calls to that provider for 60 seconds,
277- recorded as < code > gittensory_ai_provider_circuit_open_total { '{provider="..."}' } </ code > { " " }
276+ recorded as < code > loopover_ai_provider_circuit_open_total { '{provider="..."}' } </ code > { " " }
278277 (skipped calls) alongside{ " " }
279- < code > gittensory_ai_provider_failures_total { '{provider="..."}' } </ code > (real failures). It
278+ < code > loopover_ai_provider_failures_total { '{provider="..."}' } </ code > (real failures). It
280279 self-heals automatically — there is no manual reset — but it will reopen immediately if the
281280 underlying problem is still there.
282281 </ p >
@@ -293,9 +292,9 @@ curl -X DELETE "$QDRANT_URL/collections/gittensory"`}
293292 unreachable from the container.
294293 </ li >
295294 < li >
296- < code > GittensoryAiProviderCircuitOpen </ code > fires on any circuit-open event in a
297- 15-minute window — a single trip during a real but brief outage is expected; a rule that
298- keeps firing across multiple windows points at the persistent case above.
295+ < code > LoopoverAiProviderCircuitOpen </ code > fires on any circuit-open event in a 15-minute
296+ window — a single trip during a real but brief outage is expected; a rule that keeps
297+ firing across multiple windows points at the persistent case above.
299298 </ li >
300299 </ ul >
301300
@@ -326,7 +325,7 @@ npm run test:smoke:observability:metrics`}
326325 </ li >
327326 < li >
328327 If the app's own < code > /metrics</ code > check fails, that is unrelated to the OTEL
329- collector — check the app container directly (< code > docker compose logs gittensory </ code >
328+ collector — check the app container directly (< code > docker compose logs loopover </ code >
330329 ).
331330 </ li >
332331 < li >
0 commit comments