feat(metrics): expose ingestion-lag gauge at /metrics (Closes #237) - #453
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Completes #237 — the ingestion-lag gauge (
sorotrail_ingestion_lag_ledgers) is now actually served at the/metricsendpoint and covered by table-driven tests.The gauge itself (
latest_rpc_ledger − last_ingested_ledger) is already tracked by the ingester viasetIngestionLag, called after every poll pass that knows the chain head. What was missing:/metrics— the API server's/metricshandler (HTTPMetrics.Handler) only gathered its per-server histogram registry, so the globally-registeredsorotrail_ingestion_lag_ledgersgauge was never exported. It now gathersprometheus.DefaultGathereralongside the per-server registry, so a single scrape sees the whole picture.Handler()ininternal/metrics/metrics.gowas missing its closing brace (leftover from an earlier bad merge), so the package did not compile. Fixed here./metricsexport path.Tests
internal/metrics/metrics_test.go—TestIngestionLagGauge_ExportedAtMetrics: asserts the exact chain-head-minus-last-ingested value is exported at/metrics(caught up, small lag, deep lag, nothing-ingested-yet).internal/ingester/ingester_test.go—TestSetIngestionLag: assertssetIngestionLagcomputeschainHead − lastIngestedexactly, is a no-op when either side is unknown (≤ 0), and preserves negative lag when a replay runs ahead of the reported head.internal/api/server_test.go—TestMetrics_ExposesIngestionLagGauge: end-to-end assertion that the API/metricsendpoint serves the gauge.All new tests pass locally.
Validation
go build ./...✅go vet ./...✅golangci-lint run— no new issues from this change; the command currently reports pre-existing findings in unrelated in-flight work (cmd/sorotrail/main.go,internal/api/*,internal/ingester/ingester.gounused symbols / goimports).go test ./...— the packages touched by this change pass; the full suite has pre-existing failures in unrelated WIP code (twoTestEventsIngestedTotal_*cases andpkg/docsopenapi drift) that predate this PR and are out of scope for Add an ingestion-lag gauge #237.Scope
4 files changed, +91/−18. Purely observability: no changes to endpoints, config, or schema.
Closes #237