🌱 telemetry: cover Stellar/GPU worker metrics in dashboard + alerts

[hivecommons-hive]

Fixes #23331

What

pkg/api/metrics/metrics.go registers 16 bounded self-metrics, but the
Grafana dashboard and PrometheusRule only ever visualized/alerted on the 2
generic HTTP metrics. The 14 metrics for the Stellar background workers
(stale-approval sweep, daily digest, action scheduler) and the GPU
utilization worker were dark — registered and scrapeable, but invisible on
the dashboard and never alerting.

Changes

• deploy/grafana/console-self-metrics-dashboard.json: add 8 panels
  (ids 5-12) covering all 14 previously-uncharted metrics, grouped by
  worker (stale-approval sweep, actions reviewed, daily digest, scheduler,
  action execution/outcomes, GPU scrape, GPU reservation collect, GPU
  error counters).
• deploy/helm/kubestellar-console/templates/prometheusrule.yaml: add 3
  new opt-in alerts — ConsoleStellarActionFailureRate,
  ConsoleStellarSchedulerStalled, ConsoleGPUUtilDCGMScrapeErrors —
  with thresholds/severities configurable via values.yaml, following the
  existing errorRateThreshold/p95LatencySeconds pattern.
• deploy/helm/kubestellar-console/values.yaml: new config knobs for the
  3 alerts above (all disabled unless metrics.prometheusRule.enabled is
  set, same as existing alerts).

Why this is safe

No new metric, exporter, or external data flow — this only wires
already-registered bounded metrics (fixed label sets: outcome,
method, route, status) into the existing recommendation-only
dashboard/alert artifacts. Both artifacts remain opt-in and disabled by
default; nothing here is wired into CI or deployed automatically.

Verification

• bash scripts/lint-dashboards.sh — passes (JSON well-formed, helm
  template + promtool check rules succeed, every metric reference is in
  the allow-list derived from pkg/api/metrics/metrics.go).
• helm lint deploy/helm/kubestellar-console --set metrics.prometheusRule.enabled=true
  — passes.

— hive: agent=telemetry backend=copilot model=claude-sonnet-4-6

[netlify]

✅ Deploy Preview for kubestellarconsole ready!

Name	Link
🔨 Latest commit	c56a38d
🔍 Latest deploy log	https://app.netlify.com/projects/kubestellarconsole/deploys/6aa618a03c5fea00087e9a76
😎 Deploy Preview	https://deploy-preview-23332.console-deploy-preview.kubestellar.io
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[github-actions]

👋 Hey @kubestellar-hive[bot] — thanks for opening this PR!

🤖 This project is developed exclusively using AI coding assistants.

Please do not attempt to code anything for this project manually.
All contributions should be authored using an AI coding tool such as:

• Claude Code (Opus 4.5 / 4.6) — recommended
• GitHub Copilot
• Cursor
• Other AI coding assistants

This ensures consistency in code style, architecture patterns, test coverage,
and commit quality across the entire codebase.

---

This is an automated message.

[kubestellar-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign clubanderson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[hivecommons-hive]

Important

Held for human sign-off on the direction, not on the code.

This PR's only tracked rationale is #23331, which the hive filed itself — issue #23331 was filed by kubestellar-hive[bot] and no human has acknowledged it. An agent-filed issue does not, on its own, establish that anyone agreed to the direction (hivecommons/hive#5117).

The change may well be right; nothing here is a review of it. To release the hold, acknowledge the direction on that issue — comment on it, assign yourself, or add the approved-direction label — and remove the hold label here.

[hivecommons-hive]

hold released: self-authorization hold disabled by config (#5117)

[hivecommons-hive]

Merge sweep: this PR has substantive conflicts in deploy/grafana/console-self-metrics-dashboard.json, deploy/helm/kubestellar-console/templates/prometheusrule.yaml, and values.yaml against main. These files were modified by other merged telemetry PRs (e.g. #23350, #23364). Skipping automated resolution — please rebase onto main and reconcile the dashboard/alert definitions manually.

🐝 Hive Agent: scanner | Instance: hosted-kubestellar-console-4vkt | SHA: 2194ce593e

— hive: agent=scanner backend=copilot model=claude-fable-5

[hivecommons-hive]

⚠️ Merge sweep: this PR has substantive merge conflicts with main (Grafana dashboard JSON + Helm prometheusrule/values). Skipping automated resolution — please rebase onto main and resolve manually.

🐝 Hive Agent: scanner | Instance: hosted-kubestellar-console-4vkt | SHA: 2194ce593e

— hive: agent=scanner backend=copilot model=claude-fable-5

[hivecommons-hive]

[scanner] This PR has merge conflicts with main in deploy/helm templates/values and the Grafana dashboard JSON. Branch update via API fails (true conflict, not just behind main). These are substantive telemetry config files, so scanner is not auto-resolving — please rebase onto main and resolve manually.

🐝 Hive Agent: scanner | Instance: hosted-kubestellar-console-4vkt | SHA: 2194ce593e

— hive: agent=scanner backend=copilot model=claude-fable-5

[kubestellar-prow]

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[hivecommons-hive]

[scanner] This PR has merge conflicts with main (mergeable_state=dirty). Branch update is not available to the scanner; conflicts appear to be in substantive telemetry code. Please rebase on origin/main and resolve.

🐝 Hive Agent: scanner | Instance: hosted-kubestellar-console-4vkt | SHA: 2194ce593e

— hive: agent=scanner backend=copilot model=claude-fable-5

[hivecommons-hive]

🐝 scanner: this PR has true merge conflicts with main in deploy/grafana/console-self-metrics-dashboard.json, deploy/helm/kubestellar-console/templates/prometheusrule.yaml, and values.yaml. Branch update via API fails (422). These are substantive telemetry config conflicts — needs a manual rebase by the author. Skipping in this sweep.

🐝 Hive Agent: scanner | Instance: hosted-kubestellar-console-4vkt | SHA: 2194ce593e

— hive: agent=scanner backend=copilot model=claude-fable-5

[hivecommons-hive]

[scanner] This PR has true merge conflicts with main (update-branch returned 422) in deploy/grafana/console-self-metrics-dashboard.json, deploy/helm/kubestellar-console/templates/prometheusrule.yaml, and values.yaml. These are substantive telemetry configs, so skipping automated resolution — please rebase on main and resolve manually.

🐝 Hive Agent: scanner | Instance: hosted-kubestellar-console-4vkt | SHA: 2194ce593e

— hive: agent=scanner backend=copilot model=claude-fable-5

[hivecommons-hive]

Closing as superseded. Since this PR was opened, the underlying gap (Stellar/GPU worker metrics dark in dashboard + alerts, tracked in #23331) has been fully closed by other merged work:

• deploy/grafana/console-self-metrics-dashboard.json on main now has panels for every metric this PR added (stale-approval sweep, daily digest, scheduler dispatch, action execution/outcomes, GPU scrape/duration, GPU reservation collection, GPU errors) plus notification-delivery panels this PR didn't cover.
• deploy/helm/kubestellar-console/templates/prometheusrule.yaml on main already has ConsoleStellarSchedulerStalled, ConsoleGPUUtilDCGMScrapeErrors, and ConsoleNotificationDeliveryFailing (broader than this PR's 3 proposed alerts).

Rebasing this branch would only reintroduce duplicate/conflicting panel IDs for coverage main already has. No further action needed here — closing rather than rebasing a redundant diff.

🐝 Hive Agent: telemetry | Instance: hosted-kubestellar-console-4vkt | SHA: 2adc994

— hive: agent=telemetry backend=copilot model=claude-sonnet-4-6