Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

k8s-infra-metrics not working in Digitalocean kubernetes (sigNoz v0.19.0) #13

Open
kmpartner opened this issue Jun 3, 2023 · 5 comments

Comments

@kmpartner
Copy link

kmpartner commented Jun 3, 2023

I have tried to use dashboards in sigNoz (v0.19.0) that is deployed in DigitalOcean Kubernetes.
When add k8s-metrics (cpu & memory, kubernetes-metrics) dashboard added, it show panels but most of contents of the panels did not display. Dashboard of hostmetrics-k8s.json seems to work and display most of data.
How can I solve this problems?

When I tried to use k8s-metrics dashboard in locally running minikube, part of metrics displayed (not all).

Are these problem in configuration of sigNoz or dashboards json contents?

After installing metrics-server in kubernetes, I could get memory and cpu usage data using kubectl top pod or kubectl describe podMetrics commands.
Are there ways to import data of kubectl top or kubectl describe podMetrics command or metrics-server into dashboard?

@prashant-shahi
Copy link
Member

@kmpartner I have faced this issue as well in DigitalOcean Kubernetes.
kubeletstats receiver is unable to fetch metrics since the kubelet API port of the host is not accessible from within the container.

It would take some time to find proper solution around this.
Meanwhile, you can use cadvisor instead to get K8s metrics.
However, the same dashboard will likely not work since the metrics would be named differently.
https://github.com/google/cadvisor/tree/master/deploy/kubernetes

Tracking issue in OpenTelemetry: open-telemetry/opentelemetry-collector-contrib#22843

@kmpartner
Copy link
Author

kmpartner commented Jun 18, 2023

Thank you for response.
Although I managed to send cadvisor data to sigNoz via prometheus reciever, it is very difficult to handle the data in dashboard because of limited knowledge of signoz and prometheus.

@prashant-shahi
Copy link
Member

@kmpartner There is an option to import Grafana dashboard in SigNoz.

image

You could try importing Grafana dashboard JSON from here.

@ankitnayan
Copy link
Contributor

ankitnayan commented Jun 18, 2023

@kmpartner There is an option to import Grafana dashboard in SigNoz.

This usually never works as most of the dashboards have variables and SigNoz does not support converting Grafama dashboard variables today. Users will have to create dashboard variables in SigNoz using clickhouse query as guided at https://signoz.io/docs/userguide/manage-variables/

@kmpartner
Copy link
Author

Thank you for responses,
After importing cadvisor metrics and using prometheus query in sigNoz dashboard, CPU and memory usage of pod or container started to be displayed in sigNoz dashboard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants