[otel-integration] Add span metrics preset (#334)

* Remove spanmetricsprocessor; add preset Signed-off-by: Matej Gera <[email protected]> * Update README Signed-off-by: Matej Gera <[email protected]> * Bump chart version, add CHANGELOG Signed-off-by: Matej Gera <[email protected]> * Add UPGRADING section Signed-off-by: Matej Gera <[email protected]> * Formatting fixes Signed-off-by: Matej Gera <[email protected]> --------- Signed-off-by: Matej Gera <[email protected]>
coralogix · Nov 28, 2023 · 9f58190 · 9f58190
1 parent c6eb1b1
commit 9f58190
Show file tree

Hide file tree

Showing 6 changed files with 46 additions and 21 deletions.
diff --git a/logs/CHANGELOG.md b/logs/CHANGELOG.md
@@ -5,7 +5,7 @@
 ### v1.16.3 / 2023-11-23
 
 * [CHANGE] Update the coralogix API
-  
+
 ### v1.16.3 / 2023-11-22
 
 * [UPGRADE] Upgrade Fluentd version to v1.16.3

diff --git a/otel-integration/CHANGELOG.md b/otel-integration/CHANGELOG.md
@@ -2,6 +2,11 @@
 
 ## OpenTelemtry-Integration
 
+### v0.0.37 / 2023-11-27
+* [:warning: BREAKING CHANGE] [FEATURE] Add support for span metrics preset. This replaces the deprecated `spanmetricsprocessor`
+  with `spanmetricsconnector`. The new connector is disabled by default, as opposed the replaces processor.
+  To enable it, set `presets.spanMetrics.enabled` to `true`.
+
 ### v0.0.36 / 2023-11-15
 * [FIX] Change statsd receiver port to 8125 instead of 8127
 

diff --git a/otel-integration/UPGRADING.md b/otel-integration/UPGRADING.md
@@ -4,6 +4,19 @@ These upgrade guidelines only contain instructions for version upgrades which re
 If the version you want to upgrade to is not listed here, then there is nothing to do for you.
 Just upgrade and enjoy.
 
+## 0.0.36 to 0.0.37
+
+From version `0.0.37`, the deprecated `spanmetricsprocessor` has been removed and replaced by the `spanmetricsconnector` and is **disabled** by default. If you depend on the span metrics, please enable the `spanmetricsconnector` by setting the following in the `presets` section of your configuration:
+
+```yaml
+opentelemetry-agent:
+  presets:
+    spanMetrics:
+      enabled: true
+```
+
+Please beware there are also breaking changes with regards to metric naming and labels. Please see the [this section of documentation](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/connector/spanmetricsconnector/README.md#span-to-metrics-processor-to-span-to-metrics-connector) for more information.
+
 ## 0.0.32 to 0.0.33
 
 Opentelemetry Integration v0.33 has removed Kube State Metrics deployment, as the needed resource attributes and metrics are now part of OpenTelemetry Kubernetes Cluster Receiver.

diff --git a/otel-integration/k8s-helm/Chart.yaml b/otel-integration/k8s-helm/Chart.yaml
@@ -1,7 +1,7 @@
 apiVersion: v2
 name: otel-integration
 description: OpenTelemetry Integration
-version: 0.0.36
+version: 0.0.37
 keywords:
   - OpenTelemetry Collector
   - OpenTelemetry Agent
@@ -11,17 +11,17 @@ keywords:
 dependencies:
   - name: opentelemetry-collector
     alias: opentelemetry-agent
-    version: "0.77.0"
+    version: "0.77.2"
     repository: https://cgx.jfrog.io/artifactory/coralogix-charts-virtual
     condition: opentelemetry-agent.enabled
   - name: opentelemetry-collector
     alias: opentelemetry-agent-windows
-    version: "0.77.0"
+    version: "0.77.2"
     repository: https://cgx.jfrog.io/artifactory/coralogix-charts-virtual
     condition: opentelemetry-agent-windows.enabled
   - name: opentelemetry-collector
     alias: opentelemetry-cluster-collector
-    version: "0.77.0"
+    version: "0.77.2"
     repository: https://cgx.jfrog.io/artifactory/coralogix-charts-virtual
     condition: opentelemetry-cluster-collector.enabled
 sources:

diff --git a/otel-integration/k8s-helm/README.md b/otel-integration/k8s-helm/README.md
@@ -48,7 +48,7 @@ This cluster collector provides:
 
 ## Kubernetes Dashboard
 
-This chart will also collect, out of the box, all the metrics necessary for [Coralogix Kubernetes Monitoring](https://coralogix.com/docs/apm-kubernetes/), which will allow you to monitor your Kubernetes cluster and applications. To do this, it is necessary to deploy the [Kube-state-metrics](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-state-metrics) component, which makes it possible to obtain some of these extra metrics.
+This chart will also collect, out of the box, all the metrics necessary for [Coralogix Kubernetes Monitoring](https://coralogix.com/docs/apm-kubernetes/), which will allow you to monitor your Kubernetes cluster and applications.
 
 **Please be aware** that certain metrics collected by for the dashboard have high cardinality, which means that the number of unique values for a given metric is high and might result in higher costs connected with metrics ingestion and storage. This is applies in particular to the pod related metrics `kube_pod_status_reason`, `kube_pod_status_phase` and `kube_pod_status_qos_class`.
 
@@ -137,7 +137,7 @@ helm upgrade --install otel-coralogix-integration coralogix-charts-virtual/otel-
 
 ### Installing the chart on clusters with mixed operating systems (Linux and Windows)
 
-Installing `otel-integration` is also possible on clusters that support running Windows workloads on Windows node alongside Linux nodes (such as [EKS](https://docs.aws.amazon.com/eks/latest/userguide/windows-support.html), [AKS](https://learn.microsoft.com/en-us/azure/aks/windows-faq?tabs=azure-cli) or [GKE](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-cluster-windows)). The `kube-state-metrics` and collector will be installed on Linux nodes, as these components are supported only on Linux operating systems. Conversely, the agent will be installed on both Linux and Windows nodes as a daemonset, in order to collect metrics for both operating systems. In order to do so, the chart needs to be installed with few adjustments.
+Installing `otel-integration` is also possible on clusters that support running Windows workloads on Windows node alongside Linux nodes (such as [EKS](https://docs.aws.amazon.com/eks/latest/userguide/windows-support.html), [AKS](https://learn.microsoft.com/en-us/azure/aks/windows-faq?tabs=azure-cli) or [GKE](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-cluster-windows)). The collector will be installed on Linux nodes, as these components are supported only on Linux operating systems. Conversely, the agent will be installed on both Linux and Windows nodes as a daemonset, in order to collect metrics for both operating systems. In order to do so, the chart needs to be installed with few adjustments.
 
 First make sure to add our Helm charts repository to the local repos list with the following command:
 
@@ -189,6 +189,24 @@ env:
     value: "$(NODE):4317"
 ```
 
+### About span metrics
+
+The collector provides a possibility to synthesize R.E.D (Request, Error, Duration) metrics based on the incoming span data. This can be useful to obtain extra metrics about the operations you have instrumented for tracing. For more information, please refer to the [OpenTelemetry Collector documentation](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/connector/spanmetricsconnector/README.md).
+
+This feature is disabled by default and can be enabled by setting the `spanmetrics.enabled` value to `true` in the `values.yaml` file.
+
+Beware that enabling the feature will result in creation of additional metrics. Depending on how you instrument your applications, this can result in a significant increase in the number of metrics. This is especially true for cases where the span name includes specific values, such as user IDs or UUIDs. Such instrumentation practice is [strongly discouraged](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/api.md#span).
+
+In such cases, we recommend to either correct your instrumentation or to use the `spanMetrics.spanNameReplacePattern` parameter, to replace the problematic values with a generic placeholder. For example, if your span name corresponds to template `user-1234`, you can use the following pattern to replace the user ID with a generic placeholder. See the following configuration:
+
+```yaml
+spanNameReplacePattern: 
+- regex: "user-[0-9]+"
+  replacement: "user-{id}"
+```
+
+This will result in your spans having generalized name `user-{id}`.
+
 # Performance of the Collector
 
 ## Picking the right tracing SDK span processor
@@ -348,4 +366,4 @@ Optional settings:
 
 # Dependencies
 
-This chart uses [openetelemetry-collector](https://github.com/coralogix/opentelemetry-helm-charts/tree/main/charts/opentelemetry-collector) Helm chart. Also this chart currently depends on the [`kube-state-metrics`](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-state-metrics) chart to collect extra Kubernetes metrics.
+This chart uses [openetelemetry-collector](https://github.com/coralogix/opentelemetry-helm-charts/tree/main/charts/opentelemetry-collector) Helm chart.
diff --git a/otel-integration/k8s-helm/values.yaml b/otel-integration/k8s-helm/values.yaml
@@ -89,6 +89,8 @@ opentelemetry-agent:
       enabled: true
     kubeletMetrics:
       enabled: true
+    spanMetrics:
+      enabled: false
 
   config:
     extensions:
@@ -153,18 +155,6 @@ opentelemetry-agent:
             - "k8s.job.name"
             - "k8s.pod.name"
             - "k8s.node.name"
-      spanmetrics:
-        metrics_exporter: coralogix
-        dimensions:
-          # replace the below by `k8s.deployment.name` after https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/23067
-          - name: "k8s.replicaset.name"
-          - name: "k8s.statefulset.name"
-          - name: "k8s.daemonset.name"
-          - name: "k8s.cronjob.name"
-          - name: "k8s.job.name"
-          - name: "k8s.container.name"
-          - name: "k8s.node.name"
-          - name: "k8s.namespace.name"
       # Will get the k8s resource limits
       memory_limiter: null
 
@@ -220,7 +210,6 @@ opentelemetry-agent:
             - resourcedetection/env
             - resourcedetection/region
             - memory_limiter
-            - spanmetrics
             - batch
           receivers:
             - otlp