Multi Cluster Observability AddOn

Description

The multicluster-observability-addon is a pluggable addon working on OCM based on the extensibility provided by addon-framework which automates the collection and forwarding of observability signals to central stores.

This is achieved through the installation of the spoke clusters of dedicated operators for each observability signal:

• For Metrics it's required that the multicluster-observability-operator is installed. Once enalbed the addon will create the necessary resources to configure metrics collection.

• For Logs the operator installed will be cluster-logging-operator. The addon will also configure an instance of ClusterLogForwarder to forward logs to a configured store.

• For Traces the operator installed will be Red Hat build of OpenTelemetry. The addon will also configure an instance of OpenTelemetryCollector to forward traces to a configued store.

The logging-ocm-addon consists of one component:

• Addon-Manager: Not only manages the installation of the AddOn on spoke clusters. But also builds the manifests that will be deployed to the spoke clusters.

Getting started

Prerequisite

• OCM registration (>= 0.5.0)

Steps

Installing via Kustomize

1. Install the AddOn using Kustomize

   make install-crds
   kubectl apply -k deploy/

2. The addon should now be installed in you hub cluster

   kubectl get ClusterManagementAddOn multicluster-observability-addon

3. The addon will install automatically in spoke clusters once the resources referenced in ClusterManagementAddOn are created.

Installing via MCO

In 2.12, multicluster-observability-operator has the ability to install MCOA using the capabilities field.

1. Create a MultiClusterObservability resource and configure capabilities

   apiVersion: observability.open-cluster-management.io/v1beta2
   kind: MultiClusterObservability
   metadata:
     name: observability
   spec:
     capabilities:
       platform:
         logs:
           collection:
             enabled: true
       userWorkloads:
         logs:
           collection:
             clusterLogForwarder:
               enabled: true
         traces:
           collection:
             instrumentation:
               enabled: true
             openTelemetryCollector:
               enabled: false
     observabilityAddonSpec: {}
     storageConfig:
       metricObjectStorage:
         name: thanos-object-storage
         key: thanos.yaml

   Note: Deploy a custom image by adding the annotation: mco-multicluster_observability_addon-image: quay.io/YOUR_ORG_HERE/multicluster-observability-addon:YOUR_TAG_HERE

2. The addon should now be installed in you hub cluster

   kubectl get ClusterManagementAddOn multicluster-observability-addon

3. The addon will install automatically in spoke clusters once the resources referenced in ClusterManagementAddOn are created.

Default configurations references

Effective configuration references in ManagedClusterAddOn.status.configReferences must point to open-cluster-management-observability (trusted shared configurations) or the namespace of that ManagedClusterAddOn (per-cluster configurations). Empty namespaces and references to any other namespace are rejected before manifest generation, including overrides supplied through spec.configs. Treat configurations and their referenced Secrets in the shared namespace as available for distribution to managed clusters; keep unrelated hub credentials outside that namespace.

apiVersion: addon.open-cluster-management.io/v1beta1
kind: ClusterManagementAddOn
spec:
  installStrategy:
    type: Placements
    placements:
      - name: <placement_name> # Use global for selecting all clusters
        namespace: open-cluster-management-global-set
        configs:
          - group: observability.openshift.io
            resource: clusterlogforwarders
            name: instance
            namespace: open-cluster-management-observability
          - group: opentelemetry.io
            resource: opentelemetrycollectors
            name: instance
            namespace: open-cluster-management-observability
          - group: opentelemetry.io
            resource: instrumentations
            name: instance
            namespace: open-cluster-management-observability
          # Default metrics forwarding configuration for the ACM platform metrics collector
          - group: monitoring.coreos.com
            resource: prometheusagents
            name: acm-platform-metrics-collector-default
            namespace: open-cluster-management-observability
          - group: monitoring.coreos.com
            resource: scrapeconfigs
            name: platform-metrics-default
            namespace: open-cluster-management-observability
          - group: monitoring.coreos.com
            resource: prometheusrules
            name: platform-rules-default
            namespace: open-cluster-management-observability
          # Default metrics forwarding configuration for the ACM user workload metrics collector
          # There are no default configurations for the scrapeConfigs and prometheusRules
          - group: monitoring.coreos.com
            resource: prometheusagents
            name: acm-user-workload-metrics-collector-default
            namespace: open-cluster-management-observability

References

• Open-Cluster-Management: https://github.com/open-cluster-management-io/ocm
• Addon-Framework: https://github.com/open-cluster-management-io/addon-framework
• Multicluster-Observability-Operator: https://github.com/stolostron/multicluster-observability-operator
• Cluster-Logging-Operator: https://github.com/openshift/cluster-logging-operator
• OpenTelemetry-Operator: https://github.com/open-telemetry/opentelemetry-operator