tooling cluster

pre-requisites

kubernetes

follow the steps here to install kubectl.

k3d

curl -s https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash

cluster setup

k3d cluster create --config cluster.yaml
k3d cluster start tooling

using a dedicated namespace

we'll create and use separate namespaces for individual components of tooling cluster.

kubectl create namespace analytics
kubectl create namespace monitoring
kubectl create namespace visualizer

helm 3 (ubuntu/debian)

curl https://baltocdn.com/helm/signing.asc | gpg --dearmor | sudo tee /usr/share/keyrings/helm.gpg > /dev/null
sudo apt-get install apt-transport-https --yes
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/helm.gpg] https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list
sudo apt-get update
sudo apt-get install helm

charts

helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo add grafana https://grafana.github.io/helm-charts
helm repo update

installation

note: make sure that tooling cluster is up and running.

spark

helm install spark bitnami/spark -f helm/spark.yaml

prometheus

kubectl apply -f k8s/prometheus.yaml
helm install --namespace monitoring prometheus prometheus-community/prometheus -f helm/prometheus.yaml

grafana

kubectl apply -f k8s/grafana.yaml
helm install grafana grafana/grafana -f helm/grafana.yaml