create-a-kubeletconfig-crd-to-edit-kubelet-parameters.adoc

Creating a KubeletConfig CRD to edit kubelet parameters

The kubelet configuration is currently serialized as an ignition configuration, so it can be directly edited. However, there is also a new kubelet-config-controller added to the Machine Config Controller (MCC). This allows you to create a KubeletConfig custom resource (CR) to edit the kubelet parameters.

Procedure

1. Run:

   $ oc get machineconfig

   This provides a list of the available machine configuration objects you can select. By default, the two kubelet-related configs are 01-master-kubelet and 01-worker-kubelet.

2. To check the current value of max Pods per node, run:

   # oc describe node <node-ip> | grep Allocatable -A6

   Look for value: pods: <value>.

   For example:

   # oc describe node ip-172-31-128-158.us-east-2.compute.internal | grep Allocatable -A6
   Allocatable:
    attachable-volumes-aws-ebs:  25
    cpu:                         3500m
    hugepages-1Gi:               0
    hugepages-2Mi:               0
    memory:                      15341844Ki
    pods:                        250

3. To set the max Pods per node on the worker nodes, create a custom resource file that contains the kubelet configuration. For example, change-maxPods-cr.yaml:

   apiVersion: machineconfiguration.openshift.io/v1
   kind: KubeletConfig
   metadata:
     name: set-max-pods
   spec:
     machineConfigPoolSelector:
       matchLabels:
         custom-kubelet: large-pods
     kubeletConfig:
       maxPods: 500

   The rate at which the kubelet talks to the API server depends on queries per second (QPS) and burst values. The default values, 50 for kubeAPIQPS and 100 for kubeAPIBurst, are good enough if there are limited pods running on each node. Updating the kubelet QPS and burst rates is recommended if there are enough CPU and memory resources on the node:

   apiVersion: machineconfiguration.openshift.io/v1
   kind: KubeletConfig
   metadata:
     name: set-max-pods
   spec:
     machineConfigPoolSelector:
       matchLabels:
         custom-kubelet: large-pods
     kubeletConfig:
       maxPods: <pod_count>
       kubeAPIBurst: <burst_rate>
       kubeAPIQPS: <QPS>

   1. Run:

      $ oc label machineconfigpool worker custom-kubelet=large-pods

   2. Run:

      $ oc create -f change-maxPods-cr.yaml

   3. Run:

      $ oc get kubeletconfig

      This should return set-max-pods.

      Depending on the number of worker nodes in the cluster, wait for the worker nodes to be rebooted one by one. For a cluster with 3 worker nodes, this could take about 10 to 15 minutes.

4. Check for maxPods changing for the worker nodes:

   $ oc describe node

   1. Verify the change by running:

      $ oc get kubeletconfigs set-max-pods -o yaml

      This should show a status of True and type:Success