You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: latest/ug/nodes/hybrid-nodes-add-ons.adoc
+19-17
Original file line number
Diff line number
Diff line change
@@ -11,9 +11,9 @@ include::../attributes.txt[]
11
11
Configure common add-ons for hybrid nodes
12
12
--
13
13
14
-
This page describes considerations for running {aws} add-ons and Community add-ons on Amazon EKS Hybrid Nodes. To learn more about Amazon EKS add-ons and the processes for creating, upgrading, and removing add-ons from your cluster, see <<eks-add-ons>>. Unless otherwise noted on this page, the processes for creating, upgrading, and removing Amazon EKS add-ons is the same for Amazon EKS clusters with hybrid nodes as it is for Amazon EKS clusters with nodes running in {aws} Cloud. Only the add-ons included on this page have been validated for compatibility with Amazon EKS Hybrid Nodes.
14
+
This page describes considerations for running {aws} add-ons and community add-ons on Amazon EKS Hybrid Nodes. To learn more about Amazon EKS add-ons and the processes for creating, upgrading, and removing add-ons from your cluster, see <<eks-add-ons>>. Unless otherwise noted on this page, the processes for creating, upgrading, and removing Amazon EKS add-ons is the same for Amazon EKS clusters with hybrid nodes as it is for Amazon EKS clusters with nodes running in {aws} Cloud. Only the add-ons included on this page have been validated for compatibility with Amazon EKS Hybrid Nodes.
15
15
16
-
The following {aws}Add-ons are compatible with Amazon EKS Hybrid Nodes.
16
+
The following {aws}add-ons are compatible with Amazon EKS Hybrid Nodes.
17
17
18
18
[cols="1,1", options="header"]
19
19
|===
@@ -29,7 +29,7 @@ The following {aws} Add-ons are compatible with Amazon EKS Hybrid Nodes.
29
29
|{aws} Distro for OpenTelemetry (ADOT)
30
30
|v0.102.1-eksbuild.2 and above
31
31
32
-
|CloudWatch Observability Agent
32
+
|CloudWatch Observability agent
33
33
|v2.2.1-eksbuild.1 and above
34
34
35
35
|EKS Pod Identity Agent
@@ -39,7 +39,7 @@ The following {aws} Add-ons are compatible with Amazon EKS Hybrid Nodes.
39
39
|v8.1.0-eksbuild.1 and above
40
40
|===
41
41
42
-
The following Community add-ons are compatible with Amazon EKS Hybrid Nodes. To learn more about support for Community add-ons, see <<eks-add-ons>>.
42
+
The following community add-ons are compatible with Amazon EKS Hybrid Nodes. To learn more about community add-ons, see <<community-addons>>.
43
43
44
44
[cols="1,1", options="header"]
45
45
|===
@@ -52,21 +52,21 @@ The following Community add-ons are compatible with Amazon EKS Hybrid Nodes. To
52
52
53
53
In addition to the Amazon EKS add-ons in the tables above, the <<prometheus,Amazon Managed Service for Prometheus Collector>>, and the <<aws-load-balancer-controller,{aws} Load Balancer Controller>> for <<alb-ingress,application ingress>> (HTTP) and <<network-load-balancing,load balancing>> (TCP/UDP) are compatible with hybrid nodes.
54
54
55
-
There are {aws} add-ons and Community add-ons that aren't compatible with Amazon EKS Hybrid Nodes. The latest versions of these add-ons have an anti-affinity rule for the default `eks.amazonaws.com/compute-type: hybrid` label applied to hybrid nodes. This prevents them from running on hybrid nodes when deployed in your clusters. If you have clusters with both hybrid nodes and nodes running in {aws} Cloud, you can deploy these add-ons in your cluster to nodes running in {aws} Cloud. The Amazon VPC CNI is not compatible with hybrid nodes, and Cilium and Calico are supported as the Container Networking Interfaces (CNIs) for Amazon EKS Hybrid Nodes. See <<hybrid-nodes-cni>> for more information.
56
-
57
-
The rest of this page describes differences between running compatible Amazon EKS add-ons on hybrid nodes, compared to the other Amazon EKS compute types.
55
+
There are {aws} add-ons and community add-ons that aren't compatible with Amazon EKS Hybrid Nodes. The latest versions of these add-ons have an anti-affinity rule for the default `eks.amazonaws.com/compute-type: hybrid` label applied to hybrid nodes. This prevents them from running on hybrid nodes when deployed in your clusters. If you have clusters with both hybrid nodes and nodes running in {aws} Cloud, you can deploy these add-ons in your cluster to nodes running in {aws} Cloud. The Amazon VPC CNI is not compatible with hybrid nodes, and Cilium and Calico are supported as the Container Networking Interfaces (CNIs) for Amazon EKS Hybrid Nodes. See <<hybrid-nodes-cni>> for more information.
58
56
59
57
[#hybrid-nodes-add-ons-aws-add-ons]
60
-
== {aws} Add-ons
58
+
== {aws} add-ons
59
+
60
+
The sections that follow describe differences between running compatible {aws} add-ons on hybrid nodes compared to other Amazon EKS compute types.
61
61
62
62
[#hybrid-nodes-add-ons-core]
63
-
=== kube-proxy and CoreDNS
63
+
== kube-proxy and CoreDNS
64
64
65
65
EKS installs Kube-proxy and CoreDNS as self-managed add-ons by default when you create an EKS cluster with the {aws} API and {aws} SDKs, including from the {aws} CLI. You can overwrite these add-ons as Amazon EKS add-ons after cluster creation. Reference the EKS documentation for details on <<managing-kube-proxy>> and <<managing-coredns>>. If you are running a cluster with hybrid nodes and nodes in {aws} Cloud, we recommend that you have at least one CoreDNS replica on hybrid nodes and at least one CoreDNS replica on your nodes in {aws} Cloud.
66
66
67
67
[#hybrid-nodes-add-ons-cw]
68
-
=== CloudWatch Observability Agent add-on
69
-
As the CloudWatch Observability Agent runs https://kubernetes.io/docs/reference/access-authn-authz/webhook/[webhooks], you must configure a remote pod network when creating your Amazon EKS cluster, and you must make your pod IP addresses routable. Implementing Border Gateway Protocol (BGP) with the CNI is one common way to make your pod IP addresses routable.
68
+
== CloudWatch Observability agent
69
+
As the CloudWatch Observability agent runs https://kubernetes.io/docs/reference/access-authn-authz/webhook/[webhooks], you must configure a remote pod network when creating your Amazon EKS cluster, and you must make your pod IP addresses routable. Implementing Border Gateway Protocol (BGP) with the CNI is one common way to make your pod IP addresses routable.
70
70
71
71
Node-level metrics are not available for hybrid nodes because link:AmazonCloudWatch/latest/monitoring/ContainerInsights.html[CloudWatch Container Insights,type="documentation"] depends on the availability of link:AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html[Instance Metadata Service,type="documentation"] (IMDS) for node-level metrics. Cluster, workload, pod, and container-level metrics are available for hybrid nodes.
72
72
@@ -99,7 +99,7 @@ items:
99
99
----
100
100
101
101
[#hybrid-nodes-add-ons-amp]
102
-
=== Amazon Managed Prometheus managed collector for hybrid nodes
102
+
== Amazon Managed Prometheus managed collector for hybrid nodes
103
103
104
104
An Amazon Managed Service for Prometheus (AMP) managed collector consists of a scraper that discovers and collects metrics from the resources in an Amazon EKS cluster. AMP manages the scraper for you, removing the need to manage any instances, agents, or scrapers yourself.
105
105
@@ -108,14 +108,14 @@ You can use AMP managed collectors without any additional configuration specific
108
108
Follow the steps in link:prometheus/latest/userguide/AMP-collector-how-to.html[Using an {aws} managed collector,type="documentation"] in the Amazon Managed Service for Prometheus User Guide.
109
109
110
110
[#hybrid-nodes-add-ons-adot]
111
-
==={aws} Distro for OpenTelemetry (ADOT) add-on
111
+
== {aws} Distro for OpenTelemetry (ADOT)
112
112
113
113
You can use the {aws} Distro for OpenTelemetry (ADOT) Amazon EKS add-on to collect metrics, logs, and tracing data from your applications running on hybrid nodes. ADOT uses admission https://kubernetes.io/docs/reference/access-authn-authz/webhook/[webhooks] to mutate and validate the Collector Custom Resource requests. For the EKS control plane to reach webhooks running on hybrid nodes, you must configure your remote pod network when creating your Amazon EKS cluster, and you must make your pod IP addresses routable. Implementing Border Gateway Protocol (BGP) with the CNI is one common way to make your pod IP addresses routable.
114
114
115
115
Follow the steps in https://aws-otel.github.io/docs/getting-started/adot-eks-add-on[Getting Started with {aws} Distro for OpenTelemetry using EKS Add-Ons] in the _{aws} Distro for OpenTelemetry_ documentation.
116
116
117
117
[#hybrid-nodes-add-ons-lbc]
118
-
==={aws} Load Balancer Controller
118
+
== {aws} Load Balancer Controller
119
119
120
120
You can use the <<aws-load-balancer-controller,{aws} Load Balancer Controller>> and Application Load Balancer (ALB) or Network Load Balancer (NLB) with the target type ip for workloads on hybrid nodes connected with {aws} Direct Connect or {aws} Site-to-Site VPN. As the {aws} Load Balancer Controller uses https://kubernetes.io/docs/reference/access-authn-authz/webhook/[webhooks], you must configure a remote pod network when creating your Amazon EKS cluster, and you must make your pod IP addresses routable. Implementing Border Gateway Protocol (BGP) with the CNI is one common way to make your pod IP addresses routable.
The original Amazon EKS Pod Identity Agent DaemonSet relies on the availability of EC2 IMDS on the node to obtain the required {aws} credentials. As IMDS isn't available on hybrid nodes, starting in add-on version `1.3.3-eksbuild.1`, the Pod Identity Agent add-on optionally deploys a second DaemonSet that specifically targets hybrid nodes. This DaemonSet mounts the required credentials to the pods created by the Pod Identity Agent add-on.
141
141
@@ -179,13 +179,15 @@ aws eks create-addon \
179
179
----
180
180
181
181
[#hybrid-nodes-add-ons-csi-snapshotter]
182
-
=== CSI snapshot controller add-on
182
+
== CSI snapshot controller
183
183
184
184
Starting with version `v8.1.0-eksbuild.2`, the <<csi-snapshot-controller,CSI snapshot controller add-on>> applies a soft anti-affinity rule for hybrid nodes, preferring the controller `deployment` to run on EC2 in the same {aws} Region as the Amazon EKS control plane. Co-locating the `deployment` in the same {aws} Region as the Amazon EKS control plane improves latency.
185
185
186
186
[#hybrid-nodes-add-ons-community]
187
187
== Community add-ons
188
188
189
+
The sections that follow describe differences between running compatible community add-ons on hybrid nodes compared to other Amazon EKS compute types.
190
+
189
191
[#hybrid-nodes-add-ons-metrics-server]
190
-
=== Kubernetes Metrics Server add-on
192
+
== Kubernetes Metrics Server
191
193
The control plane needs to reach Metrics Server's pod IP (or node IP if hostNetwork is enabled). Therefore, unless you run Metrics Server in hostNetwork mode, you must configure a remote pod network when creating your Amazon EKS cluster, and you must make your pod IP addresses routable. Implementing Border Gateway Protocol (BGP) with the CNI is one common way to make your pod IP addresses routable.
0 commit comments