|
1 |
| -## My Project |
| 1 | +# EKS Hybrid Nodes on Raspberry Pi |
2 | 2 |
|
3 |
| -TODO: Fill this README out! |
| 3 | +## Table of Contents |
| 4 | +- [Introduction](#introduction) |
| 5 | +- [Architecture](#architecture) |
| 6 | +- [Prerequisites](#prerequisites) |
| 7 | + - [Hardware Requirements](#hardware-requirements) |
| 8 | + - [Software Requirements](#software-requirements) |
| 9 | +- [Getting Started](#getting-started) |
| 10 | +- [Setup Process](#setup-process) |
| 11 | + - [1. AWS Infrastructure Setup](#1-aws-infrastructure-setup) |
| 12 | + - [2. Raspberry Pi Setup](#2-raspberry-pi-setup) |
| 13 | + - [3. Setup CNI](#3-setup-cni) |
| 14 | +- [Verification](#verification) |
| 15 | +- [Clean Up](#clean-up) |
| 16 | +- [Disclaimer](#disclaimer) |
| 17 | +- [Contributing](#contributing) |
| 18 | +- [License](#license) |
4 | 19 |
|
5 |
| -Be sure to: |
| 20 | +--- |
6 | 21 |
|
7 |
| -* Change the title in this README |
8 |
| -* Edit your repository description on GitHub |
| 22 | +## Introduction |
9 | 23 |
|
10 |
| -## Security |
| 24 | +Amazon EKS Hybrid Nodes is a powerful feature that enables you to extend your Amazon EKS clusters to include on-premises and edge infrastructure as worker nodes. This means you can: |
11 | 25 |
|
12 |
| -See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information. |
| 26 | +✨ Run Kubernetes workloads on your own hardware while AWS manages the control plane |
| 27 | +✨ Unify Kubernetes management across cloud and on-premises environments |
| 28 | +✨ Leverage AWS services and EKS features with your on-premises nodes |
| 29 | +✨ Pay only for the vCPU resources of your hybrid nodes when they're attached to EKS clusters |
13 | 30 |
|
14 |
| -## License |
| 31 | +This repository demonstrates how to implement EKS Hybrid Nodes using one of the most accessible and cost-effective platforms available - the Raspberry Pi. By following this guide, you'll learn how to: |
| 32 | + |
| 33 | +1. Transform a Raspberry Pi into an EKS hybrid node |
| 34 | +2. Connect it securely to your EKS cluster |
| 35 | +3. Deploy and manage workloads across your hybrid infrastructure |
| 36 | + |
| 37 | +Our goal is to showcase that setting up hybrid nodes doesn't need to be complex or expensive. This implementation serves as an excellent starting point for learning about EKS Hybrid Nodes or prototyping hybrid scenarios before deploying to production environments. |
| 38 | + |
| 39 | +--- |
| 40 | + |
| 41 | +## Architecture |
| 42 | + |
| 43 | + |
| 44 | + |
| 45 | +> **Important Note on Network Architecture:** |
| 46 | +> - The EKS cluster is configured with public endpoint access only |
| 47 | +> - Raspberry Pi → EKS control plane communication flows through the internet via the public endpoint |
| 48 | +> - EKS control plane → Raspberry Pi communication is established through the VPN tunnel |
| 49 | +
|
| 50 | +--- |
| 51 | + |
| 52 | +## Prerequisites |
| 53 | + |
| 54 | +### Hardware Requirements |
| 55 | +- **Raspberry Pi 4** (2GB+ RAM) |
| 56 | +- Network connectivity (WiFi/Ethernet) |
| 57 | +- SSH access configured |
| 58 | +- Ubuntu Server 22.04 LTS (ARM64) |
| 59 | + |
| 60 | +### Software Requirements |
| 61 | +> Ensure you have the following tools installed: |
| 62 | +- [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html) |
| 63 | +- [Session Manager plugin for AWS CLI](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html) |
| 64 | +- [kubectl](https://kubernetes.io/docs/tasks/tools/) |
| 65 | +- [terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) |
| 66 | + |
| 67 | +--- |
| 68 | + |
| 69 | +## Getting Started |
| 70 | + |
| 71 | +Clone the repository: |
| 72 | + |
| 73 | +```bash |
| 74 | +git clone https://github.com/aws-samples/sample-eks-hybrid-nodes-raspberry-pi.git |
| 75 | +cd sample-eks-hybrid-nodes-raspberry-pi |
| 76 | +``` |
| 77 | + |
| 78 | +--- |
| 79 | + |
| 80 | +## Setup Process |
| 81 | + |
| 82 | +### 1. AWS Infrastructure Setup |
| 83 | + |
| 84 | +> **Note:** This demo uses the `ap-southeast-1` (Singapore) region. If you need to use a different region, modify the region variable in your terraform configuration. |
| 85 | +
|
| 86 | +```bash |
| 87 | +# Deploy AWS infrastructure using Terraform |
| 88 | +cd terraform |
| 89 | + |
| 90 | +terraform init |
| 91 | +terraform apply --auto-approve |
| 92 | + |
| 93 | +$(terraform output -raw eks_update_kubeconfig) |
| 94 | +``` |
| 95 | + |
| 96 | +**Terraform Output Files:** |
| 97 | +- `SETUP_VPN.md`: Steps to setup Wireguard on vpn_server |
| 98 | +- `SETUP_NODE.md`: Steps to setup nodeadm on the on-prem device |
| 99 | +- `cilium-values.yaml`: Config file used to setup Cilium |
| 100 | + |
| 101 | +**Important Terraform Outputs:** |
| 102 | +- `eks_update_kubeconfig`: How to access cluster |
| 103 | +- `connect_vpn_server`: How to connect to vpn_server using SSM |
| 104 | + |
| 105 | +#### VPN Server Setup |
| 106 | +We use Wireguard for site-to-site VPN in this demo. This is how our EKS Cluster communicates with the Raspberry Pi. |
| 107 | +We setup Wireguard by installing the Wireguard server on an EC2 instance running in our AWS Account. Then we will setup the Wireguard client on our on-prem device. |
| 108 | + |
| 109 | +**Step 1:** Review the Wireguard setup instructions in the SETUP_VPN.md file that was generated in your terraform directory. |
| 110 | +```bash |
| 111 | +cat SETUP_VPN.md |
| 112 | +``` |
| 113 | + |
| 114 | +**Step 2:** Get the VPN server's public IP address using Terraform |
| 115 | +```bash |
| 116 | +terraform output vpn_server_public_ip |
| 117 | +``` |
| 118 | +> **Note:** Save this IP address as you will need it for the Raspberry Pi Wireguard configuration. |
| 119 | +
|
| 120 | +**Step 3:** Connect to VPN server using SSM |
| 121 | +```bash |
| 122 | +$(terraform output -raw connect_vpn_server) |
| 123 | +``` |
| 124 | + |
| 125 | +**Step 4:** Follow the instructions from SETUP_VPN.md to set up Wireguard on the VPN server. |
| 126 | + |
| 127 | +**Step 5:** Get the required key values for Raspberry Pi setup |
| 128 | +```bash |
| 129 | +# Switch to root user |
| 130 | +sudo -i |
| 131 | + |
| 132 | +# Get the public key |
| 133 | +cat /etc/wireguard/public.key |
| 134 | + |
| 135 | +# Get the client private key |
| 136 | +cat /etc/wireguard/client-private.key |
| 137 | +``` |
| 138 | +> **Note:** Save these key values as you will need them later for the Raspberry Pi Wireguard configuration. |
| 139 | +
|
| 140 | +--- |
| 141 | + |
| 142 | +### 2. Raspberry Pi Setup |
| 143 | + |
| 144 | +1. **Install Wireguard:** |
| 145 | +```bash |
| 146 | +sudo apt update && sudo apt install -y wireguard |
| 147 | +sudo mkdir -p /etc/wireguard |
| 148 | +``` |
| 149 | + |
| 150 | +2. **Create WireGuard Configuration:** |
| 151 | +```bash |
| 152 | +sudo nano /etc/wireguard/wg0.conf |
| 153 | +``` |
| 154 | + |
| 155 | +Add the following configuration (replace placeholders): |
| 156 | +```ini |
| 157 | +[Interface] |
| 158 | +PrivateKey = <client-private.key> |
| 159 | +Address = 192.168.3.0/24 |
15 | 160 |
|
16 |
| -This library is licensed under the MIT-0 License. See the LICENSE file. |
| 161 | +[Peer] |
| 162 | +# Public key from AWS server (/etc/wireguard/public.key) |
| 163 | +PublicKey = <public.key> |
| 164 | +# Your EC2 instance's public IP |
| 165 | +Endpoint = <ec2-public-ip>:51820 |
| 166 | +# AWS VPC CIDR and WireGuard server network |
| 167 | +AllowedIPs = 172.16.0.1/24,10.0.0.0/24 |
| 168 | +PersistentKeepalive = 25 |
| 169 | +``` |
| 170 | + |
| 171 | +> **Important:** Replace the following: |
| 172 | +> - `<client-private.key>` with the client-private.key from VPN Server |
| 173 | +> - `<public.key>` with the public.key from VPN Server |
| 174 | +> - `<ec2-public-ip>` with VPN Server's Public IP |
| 175 | +
|
| 176 | +3. **Enable and Start Wireguard:** |
| 177 | +```bash |
| 178 | +sudo echo "net.ipv4.ip_forward=1" | sudo tee -a /etc/sysctl.conf |
| 179 | +sudo sysctl -p |
| 180 | + |
| 181 | +sudo systemctl enable wg-quick@wg0 |
| 182 | +sudo systemctl start wg-quick@wg0 |
| 183 | + |
| 184 | +# Verify connection |
| 185 | +sudo wg show |
| 186 | +``` |
| 187 | + |
| 188 | +4. **Complete Node Setup:** |
| 189 | +```bash |
| 190 | +# View setup instructions from the folder containing your terraform files |
| 191 | +cat SETUP_NODE.md |
| 192 | +``` |
| 193 | + |
| 194 | +> **Troubleshooting:** If you encounter issues, refer to the [official troubleshooting guide](https://docs.aws.amazon.com/eks/latest/userguide/hybrid-nodes-troubleshooting.html). |
| 195 | +> |
| 196 | +> ⚠️ **Common Issue:** If you get `ExpiredTokenException: The security token included in the request is expired`, the SSM Hybrid Activation Credentials have expired. Rerun `terraform init` & `terraform apply` and restart the `SETUP_NODE.md` steps. |
| 197 | +
|
| 198 | +--- |
| 199 | + |
| 200 | +### 3. Setup CNI |
| 201 | + |
| 202 | +1. **Install Helm** (if not already installed): |
| 203 | +```bash |
| 204 | +curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 |
| 205 | +chmod 700 get_helm.sh |
| 206 | +./get_helm.sh |
| 207 | +``` |
| 208 | + |
| 209 | +2. **Add Cilium Repository:** |
| 210 | +```bash |
| 211 | +# New installation |
| 212 | +helm repo add cilium https://helm.cilium.io/ |
| 213 | + |
| 214 | +# Or update existing |
| 215 | +helm repo update |
| 216 | +``` |
| 217 | + |
| 218 | +3. **Install Cilium:** |
| 219 | +```bash |
| 220 | +helm install cilium cilium/cilium \ |
| 221 | + --version 1.16.6 \ |
| 222 | + --namespace kube-system \ |
| 223 | + --values cilium-values.yaml |
| 224 | +``` |
| 225 | + |
| 226 | +4. **Configure CoreDNS:** |
| 227 | +If CoreDNS pods are not starting up properly, you can patch them to use the cluster endpoint: |
| 228 | +```bash |
| 229 | +# First get the endpoint |
| 230 | +ENDPOINT=$(terraform output -raw eks_cluster_endpoint) |
| 231 | + |
| 232 | +# Then use it in the patch command |
| 233 | +kubectl -n kube-system patch deployment coredns --patch '{"spec":{"template":{"spec":{"containers":[{"name":"coredns","env":[{"name":"KUBERNETES_SERVICE_HOST","value":"'$ENDPOINT'"},{"name":"KUBERNETES_SERVICE_PORT","value":"443"}]}]}}}}' |
| 234 | +``` |
| 235 | + |
| 236 | +This configures CoreDNS to use the same endpoint as Cilium for API server communication. |
| 237 | + |
| 238 | +--- |
| 239 | + |
| 240 | +## Verification |
| 241 | + |
| 242 | +```bash |
| 243 | +# Check node status |
| 244 | +kubectl get nodes |
| 245 | + |
| 246 | +# View running pods |
| 247 | +kubectl get pods -A |
| 248 | +``` |
| 249 | + |
| 250 | +✅ Check your AWS Console - a new node should appear in your EKS Cluster once Cilium is properly configured. |
| 251 | + |
| 252 | +🎉 **Congratulations!** Your Raspberry Pi is now a Hybrid Node in your EKS Cluster. |
| 253 | + |
| 254 | +--- |
| 255 | + |
| 256 | +## Clean Up |
| 257 | + |
| 258 | +### 1. Remove Node from Cluster |
| 259 | +```bash |
| 260 | +# Get node name |
| 261 | +kubectl get nodes |
| 262 | + |
| 263 | +# Drain and delete node |
| 264 | +kubectl drain <node-name> --ignore-daemonsets --delete-emptydir-data |
| 265 | +kubectl delete node <node-name> |
| 266 | + |
| 267 | +# Uninstall Cilium |
| 268 | +helm uninstall cilium -n kube-system |
| 269 | +``` |
| 270 | + |
| 271 | +### 2. Clean up Raspberry Pi |
| 272 | +Use the `cleanup-pi.sh` script in the cleanup folder to remove Hybrid Node and Wireguard configuration. |
| 273 | + |
| 274 | +### 3. Destroy Cluster |
| 275 | +```bash |
| 276 | +terraform init |
| 277 | +terraform destroy --auto-approve |
| 278 | +``` |
| 279 | + |
| 280 | +--- |
| 281 | + |
| 282 | +## Disclaimer |
| 283 | + |
| 284 | +⚠️ **This repository is intended for demonstration and learning purposes only.** |
| 285 | +It is **not** intended for production use. The code provided here is for educational purposes and should not be used in a live environment without proper testing, validation, and modifications. |
| 286 | + |
| 287 | +Use at your own risk. The authors are not responsible for any issues, damages, or losses that may result from using this code in production. |
| 288 | + |
| 289 | +## Contributing |
| 290 | + |
| 291 | +Contributions welcome! Please read our [Contributing Guidelines](CONTRIBUTING.md) and [Code of Conduct](CODE_OF_CONDUCT.md). |
| 292 | + |
| 293 | +## License |
17 | 294 |
|
| 295 | +This project is licensed under the MIT License - see [LICENSE](LICENSE) file. |
0 commit comments