kubernetes-eks.md

Deploying Malcolm on Amazon Elastic Kubernetes Service (EKS)

• Deploying Malcolm on Amazon Elastic Kubernetes Service (EKS)
  • Prerequisites
  • Procedure
• Attribution

This document outlines the process of setting up a cluster on Amazon Elastic Kubernetes Service (EKS) using Amazon Web Services in preparation for Deploying Malcolm with Kubernetes.

This is a work-in-progress document that is still a bit rough around the edges. Users will need to replace things such as cluster-name and us-east-1 with the values that are appliable to the cluster. Any feedback is welcome in the relevant issue on GitHub.

This document assumes good working knowledge of Amazon Web Services (AWS) and Amazon Elastic Kubernetes Service (EKS). Good documentation resources can be found in the AWS documentation, the EKS documentation and the EKS Workshop.

Prerequisites

• aws cli - the AWS Command Line Interface with functioning access to the AWS infrastructure
• eksctl - the official CLI for Amazon EKS

Procedure

1. Create a Virtual Private Cloud (VPC)

   • subnets in at least 2 availability zones
   • tag private subnets with kubernetes.io/role/internal-elb: 1
   • tag public subnets with kubernetes.io/role/elb: 1
   • enable "auto-assign public IP address" for public subnets

2. Create a security group for the VPC

3. Create an Elastic Kubernetes Service (EKS) cluster

4. Generate a kubeconfig file to use with Malcolm's control scripts (malcolmeks.yaml is used in this example)

   aws eks update-kubeconfig --region us-east-1 --name cluster-name --kubeconfig malcolmeks.yaml

5. Create a node group

   • For x86-64 instances c4.4xlarge, t2.2xlarge, and t3a.2xlarge seem to be good instance types for Malcolm; or , for arm64 instances, m6gd.2xlarge, m6g.2xlarge, m7g.2xlarge, and t4g.2xlarge; but users' needs may vary (see recommended system requirements for Malcolm)
   • set the nodes to run on the VPC's public subnets

6. Deploy metrics-server

   kubectl --kubeconfig=malcolmeks.yaml apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

7. Associate IAM OIDC provider with cluster

   eksctl utils associate-iam-oidc-provider --region=us-east-1 --cluster=cluster-name --approve

8. Deploy the AWS Load Ballancer Controller add-on

   • See Ingress Controllers under Deploying Malcolm with Kubernetes
   • [kubernetes/99-ingress-aws-alb.yml.example]({{ site.github.repository_url }}/blob/{{ site.github.build_revision }}/kubernetes/99-ingress-aws-alb.yml.example) is an example ingress manifest for Malcolm using the ALB controller for HTTP(S) requests and the NLB controller for TCP connections to Logstash and Filebeat
   • Users must set type: LoadBalancer for the nginx-proxy service in [98-nginx-proxy.yml]({{ site.github.repository_url }}/blob/{{ site.github.build_revision }}/kubernetes/98-nginx-proxy.yml), the filebeat service in [12-filebeat.yml]({{ site.github.repository_url }}/blob/{{ site.github.build_revision }}/kubernetes/12-filebeat.yml) and the the logstash service in [13-logstash.yml]({{ site.github.repository_url }}/blob/{{ site.github.build_revision }}/kubernetes/13-logstash.yml)
   • How do I set up the AWS Load Balancer Controller on an Amazon EKS cluster...?
   • Installing the AWS Load Balancer Controller add-on
   • Application load balancing on Amazon EKS
   • Network load balancing on Amazon EKS

9. deploy Amazon EFS CSI driver

   • review Prerequisites
   • follow steps for Create an IAM policy and role
   • follow steps for Install the Amazon EFS driver
   • follow steps for Create an Amazon EFS file system

10. Set up access points, and note the Access point IDs to put in the YAML in the next step

    name	mountpoint	access point ID
    config	/malcolm/config	fsap-…
    opensearch	/malcolm/opensearch	fsap-…
    opensearch-backup	/malcolm/opensearch-backup	fsap-…
    pcap	/malcolm/pcap	fsap-…
    runtime-logs	/malcolm/runtime-logs	fsap-…
    suricata-logs	/malcolm/suricata-logs	fsap-…
    zeek-logs	/malcolm/zeek-logs	fsap-…

11. Create manifest for persistent volumes and volume claims from the EFS file system ID and access point IDs

    • See PersistentVolumeClaim Definitions under Deploying Malcolm with Kubernetes
    • [kubernetes/01-volumes-aws-efs.yml.example]({{ site.github.repository_url }}/blob/{{ site.github.build_revision }}/kubernetes/01-volumes-aws-efs.yml.example) is an example manifest to use as a starting point. Copy 01-volumes-aws-efs.yml.example to 01-volumes.yml and replace fs-FILESYSTEMID with the EFS file system and each fsap-… value with the corresponding access point ID from the previous step.

12. Finish the configuration then start Malcolm as described in Deploying Malcolm with Kubernetes

Attribution

Amazon Web Services, AWS, the Powered by AWS logo, and Amazon Elastic Kubernetes Service (EKS) are trademarks of Amazon.com, Inc. or its affiliates. The information about providers and services contained in this document is for instructional purposes and does not constitute endorsement or recommendation.