Amazon Elastic Kubernetes Service (EKS) Cluster Storage Configuration Guide

This guide will walk you through setting up storage classes in Amazon EKS cluster by using Amazon Elastic Block Store and Amazon Elastic File System.

Introduction

SAP Edge Integration Cell runs on compute clusters managed by the Kubernetes. In-order to successfully deploy the Edge Integration Cell runtime into Kubernetes, Edge Integration Cell requires Kubernetes cluster has two kinds of persistent storage with ability to dynamic provisioning:

[Required] Persistence volume with ReadWriteOnce (RWO) access mode.
[Optional] Persistence volume with ReadWriteMany (RWX) access mode. This is optional for Quick Setup.
ReadWriteOnce (RWO)
- Only one instance can mount the EBS volume at a time, allowing that instance to read and write data.
ReadWriteMany (RWX)
- Multiple instances can mount the file system simultaneously, allowing them to read and write to the file system concurrently.

To fulfill these storage requirements, we leverage Amazon Elastic Block Store (EBS) and Amazon Elastic File System (EFS) in our Amazon EKS (Elastic Kubernetes Service) cluster.

[Required] Amazon Elastic Block Store (EBS) volume are designed to provide high-performance block storage that can be attached to a single EC2 instance at a time in read-write mode. This nature makes Amazon EBS to be a good choice for the K8S cluster with storage access mode as ReadWriteOnce (RWO).

[Optional] Amazon Elastic File System (EFS) is a scalable, fully-managed file system that can be mounted by multiple EC2 instances or Kubernetes pods concurrently. This makes it ideal for shared storage scenarios, where multiple pods need read/write access to the same data. This nature makes Amazon EFS to be a good choice for the K8S cluster with storage access mode as ReadWriteMany (RWX).

Prerequisites
Step 1. Create IAM Role for Amazon EBS CSI driver
Step 2. Install the Amazon EBS CSI driver
Step 3. Create Amazon EBS type storage class
Step 4. Create an IAM role for Amazon EFS CSI driver
Step 5. Install the Amazon EFS CSI driver
Step 6. Create Amazon EFS file system for Amazon EKS cluster
Step 7. Create Amazon EFS type storage class

Prerequisites

Before you begin, ensure you have the following:

Finish instruction Amazon Elastic Kubernetes Service (EKS) Configuration Guide
A running Kubernetes cluster on Amazon EKS
kubectl installed and configured.
AWS CLI installed and configured.
IAM permissions for creating EBS and EFS resources.

Step 1. Create an IAM role for Amazon EBS CSI driver

View your EKS cluster's OIDC provider URL by using below AWS ClI Command. Replace my-cluster with the name of the EKS cluster created in the previous step.

aws eks describe-cluster \
    --name my-cluster \
    --query "cluster.identity.oidc.issuer" \
    --output text

An example output is as follows.

https://oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE

Copy the following contents to a file named aws-ebs-csi-driver-trust-policy.json. Replace 111122223333 with your AWS account ID. Replace EXAMPLED539D4633E53DE1B71EXAMPLE and region-code with the values returned in the previous step. If your cluster is in the AWS GovCloud (US-East) or AWS GovCloud (US-West) AWS Regions, then replace arn:aws: with arn:aws-us-gov:.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::111122223333:oidc-provider/oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:aud": "sts.amazonaws.com",
          "oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:sub": "system:serviceaccount:kube-system:ebs-csi-controller-sa"
        }
      }
    }
  ]
}

Create the IAM role for the EBS CSI Driver. You can change AmazonEKS_EBS_CSI_DriverRole to a different name. If you change it, make sure to change it in later steps.
```
aws iam create-role \
    --role-name AmazonEKS_EBS_CSI_DriverRole \
    --assume-role-policy-document file://"aws-ebs-csi-driver-trust-policy.json"
```
Attach the required AWS managed policy to the role with the following command. If your cluster is in the AWS GovCloud (US-East) or AWS GovCloud (US-West) AWS Regions, then replace arn:aws: with arn:aws-us-gov:
```
aws iam attach-role-policy \
    --policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \
    --role-name AmazonEKS_EBS_CSI_DriverRole
```

Step 2. Install the Amazon EBS CSI driver via Amazon EKS add-on

Go back to the AWS EkS console and select the EKS cluster we created previously. On your EKS cluster overview page, choose Add-ons, then click Get more add-ons.
On the Select add-ons page, do the following:
- In the Amazon EKS-addons section, select the Amazon EBS CSI Driver check box.
- Choose Next
On the Configure selected add-ons settings page, do the following:
- Select the Version you'd like to use.
- For Select IAM role, select the name of an IAM role that we created in the Step 1
- Choose Next
On the Review and add page, choose Create. After the add-on installation is complete, you see your installed add-on.

Step 3. Create Amazon EBS type storage class

Copy the following contents to a file named ebs-storageclass.yaml. It will create a storage class called ebs-sc which will be used for the Persistent Volume Claim with ReadWriteOnce access mode.

---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ebs-sc
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: ebs.csi.aws.com
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true

Deploy the ebs-sc storage class by using the following kubectl command
```
kubectl apply -f ebs-storageclass.yaml
```

Execute the follow command to unset the storage class gp2 as default.

kubectl patch storageclass gp2 -p '{"metadata": {"annotations": {"storageclass.kubernetes.io/is-default-class": "false"}}}'

Now we've configure the Amazon EBS type storage class successfully, and allow EKS cluster able to dynamically provision persistent volumes for the access mode ReadWriteOnce by using the persistence volume claim we created.

**Up Next

If you are following the Quick Setup guide, you may proceed to the next step to create a key pair. EFS is optional for quick setup.
🔗 Quick setup: Configure key pair

Step 4. Create an IAM role for Amazon EFS CSI driver

View your EKS cluster's OIDC provider URL by using below AWS ClI Command. Replace my-cluster with the name of the EKS cluster created in the previous step.

aws eks describe-cluster \
    --name my-cluster \
    --query "cluster.identity.oidc.issuer" \
    --output text

An example output is as follows.

https://oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE

Copy the following contents to a file named aws-efs-csi-driver-trust-policy.json. Replace 111122223333 with your AWS account ID. Replace EXAMPLED539D4633E53DE1B71EXAMPLE and region-code with the values returned in the previous step. If your cluster is in the AWS GovCloud (US-East) or AWS GovCloud (US-West) AWS Regions, then replace arn:aws: with arn:aws-us-gov:.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::111122223333:oidc-provider/oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringLike": {
          "oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:sub": "system:serviceaccount:kube-system:efs-csi-*",
          "oidc.eks.region-code.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:aud": "sts.amazonaws.com"
        }
      }
    }
  ]
}

Create the IAM role for the EFS CSI Driver. You can change AmazonEKS_EFS_CSI_DriverRole to a different name. If you change it, make sure to change it in later steps.
```
aws iam create-role \
    --role-name AmazonEKS_EFS_CSI_DriverRole \
    --assume-role-policy-document file://"aws-efs-csi-driver-trust-policy.json"
```
Attach the required AWS managed policy to the role with the following command. If your cluster is in the AWS GovCloud (US-East) or AWS GovCloud (US-West) AWS Regions, then replace arn:aws: with arn:aws-us-gov:.
```
aws iam attach-role-policy \
    --policy-arn arn:aws:iam::aws:policy/service-role/AmazonEFSCSIDriverPolicy \
    --role-name AmazonEKS_EFS_CSI_DriverRole
```
Now we've created IAM role for Amazon EFS CSI driver successfully.

Step 5. Install the Amazon EFS CSI driver via Amazon EKS add-on

Go back to the AWS EKS console and select the EKS cluster we created previously. On your EKS cluster overview page, choose Add-ons, then click Get more add-ons.
On the Select add-ons page, do the following:
- In the Amazon EKS-addons section, select the Amazon EFS CSI Driver check box.
- Choose Next
On the Configure selected add-ons settings page, do the following:
- Select the Version you'd like to use.
- For Select IAM role, select the name of an IAM role that we created in the Step 4
- Choose Next
On the Review and add page, choose Create. After the add-on installation is complete, you see your installed add-on.

Step 6. Create Amazon EFS file system for Amazon EKS cluster

Please follow the AWS official step-by-step guide to create a Amazon EFS file system for your EKS cluster.

Step 7. Create Amazon EFS type storage class

Execute AWS ClI command below to retrieve the Amazon EFS file system ID that we created in the Step 6. Create Amazon EFS file system for Amazon EKS cluster

You could also find your Amazon EFS file system ID in the Amazon EFS console.
```
aws efs describe-file-systems --query "FileSystems[*].FileSystemId" --output text
```

Copy the following contents to a file named efs-storageclass.yaml. Replace fs-EXAMPLEEFSFILESYSID with your Amazon EFS file system ID obtained in the previous step. It will create a storage class called efs-sc and referring to the Amazon EFS file system for us.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
    name: efs-sc
provisioner: efs.csi.aws.com
volumeBindingMode: Immediate
parameters:
    provisioningMode: efs-ap # The type of volume to be provisioned by Amazon EFS. Currently, only access point based provisioning is supported (efs-ap)
    fileSystemId: fs-EXAMPLEEFSFILESYSID
    directoryPerms: "700" # The directory permissions of the root directory created by the access point.
    gidRangeStart: "1000" # The starting range of the Posix group ID to be applied onto the root directory of the access point. The default value is 50000
    gidRangeEnd: "2000" # The ending range of the Posix group ID. The default value is 7000000
    basePath: "/dynamic_provisioning"
    subPathPattern: "${.PVC.namespace}/${.PVC.name}"
    ensureUniqueDirectory: "true"
    reuseAccessPoint: "false"
allowVolumeExpansion: true

Deploy the efs-sc storage class by using the following kubectl command.
```
kubectl apply -f efs-storageclass.yaml
```
Now we've configure the Amazon EFS type storage class successfully. This storage class will enable Amazon EKS Cluster dynamically provision persistent volumes with the access mode ReadWriteMany, and allow volume expansion in general.

Conclusion

Your Amazon EKS cluster is now set up Amazon EBS and Amazon EFS type storage class. You could go ahead for the next step.

References

SAP
AWS
- Use Amazon EBS storage
- Use Amazon EFS storage

Up Next

🔗 Quick setup: Configure key pair
🔗 HA setup: Configure load balancer

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

step3-configure-storage-class.md

step3-configure-storage-class.md

Amazon Elastic Kubernetes Service (EKS) Cluster Storage Configuration Guide

Introduction

Table of Contents

Prerequisites

Step 1. Create an IAM role for Amazon EBS CSI driver

Step 2. Install the Amazon EBS CSI driver via Amazon EKS add-on

Step 3. Create Amazon EBS type storage class

**Up Next

Step 4. Create an IAM role for Amazon EFS CSI driver

Step 5. Install the Amazon EFS CSI driver via Amazon EKS add-on

Step 6. Create Amazon EFS file system for Amazon EKS cluster

Step 7. Create Amazon EFS type storage class

Conclusion

References

Up Next

Files

step3-configure-storage-class.md

Latest commit

History

step3-configure-storage-class.md

File metadata and controls

Amazon Elastic Kubernetes Service (EKS) Cluster Storage Configuration Guide

Introduction

Table of Contents

Prerequisites

Step 1. Create an IAM role for Amazon EBS CSI driver

Step 2. Install the Amazon EBS CSI driver via Amazon EKS add-on

Step 3. Create Amazon EBS type storage class

**Up Next

Step 4. Create an IAM role for Amazon EFS CSI driver

Step 5. Install the Amazon EFS CSI driver via Amazon EKS add-on

Step 6. Create Amazon EFS file system for Amazon EKS cluster

Step 7. Create Amazon EFS type storage class

Conclusion

References

Up Next