Setting up Kubernetes on AWS

I thought it was about time to do a definitive guide on how to set up EKS.

All the cool kids are on AWS, and all the cool kids are definitely using Kubernetes. So I thought it was about time to do a definitive guide on how to set it up. I will be setting up EKS - Elastic (because amazon) Kubernetes Service - which is amazon-speak for Kubernetes.

Prerequisites

Before we even get started, we need a few tools to get run the required commands. (I'm going to assume that you are on a Mac and have docker desktop and homebrew installed - because, who doesn't?)

Installing aws cli:

curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"
sudo installer -pkg AWSCLIV2.pkg -target /

For general use, the aws configure command is the fastest way to set up your AWS CLI installation. When you enter this command, the AWS CLI prompts you for four pieces of information:

Run the configure command:

aws configure

Installing kubectl:

curl -o /usr/local/bin/kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.21.2/2021-07-05/bin/darwin/amd64/kubectl
chmod +x /usr/local/bin/kubectl

Installing eksctl:

brew tap weaveworks/tap
brew install weaveworks/tap/eksctl

Installation

Log in to AWS console (and assume an administrator role, if applicable).

Create a cluster role:

  1. Open the IAM console.
  2. Choose Roles, then Create role.
  3. Under Use case, choose EKS from the list of services under Use cases for other AWS services.
  4. Choose EKS - Cluster for your use case, and then choose Next.
  5. On the Add permissions tab, choose Next.
  6. For Role name, enter a unique name for your role, such as eksClusterRole.
  7. For Description, enter descriptive text such as Amazon EKS - Cluster role.
  8. Choose Create role.

You will need the cluster role before you can create your cluster.

Create a VPC:

  1. Open the Amazon VPC console.
  2. On the VPC Dashboard, choose Launch VPC Wizard.
  3. Under Step 1: Select a VPC Configuration, on VPC with a Single Public Subnet, choose Select.

Enter the following information into the wizard and choose Create VPC.

IP CIDR block: 10.0.0.0/16
VPC name: ADS VPC
Public subnet: 10.0.0.0/24
Availability Zone: No Preference
Subnet name: ADS Subnet 1

You will need a VPC before you can create your cluster.

If you want to create services on your cluster which are publicly accessible - such as a web server - then you will need a load balancer, which requires at least two availability zones, and therefore a second subnet.

Create a second subnet:

  1. Open the Amazon VPC console.
  2. In the navigation pane, select Subnets, select the subnet with the name ADS Subnet 1, and select the Summary tab at the bottom of the page. Make a note of the Availability Zone of this subnet.
  3. Click Create Subnet and enter the following information in the Create Subnet dialog box and click Yes, Create.

Name tag: ADS Subnet 2
VPC: Select your VPC. This is the VPC with the name ADS VPC.
Availability Zone: Select any Availability Zone other than the one used above.
CIDR Block: 10.0.1.0/24

Create a cluster:

1. Open the EKS console.
2. Choose Create cluster.
3. On the Configure cluster page, fill in the following fields:

  • Name – A unique name for your cluster.
  • Kubernetes version – 1.21 is the latest stable at the time of this writing.
  • Cluster Service Role – Choose the role you just created above.

4. Select Next.
5. On the Specify networking page, select values for the following fields:
6. VPC – Choose the VPC you just created above.
7. Subnets – Choose the subnet(s) you just created above.
8. Security groups – There should be a default security group which you can use, for now.
9. For Cluster endpoint access – Choose Public and private.
10. Select Next.
11. On the Configure logging page, probably enable everything - you won't regret it.
12. Select Next.
13. Review and create page.

Create an IAM OIDC identity provider:

  1. Open the Amazon EKS console.
  2. Select the name of your cluster and then select the Configuration tab.
  3. In the Details section, note the value of the OpenID Connect provider URL.
  4. Open the IAM console.
  5. In the left navigation pane, choose Identity Providers under Access management. If a Provider is listed that matches the URL for your cluster, then you already have a provider for your cluster. If a provider isn't listed that matches the URL for your cluster, then you must create one.
  6. To create a provider, choose Add Provider.
  7. For Provider Type, choose OpenID Connect.
  8. For Provider URL, paste the OIDC issuer URL for your cluster, and then choose Get thumbprint.
  9. For Audience, enter sts.amazonaws.com and choose Add provider.

Create a node role:

To create your Amazon EKS node role in the IAM console

  1. Open the IAM console.
  2. Choose Roles, then Create role.
  3. Under Use case, choose EC2 and then choose Next.
  4. In the Filter policies box Add permissions page, enter AmazonEKSWorkerNodePolicy and then select the check box to the left of AmazonEKSWorkerNodePolicy in the search results.
  5. Choose Clear filters.

In the Filter policies box, enter AmazonEC2ContainerRegistryReadOnly. Then select the check box to the left of AmazonEC2ContainerRegistryReadOnly in the search results.

Either the AmazonEKS_CNI_Policy managed policy, or an IPv6 policy that you create must also be attached to either this role or to a different role that's mapped to the aws-node Kubernetes service account. We recommend assigning the policy to the role associated to the Kubernetes service account instead of assigning it to this role. For more information, see Configuring the Amazon VPC CNI plugin to use IAM roles for service accounts.

  1. Choose Next.
  2. For Role name, enter a unique name for your role, such as AmazonEKSNodeRole.
  3. For Description, replace the current text with descriptive text such as Amazon EKS - Node role.
  4. Choose Create role.

Create a node group:

  1. Wait for your cluster status to show as ACTIVE. You can't create a managed node group for a cluster that isn't already ACTIVE (You should be used to waiting for things to become active by now).
  2. Open the Amazon EKS console.
  3. Choose the name of the cluster that you want to create a managed node group in.
  4. Select the Configuration tab.
  5. On the Configuration tab, select the Compute tab, and then choose Add Node Group.
  6. On the Configure Node Group page, fill out the parameters accordingly, and then choose Next.
  • Name – Enter a unique name for your managed node group.
  • Node IAM Role – Choose the node role you created above.
  • Leave the rest of the options at their default.

7. On the Specify networking page, fill out the parameters accordingly, and then choose Next.

  • Subnets – Choose the subnet(s) created above.

Get kubectl configuration:

Firstly, verify that you have an appropriate IAM role. Run:

aws sts get-caller-identity

This should return a json document with your account information.

Next, you should add the kubectl configuration:

  aws eks update-kubeconfig --region your-region --name your-cluster-name

Now your cluster if finally (almost) ready. You can access it using:

kubectl get svc

You should see something like this:

NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.100.0.1   <none>        443/TCP   4h4m

Create an Ingress:

This is actually optional, you only need it if you want to use the cluster for things that are accessible to the world (so not really optional).

It's also not really AWS specific, so it is a bit easier:

helm repo add nginx-stable https://helm.nginx.com/stable
helm repo update
helm install nginx-ingress nginx-stable/nginx-ingress
kubectl scale deployment nginx-ingress-nginx-ingress --replicas=3

Alternatively

Of course, there is an easier (and cheaper) way.

1. Create a Digital Ocean account.
2. Click Create, choose Kubernetes. Scroll to the bottom. Click Create Cluster.

3. Click Add Node Pool. Click Add Pool.

Get kubectl configuration:

Generate an API token from the API tab:

Use the API token to grant doctl access to your DigitalOcean account. Pass in the token string when prompted by doctl auth init, and give this authentication context a name.

brew install doctl
doctl auth init --context <name>
doctl kubernetes cluster kubeconfig save <id>

Where the id is found on your new Kubernetes page in DO.

Create an Ingress:

Same as above.

helm repo add nginx-stable https://helm.nginx.com/stable
helm repo update
helm install nginx-ingress nginx-stable/nginx-ingress
kubectl scale deployment nginx-ingress-nginx-ingress --replicas=3

Conclusion

I'll leave it up to you to choose which option you'll take.

(I promise this isn't ad ad for DO - I just like them a lot better than AWS).

Mastodon