Profile Applicability:
Level 1
Description:
In Amazon Elastic Kubernetes Service (EKS) or other Kubernetes environments, private nodes are Kubernetes worker nodes that are deployed within private subnets, ensuring that they do not have direct access to the internet. This is a security best practice for workloads that do not need direct internet access. By creating private nodes, you limit exposure to potential threats from the public internet and protect sensitive workloads by restricting internet communication.
When setting up Kubernetes clusters, it is important to ensure that worker nodes (also known as node groups) are created within private subnets. This allows traffic to flow through internal network paths and can be combined with NAT gateways or VPC peering to allow private nodes to access the internet securely if needed for updates or other requirements.
Rationale:
Creating private nodes in your Kubernetes cluster enhances security by:
Limiting Public Exposure: Private nodes are not exposed to the public internet, reducing the attack surface and preventing unauthorized external access.
Improved Security Posture: Keeps sensitive workloads isolated from external threats by routing all external traffic through private networking (e.g., VPC, NAT Gateways).
Compliance: Meets security requirements in compliance frameworks that require network segmentation and isolation (e.g., PCI-DSS, SOC 2, HIPAA).
Segmentation and Least Privilege: Helps in creating a zero trust environment where all communication is tightly controlled and monitored.
Impact:
Pros:
Increased Security: Restricting nodes to private subnets prevents direct access to the nodes, reducing attack vectors.
Improved Network Isolation: Resources in private subnets are isolated from public internet traffic, ensuring that only trusted internal systems can access them.
Compliance: Helps with meeting regulatory requirements that mandate network isolation for sensitive data.
Control Over Internet Access: You can use NAT Gateways or other secured mechanisms for controlled internet access if necessary.
Cons:
Increased Complexity: Private node deployment requires more complex networking configurations, such as setting up private subnets and ensuring proper routing.
Costs: You may incur additional costs for NAT Gateways, VPC peering, or VPN connections for external communication.
Limited Direct Internet Access: Private nodes cannot communicate directly with the internet, which requires configuration of intermediary systems like NAT Gateways.
Default Value:
By default, EKS clusters might create nodes in public subnets, depending on the configuration selected during cluster setup. You need to explicitly configure private nodes when creating the cluster to ensure worker nodes are placed in private subnets.
Pre-requisite:
AWS IAM Permissions:
eks:CreateCluster
eks:UpdateClusterConfig
eks:DescribeCluster
ec2:DescribeSubnets
ec2:CreateSecurityGroup
AWS CLI installed and configured.
Understanding of VPC, subnetting, and networking in AWS.
Ensure VPC is set up with both public and private subnets.
NAT Gateway or equivalent should be set up if you need internet access from private nodes.
Remediation:
Test Plan:
Using AWS Console:
Sign in to the AWS Management Console.
Navigate to Amazon EKS under Services.
Select the EKS cluster you want to check or modify.
In the Cluster Configuration, ensure that the worker nodes are deployed within private subnets.
For new clusters, during the creation process, choose private subnets in the VPC configuration for node group placement.
Verify the VPC configuration to ensure that the worker nodes are in private subnets.
Under Networking, check the subnet selection and ensure they are private.
If worker nodes are not in private subnets, create a new node group or modify the existing node group to deploy in private subnets.
Using AWS CLI:
To check if your EKS worker nodes are in private subnets, run:
aws eks describe-cluster --name <cluster-name> --query "cluster.resourcesVpcConfig.subnetIds"
Verify that the returned subnet IDs correspond to private subnets. If public subnets are returned, update the node group to use private subnets.
To create a new node group with private subnets, use:
aws eks create-nodegroup --cluster-name <cluster-name> --nodegroup-name <nodegroup-name> --subnets <private-subnet-id1> <private-subnet-id2> --node-role <node-role-arn> --disk-size 20 --instance-types t3.medium
To verify if your nodes are private, you can also describe the node group:
aws eks describe-nodegroup --cluster-name <cluster-name> --nodegroup-name <nodegroup-name>
Implementation Steps:
Using AWS Console:
Log in to the AWS Management Console and navigate to EKS.
In the EKS Dashboard, select the cluster you want to modify or create.
During the cluster creation process, select private subnets for the worker nodes.
Ensure that the private subnets are associated with private IPs, and public access is not allowed.
If modifying an existing cluster, navigate to Configuration > Node Groups and create or update the node group to place it in private subnets.
Save and apply the changes.
Using AWS CLI:
To create a new EKS node group in private subnets, run the following:
aws eks create-nodegroup --cluster-name <cluster-name> --nodegroup-name <nodegroup-name> --subnets <private-subnet-id1> <private-subnet-id2> --node-role <node-role-arn> --instance-types t3.medium --disk-size 20
Verify that the node group is placed in private subnets by running:
aws eks describe-nodegroup --cluster-name <cluster-name> --nodegroup-name <nodegroup-name>
Backout Plan:
If creating private nodes causes issues (e.g., connectivity problems or application failure):
Identify the affected node group and review its subnet configuration.
Revert the node group to use public subnets by running:
aws eks update-nodegroup-config --cluster-name <cluster-name> --nodegroup-name <nodegroup-name> --subnets <public-subnet-id1> <public-subnet-id2>
Verify that worker nodes are now placed in public subnets and the cluster is functioning as expected.
Note (Optional):
VPN or Direct Connect: If remote access is needed, you can use VPN or AWS Direct Connect to securely access private nodes in the VPC.
Testing: Ensure that your applications and workloads function correctly in the private node setup before applying changes to production environments.