Profile Applicability:
 • Level 1

Description:
Allow Kubelet to manage iptables configurations automatically. By setting --make-iptables-util-chains to true, the Kubelet will manage iptables and ensure that the iptables configuration stays in sync with pod network configurations.

Rationale:
 Kubelets can automatically manage required iptables changes, ensuring network configurations are consistent and accurate. If iptables is not managed by Kubelet, manually configured iptables rules may become out of sync with pod network configurations, potentially disrupting pod communication or access to external networks.

Impact:
 Pros:

  • Ensures iptables configuration remains consistent with Kubernetes network changes.

  • Reduces manual configuration errors and network misconfigurations.

  • Ensures iptables does not block legitimate pod traffic.

Cons:

  • Conflicts may arise if another iptables management solution is used simultaneously.

Default Value:
 The default value may vary depending on the Kubernetes distribution. Check AWS EKS documentation for default values.

Pre-requisites:

  • SSH or Session Manager access to worker nodes.

  • Permissions to modify Kubelet configuration files or systemd service arguments.

  • Kubernetes access to verify changes via /configz or deploy privileged pods.

Remediation

Test Plan:

Using AWS Console:

  1. SSH into each worker node and run the following command to find the Kubelet process:
    ps -ef | grep kubelet

  1. If the --make-iptables-util-chains argument is present, verify it is set to true.

  2. If the argument is not present, look for the Kubelet config file by checking the --config argument:
    --config /etc/kubernetes/kubelet/kubelet-config.json

  1. Open the Kubelet config file:
    cat /etc/kubernetes/kubelet/kubelet-config.json

  1. Verify that makeIPTablesUtilChains is set to true in the configuration file.

Using AWS CLI:

  1. Get the list of nodes in your cluster:

    kubectl get nodes

  1. Start a proxy to the Kubernetes API:

    kubectl proxy --port=8001

  1. In a new terminal, for each node, run:

    export NODE_NAME=my-node-name
curl -sSL "http://localhost:8001/api/v1/nodes/${NODE_NAME}/proxy/configz"


  1. Verify that the makeIPTablesUtilChains is set to true in the returned JSON.

Implementation Plan

Using AWS Console:

  1. SSH into the worker node and run ps -ef | grep kubelet to find the Kubelet process.

  2. If a Kubelet configuration file is specified by --config, open the file:
    sudo less /path/to/kubelet-config.json

  1. Add or modify the entry as follows:
    "makeIPTablesUtilChains": true

  1. Ensure that the Kubelet service file at /etc/systemd/system/kubelet.service.d/10-kubelet-args.conf does not set --make-iptables-util-chains, as this would override the configuration file setting.

  2. Reload systemd and restart the Kubelet service:
    systemctl daemon-reload
    systemctl restart kubelet.service
     systemctl status kubelet -l

Using AWS CLI:

  1. Open the Kubelet configuration file on the node and set makeIPTablesUtilChains to true.

  2. For systemd-based systems, add the following argument to the service file:

    --make-iptables-util-chains=true

  1. Reload systemd and restart the Kubelet service:

    systemctl daemon-reload
 systemctl restart kubelet.service
systemctl status kubelet -l

Backout Plan

Using AWS Console:

  1. If issues arise, revert the Kubelet configuration file to its previous state without the makeIPTablesUtilChains setting or remove the systemd argument --make-iptables-util-chains.

  2. Restart the Kubelet service after rolling back changes.

    Using AWS CLI:

    1. Remove the --make-iptables-util-chains flag from the systemd unit or revert the configuration file changes if necessary.

    2. Restart the Kubelet service:

      systemctl daemon-reload
 systemctl restart kubelet.service

References:

  1. Kubelet Documentation

  2. Reconfigure a Node's Kubelet