Safely Draining a K8s Node

This is a quick guide about the draining a K8s nodes in the K8s Clusters

Relevant Documentation: https://kubernetes.io/docs/tasks/administer-cluster/safely-drain-node/

In managing the cluster, some other important tasks are kubectl drain and cordon. So what are these? Now you can use kubectl drain to gracefully terminate all pods on a node and mark the node as unschedulable. An in case kubectl drain doesn’t terminate all of the nodes, you can use — force to make it a little bit stronger. There’s also a kubectl cordon. Kubectl cordon marks a node as unschedulable. The difference with kubectl drain is that it keeps running pods on it. So if you are going to do maintenance, it’s a nice idea to start with kubectl cordon to make sure that nobody is going to run new pods, and then give it a few minutes before you are using kubectl drain to remove the Pods that are currently running. To undo the drain as well as the cordon, you can use kubectl uncordon.

Starting with the senario where we have 1 contorl node and 2 worker nodes running:

Begin by creating some objects. We will examine how these objects are affected by the drain process.

Take an example where we dont have any pod running for creating a pod here is pod.yml

kubectl apply -f pod.yml

Create a deployment with two replica as deployment.yml

kubectl apply -f deployment.yml

Get a list of pods. You should see the pods you just created (including the two replicas from the deployment). Take node ofwhich node these pods are running on.

kubectl get pods -o wide

Drain the node which the my-pod pod is running

kubectl drain <node name> — ignore-daemonsets — force

Check your list of pods again. You should see the deployment replica pods being moved to the remaining node. The regularpod will be deleted.

kubectl get pods -o wide

Uncordon the node to allow new pods to be scheduled there again.

kubectl uncordon <node name>

Refernence:

AcloudGuru: CKA Course