When working with Kubernetes clusters, there are times when you need to perform maintenance on a node — such as upgrading the OS, applying security patches, or troubleshooting hardware issues. But before you do that, it’s crucial to ensure that workloads are not disrupted unexpectedly.
Kubernetes provides three handy commands to manage node availability during maintenance:
- cordon — stop new pods from being scheduled on the node
- drain — evict existing pods safely from the node
- uncordon — allow scheduling on the node again
Let’s dive into each one, with examples and best practices.
cordon
The cordon command marks a node as unschedulable. This means the Kubernetes scheduler will not assign any new pods to the node. However, any pods currently running on the node will continue to run normally.
kubectl cordon <node-name>
When to Use:
- You’re planning maintenance soon but not immediately.
- You want to prevent more pods from being scheduled before a rollout.
Check Node Status:
kubectl get nodes
You’ll see:
NAME STATUS ROLES AGE VERSION
worker-node-1 Ready,SchedulingDisabled <role> 30d v1.29.2
Use cordon when you want to prevent new workloads from being scheduled on a node but don’t want to disrupt currently running workloads.
drain
The drain command cordons the node and then evicts all non-critical pods from the node. It’s used to safely remove workloads before shutting down or rebooting a node.
kubectl drain <node-name> --ignore-daemonsets --delete-emptydir-data
When to Use:
- You are about to shut down or upgrade a node.
- You want to safely reschedule pods elsewhere in the cluster.
Important Flags:
- - -ignore-daemonsets: Ensures pods created by DaemonSets are not evicted.
- - -delete-emptydir-data: Removes data from emptyDir volumes.
Note: drain will fail if pods are managed by controllers that don’t allow eviction (like stateful apps without a proper Pod Disruption Budget).
Drain is a more forceful command. It not only cordons the node (marks it unschedulable), but it also evicts all running pods from the node except for DaemonSets and static pods.
uncordon
After you’ve completed maintenance and want to let the node accept pods again, use uncordon. This marks the node as schedulable.
kubectl uncordon <node-name>
When to Use:
- After maintenance is complete and the node is healthy.
- To allow pods to be scheduled on the node again.
Summary Table
| Command | Action | Affects Running Pods? | Scheduler Behavior |
|---|---|---|---|
cordon |
Marks node unschedulable | ❌ No | Stops new pods from being added |
drain |
Evicts pods and cordons node | ✅ Yes | Clears node of pods |
uncordon |
Marks node schedulable again | N/A | Allows new pods on node |
Pro Tips
- Always combine - -ignore-daemonsets with drain unless you intend to remove system-level pods.
- If using emptyDir, remember drain will delete data unless configured otherwise.
- Automate these steps in scripts for cluster-wide maintenance.
Wrapping Up
Understanding how cordon, drain, and uncordon work is key to safe and controlled node maintenance in Kubernetes. Used correctly, these commands allow you to maintain high availability while keeping your cluster stable and workloads happy.
Need help with automating node draining across multiple clusters? Or want a sample script? Just ask!