K8S: Cluster version upgrade(En)

Today, we’ll discuss how to upgrade Kubernetes versions (specifically, z-version). Kubernetes gets updated frequently, so knowing how to perform version upgrades is a crucial skill for administrators. Before upgrading, it’s important to check the available upgrade paths because not every version can be directly upgraded to your desired one.

Which path is the right one for me?

The upgrade sequence is Master => Worker.

Alright, let’s get started!

1. Checking Status

[master]# kubectl get nodes

2. Upgrading the Control-Plane (1.25.11 -> 1.25.12)

#-----------------------------------------------
# S2-1. Verify the current version
#-----------------------------------------------
[master01]# kubeadm version -o json

#-----------------------------------------------
# S2-2. Confirm the target version
#-----------------------------------------------
[master01]# kubeadm upgrade plan

#-----------------------------------------------
# S2-3. Install kubeadm (all master nodes)
#-----------------------------------------------
[master01]# yum list --showduplicates kubeadm --disableexcludes=kubernetes
[master01]# yum install kubeadm-1.25.12-0

#-----------------------------------------------
# S2-4. drain workload
#-----------------------------------------------
[master01]# kubectl drain <master-node> --ignore-daemonsets

※ If there are pods using local storage, an error may occur. 
   If it's not needed, you can add `--delete-local-data`

※ Drain the master node to mark it as unschedulable, evicting workloads to 
   put the node in maintenance mode, and gracefully terminate the pods.

#-----------------------------------------------
# S2-5. Run kubeadm upgrade 
#-----------------------------------------------
[master01]# kubeadm upgrade apply v1.25.12
[master01]# kubeadm version -o json

※ To another master node
# kubeadm upgrade node

#-----------------------------------------------
# S2-6. Upgrade kubectl and kubelet (all masters)
#-----------------------------------------------
[master01]# yum list --showduplicates kubelet --disableexcludes=kubernetes
[master01]# yum install -y kubelet-1.25.12-0 kubectl-1.25.12-0 --disableexcludes=kubernetes 
[master01]# systemctl daemon-reload ; systemctl restart kubelet
 
[master01]# kubectl get nodes

#-----------------------------------------------
# S2-7. uncordon workload (all masters)
#-----------------------------------------------
[master01]# kubectl uncordon master01.test.example.poc
[master01]# kubectl uncordon master02.test.example.poc
[master01]# kubectl uncordon master03.test.example.poc

※ During the upgrade process, kubeadm will back up etcd data in the following locations on the nodes:

# cd /etc/kubernetes/tmp
# ls -al

3. Upgrade Workers (from version 1.25.11 to 1.25.12)

#-----------------------------------------------
# S3-1. Upgrade kubeadm (workers)
#-----------------------------------------------
[worker01]# yum install kubeadm-1.25.12-0
[worker02]# yum install kubeadm-1.25.12-0

#-----------------------------------------------
# S3-2. drain workload
#-----------------------------------------------
[master01]# kubectl drain <worker-node> --ignore-daemonsets
[master01]# kubectl get nodes

#-----------------------------------------------
# S3-3. upgrade node (workers)
#-----------------------------------------------
[worker01]# kubeadm upgrade node

#-----------------------------------------------
# S3-4. Upgrade kubectl and kubelet (all workers)
#-----------------------------------------------
[worker01]# yum list --showduplicates kubelet --disableexcludes=kubernetes
[worker01]# yum install -y kubelet-1.25.12-0 kubectl-1.25.12-0 --disableexcludes=kubernetes 
[worker01]# systemctl daemon-reload ; systemctl restart kubelet

[master01]# kubectl get nodes

#-----------------------------------------------
# S3-5. uncordon workload (all workers)
#-----------------------------------------------
[lb01]# kubectl uncordon worker01.test.example.poc
[lb01]# kubectl uncordon worker02.test.example.poc

#-----------------------------------------------
# S3-6. Make sure to check the cluster status
#-----------------------------------------------
[lb01]# kubectl get --raw='/readyz?verbose'

#-----------------------------------------------
# S3-7. Confirm the pod status.
#-----------------------------------------------
[lb01]# kubectl get po -n kube-system

The minor version upgrade is done. Such upgrades are usually performed when addressing critical issues, but it’s crucial to assess the potential impact on your applications before proceeding with the upgrade.

For major version upgrades, you’ll do something similar, but pay close attention to the upgrade path and its potential effects.

And always remember, before you upgrade, make sure to back up your data!

Reference:

• How to Upgrade Kubernetes Cluster Using Kubeadm?
• Upgrading kubeadm clusters