[k8s] kubernetes Upgrade-Node

Control plane upgrade (hk8s)

• Kubernetes cluster upgrade
  • kubeadm, kubelet, kubectl을 각각 업그레이드
• Woker node Upgrade
  1. Upgrade할 node에 접속
  2. kubeadm 업그레이드
  3. "kubeadm upgrade" 호출
  4. 노드 드레인
  5. kubelet과 kubectl 업그레이드
  6. 노드 uncordon

 

Solution

• Worker node Upgrade (https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/upgrading-linux-nodes/)

1. Upgrade할 node에 접속

# worker1 업그레이드를 위해 접속. (나머지 worker도 전 과정을 똑같이 진행해주면 된다.)
$ ssh hk8s-worker1 
Welcome to Ubuntu 22.04.1 LTS (GNU/Linux 5.15.0-76-generic x86_64)

$ sudo -i

2. kubeadm 업그레이드

$ apt-mark unhold kubeadm && \
  apt-get update && apt-get install -y kubeadm=1.27.1-00 && \
  apt-mark hold kubeadm
...
The following packages will be upgraded:
  kubeadm
1 upgraded, 0 newly installed, 0 to remove and 78 not upgraded.
Need to get 9928 kB of archives.
...

3. "kubeadm upgrade" 호출

$ sudo kubeadm upgrade node
[upgrade] Reading configuration from the cluster...
[upgrade] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[preflight] Running pre-flight checks
[preflight] Skipping prepull. Not a control plane node.
[upgrade] Skipping phase. Not a control plane node.
[upgrade] Backing up kubelet config file to /etc/kubernetes/tmp/kubeadm-kubelet-config495210393/config.yaml
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[upgrade] The configuration for this node was successfully updated!
[upgrade] Now you should go ahead and upgrade the kubelet package using your package manager.

4. 노드 드레인

# worker의 drain 작업은 콘솔환경으로 나가서 진행해야 한다.
$ exit
logout #root
$ exit
logout #hk8s-worker1
Connection to hk8s-worker1 closed.

$ kubectl drain hk8s-worker1 --ignore-daemonsets
node/hk8s-worker1 cordoned
evicting pod kube-system/metrics-server-7fcb88f75-8wnhn
evicting pod calico-apiserver/calico-apiserver-79856c48fc-hpxjm
evicting pod calico-apiserver/calico-apiserver-79856c48fc-x9hv2
evicting pod calico-system/calico-kube-controllers-7f55d475d-ctdqx
evicting pod calico-system/calico-typha-7b57995ddf-zwh2k
evicting pod kube-system/coredns-5d78c9869d-6v46k
evicting pod kube-system/coredns-5d78c9869d-qctzk

$ kubectl get nodes
NAME           STATUS                     ROLES           AGE    VERSION
hk8s-master    Ready                      control-plane   131d   v1.27.1
hk8s-worker1   Ready,SchedulingDisabled   <none>          131d   v1.26.0
hk8s-worker2   NotReady                   <none>          131d   v1.26.0

5. kubelet과 kubectl 업그레이드

$ ssh hk8s-worker1 
Welcome to Ubuntu 22.04.1 LTS (GNU/Linux 5.15.0-76-generic x86_64)
$ sudo -i

# kubelet과 kubectl 업그레이드
$ apt-mark unhold kubelet kubectl && \
  apt-get update && apt-get install -y kubelet=1.27.1-00 kubectl=1.27.1-00 && \
  apt-mark hold kubelet kubectl
...
The following packages will be upgraded:
  kubectl kubelet
2 upgraded, 0 newly installed, 0 to remove and 77 not upgraded.
Need to get 29.0 MB of archives.
...

# kubelet을 다시 시작
$ sudo systemctl daemon-reload
$ sudo systemctl restart kubelet

6. 노드 uncordon

$ exit
logout #root
$ exit
logout #hk8s-worker1
Connection to hk8s-worker1 closed.

# uncordon
$ kubectl uncordon hk8s-worker1
node/hk8s-worker1 uncordoned

# 업그레이드 확인
$ kubectl get nodes
NAME           STATUS     ROLES           AGE    VERSION
hk8s-master    Ready      control-plane   131d   v1.27.1
hk8s-worker1   Ready      <none>          131d   v1.27.1
hk8s-worker2   NotReady   <none>          131d   v1.26.0