最近在看cks的相关材料,模拟一些大纲的考题练习,今天打开虚拟机,突然发现执行kubectl get nodes 一直显示连接集群失败
root@xxx:~# kubectl get nodes
The connection to the server 192.168.26.65:6443 was refused - did you specify the right host or port?
一开始以为是master的kubelet没有启动,
就登录master节点执行systemctl restart kubelet,发现无法running .
于是查看日志,发现证书过期了
$ journalctl -xe -u kubelet --no-page
.................
Apr 17 17:41:03 xxx.rhce.cc kubelet[2833]: E0417 17:41:03.020796 2833 bootstrap.go:265] part of the existing bootstrap client certificate in /etc/kubernetes/kubelet.conf is expired: 2022-03-26 16:09:29 +0000 UTC
Apr 17 17:41:03 xxx.rhce.cc kubelet[2833]: E0417 17:41:03.021144 2833 server.go:294] "Failed to run kubelet" err="failed to run Kubelet: unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubelet.conf: no such file or directory"
Apr 17 17:41:03 xxx.rhce.cc systemd[1]: kubelet.service: Main process exited, code=exited, status=1/FAILURE
...........................
1.在master节点操作,重新生产证书
root@xxx:~# cp -r /etc/kubernetes /etc/kubernetes-bak
root@xxx:~# rm -rf $HOME/.kube
root@xxx:~# mkdir -p $HOME/.kube
root@xxx:~# cp -i /etc/kubernetes/admin.conf /root/.kube/config
root@xxx:~# rm -rf /etc/kubernetes/*.conf
root@xxx:~# kubeadm init phase kubeconfig all
I0417 17:48:11.043177 5048 version.go:255] remote version is much newer: v1.23.5; falling back to: stable-1.22
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
2.重启master节点的kubelet
root@xxx:~# systemctl restart kubelet
root@xxx:~# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: active (running) since Sun 2022-04-17 17:48:22 CST; 35s ago
Docs: https://kubernetes.io/docs/home/
Main PID: 5106 (kubelet)
Tasks: 39 (limit: 4631)
CGroup: /system.slice/kubelet.service
├─5106 /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.co
├─6387 /opt/cni/bin/calico
├─6466 /opt/cni/bin/calico
└─6581 /opt/cni/bin/calico
.................................
3.检查集群状态
root@vms65:~# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master Ready control-plane,master 386d v1.22.2
node1 Ready <none> 386d v1.22.2
4.删除node 节点
root@xxx:~# kubectl delete node node1
node "node1" deleted
5.master节点生产新认证
root@xxx:~# kubeadm token create --print-join-command
kubeadm join 192.168.26.65:6443 --token 42wzhs.gcrwvwajn979j8zn --discovery-token-ca-cert-hash sha256:fe495fcb1bb1b014c7cde9aa5fed38ac11db1f8b6f7419fc8fb6cdbe9b622297
6.登录node节点,更新节点证书
root@node1:~# cd /etc/kubernetes/
root@node1:/etc/kubernetes#
root@node1:/etc/kubernetes# rm -rf *.conf
root@node1:/etc/kubernetes# cd pki/
root@node1:/etc/kubernetes/pki# rm -rf ca.crt
root@node1:/etc/kubernetes/pki# kubeadm join 192.168.26.65:6443 --token 42wzhs.gcrwvwajn979j8zn --discovery-token-ca-cert-hash sha256:fe495fcb1bb1b014c7cde9aa5fed38ac11db1f8b6f7419fc8fb6cdbe9b622297
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
证书更新完成