一、各节点准备环境
| ip | hostname | role |
|---|---|---|
| 10.39.14.204 | k8s-etcd-host0 | etcd-node |
| 10.39.14.205 | k8s-etcd-host1 | etcd-node |
| 10.39.14.206 | k8s-etcd-host2 | etcd-node |
1、 安装 docker, kubelet, kubeadm
check docker
$ docker --version
Docker version 17.05.0-ce, build e1bfc47
$ systemctl enable docker && systemctl restart docker && systemctl status docker
docker is active
下载 kubelet、kubeadm 到 /usr/bin
链接: https://pan.baidu.com/s/1K5rA3Di_uVgE96pK3tst4Q 提取码: hm6r
chmod +x /usr/bin/kubelet
chmod +x /usr/bin/kubeadm
2、 import etcd、pause image
tar -xvf k8s-images.tar
docker load < k8s-images/etcd.tar
docker load < k8s-images/pause.tar
3、将 kubelet 配置为 etcd 的服务管理器。
必须通过创建具有更高优先级的新文件来覆盖 kubeadm 提供的 kubelet 单元文件。
$ cat << EOF > /etc/systemd/system/kubelet.service
[Unit]
Description=kubelet: The Kubernetes Node Agent
Documentation=https://kubernetes.io/docs/
[Service]
ExecStart=/usr/bin/kubelet
Restart=always
StartLimitInterval=0
RestartSec=10
[Install]
WantedBy=multi-user.target
EOF
$ mkdir -p /etc/systemd/system/kubelet.service.d/
$ cat << EOF > /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
[Service]
Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
EnvironmentFile=-/var/lib/kubelet/kubeadm-flags.env
EnvironmentFile=-/etc/sysconfig/kubelet
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS
EOF
$ cat << EOF > /etc/systemd/system/kubelet.service.d/20-etcd-service-manager.conf
[Service]
ExecStart=
ExecStart=/usr/bin/kubelet --address=127.0.0.1 --pod-manifest-path=/etc/kubernetes/manifests --allow-privileged=true
Restart=always
EOF
重启 kubelet
systemctl daemon-reload
systemctl enable kubelet && systemctl restart kubelet && systemctl status kubelet
二、host0: 生成并分发证书
在 host0 上生成所有证书，并仅将必要的文件分发给其他节点
1、 为kubeadm创建配置文件
使用以下脚本为每个将在其上运行 etcd 成员的主机生成一个 kubeadm 配置文件
export HOST0=10.39.14.204
export HOST1=10.39.14.205
export HOST2=10.39.14.206
rm -rf /tmp/${HOST0}/ /tmp/${HOST1}/ /tmp/${HOST2}/
mkdir -p /tmp/${HOST0}/ /tmp/${HOST1}/ /tmp/${HOST2}/
ETCDHOSTS=(${HOST0} ${HOST1} ${HOST2})
NAMES=("infra0" "infra1" "infra2")
for i in "${!ETCDHOSTS[@]}"; do
HOST=${ETCDHOSTS[$i]}
NAME=${NAMES[$i]}
cat << EOF > /tmp/${HOST}/kubeadmcfg.yaml
apiVersion: "kubeadm.k8s.io/v1beta1"
kind: ClusterConfiguration
etcd:
local:
serverCertSANs:
- "${HOST}"
peerCertSANs:
- "${HOST}"
extraArgs:
initial-cluster: infra0=https://${ETCDHOSTS[0]}:2380,infra1=https://${ETCDHOSTS[1]}:2380,infra2=https://${ETCDHOSTS[2]}:2380
initial-cluster-state: new
name: ${NAME}
listen-peer-urls: https://${HOST}:2380
listen-client-urls: https://${HOST}:2379
advertise-client-urls: https://${HOST}:2379
initial-advertise-peer-urls: https://${HOST}:2380
kubernetesVersion: 1.13.0
EOF
done
2、生成证书颁发机构
rm -rf /etc/kubernetes/pki
kubeadm init phase certs etcd-ca
这会创建两个文件
/etc/kubernetes/pki/etcd/ca.crt
/etc/kubernetes/pki/etcd/ca.key
3、为每个成员创建证书
# host2
kubeadm init phase certs etcd-server --config=/tmp/${HOST2}/kubeadmcfg.yaml
kubeadm init phase certs etcd-peer --config=/tmp/${HOST2}/kubeadmcfg.yaml
kubeadm init phase certs etcd-healthcheck-client --config=/tmp/${HOST2}/kubeadmcfg.yaml
kubeadm init phase certs apiserver-etcd-client --config=/tmp/${HOST2}/kubeadmcfg.yaml
cp -R /etc/kubernetes/pki /tmp/${HOST2}/
find /etc/kubernetes/pki -not -name ca.crt -not -name ca.key -type f -delete
# host1
kubeadm init phase certs etcd-server --config=/tmp/${HOST1}/kubeadmcfg.yaml
kubeadm init phase certs etcd-peer --config=/tmp/${HOST1}/kubeadmcfg.yaml
kubeadm init phase certs etcd-healthcheck-client --config=/tmp/${HOST1}/kubeadmcfg.yaml
kubeadm init phase certs apiserver-etcd-client --config=/tmp/${HOST1}/kubeadmcfg.yaml
cp -R /etc/kubernetes/pki /tmp/${HOST1}/
find /etc/kubernetes/pki -not -name ca.crt -not -name ca.key -type f -delete
# host0
kubeadm init phase certs etcd-server --config=/tmp/${HOST0}/kubeadmcfg.yaml
kubeadm init phase certs etcd-peer --config=/tmp/${HOST0}/kubeadmcfg.yaml
kubeadm init phase certs etcd-healthcheck-client --config=/tmp/${HOST0}/kubeadmcfg.yaml
kubeadm init phase certs apiserver-etcd-client --config=/tmp/${HOST0}/kubeadmcfg.yaml
# No need to move the certs because they are for HOST0
# clean up certs that should not be copied off this host
find /tmp/${HOST2} -name ca.key -type f -delete
find /tmp/${HOST1} -name ca.key -type f -delete
4、到每个节点上复制证书和 kubeadm 配置
已生成证书，现在必须将它们移动到各自的主机。
（可以使用一个能 ssh 到所有节点的跳板机来做中转。）
[跳板机]$ scp -r ~/host1 root@<host1_ip>:~
[跳板机]$ ssh root@<host1_ip>
chown -R root:root ~/host1/pki/
rm -rf /etc/kubernetes/
mkdir /etc/kubernetes
mv ~/host1/pki/ /etc/kubernetes/
mv -f ~/host1/kubeadmcfg.yaml ~
cd ~
rm -rf host1/
5、确保存在所有预期文件
所需文件的完整列表$HOST0是：
/tmp/${HOST0}
└── kubeadmcfg.yaml
/etc/kubernetes/pki
├── apiserver-etcd-client.crt
├── apiserver-etcd-client.key
└── etcd
├── ca.crt
├── ca.key
├── healthcheck-client.crt
├── healthcheck-client.key
├── peer.crt
├── peer.key
├── server.crt
└── server.key
$HOST1：
$HOME
└── kubeadmcfg.yaml
---
/etc/kubernetes/pki
├── apiserver-etcd-client.crt
├── apiserver-etcd-client.key
└── etcd
├── ca.crt
├── healthcheck-client.crt
├── healthcheck-client.key
├── peer.crt
├── peer.key
├── server.crt
└── server.key
$HOST2:
$HOME
└── kubeadmcfg.yaml
---
/etc/kubernetes/pki
├── apiserver-etcd-client.crt
├── apiserver-etcd-client.key
└── etcd
├── ca.crt
├── healthcheck-client.crt
├── healthcheck-client.key
├── peer.crt
├── peer.key
├── server.crt
└── server.key
三、创建静态pod清单
现在证书和配置已到位，是时候创建清单了。在每个主机上运行 kubeadm 命令以生成 etcd 的静态清单。
root@HOST0 $ kubeadm init phase etcd local --config=/tmp/${HOST0}/kubeadmcfg.yaml
root@HOST1 $ kubeadm init phase etcd local --config=/root/kubeadmcfg.yaml
root@HOST2 $ kubeadm init phase etcd local --config=/root/kubeadmcfg.yaml
检查群集运行状况
$ docker run --rm -it --net host -v /etc/kubernetes:/etc/kubernetes k8s.gcr.io/etcd:3.2.24 etcdctl --cert-file /etc/kubernetes/pki/etcd/peer.crt --key-file /etc/kubernetes/pki/etcd/peer.key --ca-file /etc/kubernetes/pki/etcd/ca.crt --endpoints https://10.39.14.204:2379 cluster-health
member a04d01ef922eadf5 is healthy: got healthy result from https://10.39.14.205:2379
member d0c00871a44c4535 is healthy: got healthy result from https://10.39.14.204:2379
member e54116cb3d93012b is healthy: got healthy result from https://10.39.14.206:2379
cluster is healthy
