一、前言

其实这次实践算不上特别复杂，只是在实践过程中遇到了一些坑，以及填坑的方法是非常值得在以后的学习过程中参考借鉴的

二、知识准备

1.harbor是一个企业级的镜像仓库，它比起docker registry提供了更多的功能
2.在私有仓库中的镜像是需要经过一系列的验证才能够被pull，比如insecure-registries等
3.本文主要描述通过k8s的secret来进行验证

三、环境准备

四、创建secret

根据官方文档 https://v1-10.docs.kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/

root@k8s-master:~# kubectl create secret docker-registry regcred \
>         --docker-server=repo.mrvolleyball.com/library \
>         --docker-username=admin \
>         --docker-password='Harbor12345' \
>         --docker-email=chaisd63@163.com
secret "regcred" created

root@k8s-master:~# kubectl apply -f <(echo 'apiVersion: v1
> kind: Pod
> metadata:
>   name: private-reg
> spec:
>   containers:
>   - name: private-reg-container
>     image: repo.mrvolleyball.com/library/busybox:latest
>   imagePullSecrets:
>   - name: regcred')
pod "private-reg" created

根本没有难度啊，2条命令解决，当我满怀欣喜的查看状态的时候：

root@k8s-master:~# kubectl get pods private-reg
NAME          READY     STATUS             RESTARTS   AGE
private-reg   0/1       ImagePullBackOff   0          11s

root@k8s-master:~# kubectl describe pods private-reg
...
Events:
  Type     Reason                 Age                From                 Message
  ----     ------                 ----               ----                 -------
  Normal   Scheduled              40s                default-scheduler    Successfully assigned private-reg to k8s-master
  Normal   SuccessfulMountVolume  40s                kubelet, k8s-master  MountVolume.SetUp succeeded for volume "default-token-v9nkm"
  Normal   SandboxChanged         38s                kubelet, k8s-master  Pod sandbox changed, it will be killed and re-created.
  Normal   Pulling                22s (x2 over 39s)  kubelet, k8s-master  pulling image "repo.mrvolleyball.com/library/busybox:latest"
  Warning  Failed                 22s (x2 over 38s)  kubelet, k8s-master  Failed to pull image "repo.mrvolleyball.com/library/busybox:latest": rpc error: code = Unknown desc = Error response from daemon: pull access denied for repo.mrvolleyball.com/library/busybox, repository does not exist or may require 'docker login'
  Warning  Failed                 22s (x2 over 38s)  kubelet, k8s-master  Error: ErrImagePull
  Normal   BackOff                6s (x4 over 36s)   kubelet, k8s-master  Back-off pulling image "repo.mrvolleyball.com/library/busybox:latest"
  Warning  Failed                 6s (x4 over 36s)   kubelet, k8s-master  Error: ImagePullBackOff

这简直是给了当头一棒啊，我反反复复的检查之后，用户名、密码、仓库地址，官网提供的命令，都没问题啊，这TM到底是几个意思！！

在我一遍又一遍的check中发现了蛛丝马迹

root@k8s-master:~# kubectl get secret regcred  -o yaml
apiVersion: v1
data:
  .dockercfg: eyJhdXRocyI6eyJqZC1yZXBvLmludC5hbmtlcmppZWRpYW4uY29tIjp7InVzZXJuYW1lIjoiYWRtaW4iLCJwYXNzd29yZCI6IkhhcmJvcjEyMzQ1IiwiZW1haWwiOiJ5dXhpYW9jQGp1bWVpLmNvbSIsImF1dGgiOiJZV1J0YVc0NlNHRnlZbTl5TVRJek5EVT0ifX19
kind: Secret
metadata:
  creationTimestamp: 2018-10-29T09:33:40Z
  name: regcred
  namespace: default
  resourceVersion: "7670942"
  selfLink: /api/v1/namespaces/default/secrets/regcred
  uid: b82b7527-db5d-11e8-9e67-b2aca3e7fa90
type: kubernetes.io/dockercfg

官网提供的信息：

apiVersion: v1
data:
  .dockerconfigjson: eyJodHRwczovL2luZGV4L ... J0QUl6RTIifX0=
kind: Secret
metadata:
  ...
  name: regcred
  ...
type: kubernetes.io/dockerconfigjson

细心的朋友已经看出来了：
通过命令创建的data是.dockercfg，并且类型也不同：type: kubernetes.io/dockercfg
而官网提供的的data是.dockerconfigjson，类型是：type: kubernetes.io/dockerconfigjson

细微差别，千里之外啊，赶紧改了试一试：

root@k8s-master:~# kubectl get secret regcred  -o yaml > secret.yaml

修改之：

root@k8s-master:~# more secret.yaml
apiVersion: v1
data:
  .dockerconfigjson: eyJhdXRocyI6eyJqZC1yZXBvLmludC5hbmtlcmppZWRpYW4uY29tIjp7InVzZXJuYW1lIjoiYWRtaW4iLCJwYXNzd29yZCI6IkhhcmJvcjEyMzQ1IiwiZW1haWwiOiJ5dXhpYW9jQGp1bWVpLmNvbSIsImF1dGgiOiJZV1J0YVc0NlNHRnlZbTl5TVRJek5EVT0ifX19
kind: Secret
metadata:
  name: regcred
  namespace: default
  selfLink: /api/v1/namespaces/default/secrets/regcred
type: kubernetes.io/dockerconfigjson

然后重建之：

root@k8s-master:~# kubectl delete secret regcred
root@k8s-master:~# kubectl create -f secret.yaml
root@k8s-master:~# kubectl delete -f <(echo 'apiVersion: v1
kind: Pod
metadata:
  name: private-reg
spec:
  containers:
  - name: private-reg-container
    image: repo.mrvolleyball.com/library/busybox:latest
  imagePullSecrets:
  - name: regcred')
pod "private-reg" deleted
root@k8s-master:~# kubectl apply -f <(echo 'apiVersion: v1
kind: Pod
metadata:
  name: private-reg
spec:
  containers:
  - name: private-reg-container
    image: repo.mrvolleyball.com/library/busybox:latest
  imagePullSecrets:
  - name: regcred')
pod "private-reg" created

root@k8s-master:~# kubectl get pods private-reg
NAME          READY     STATUS    RESTARTS   AGE
private-reg   1/1       Running   0          15s

终于，达到了需要的效果

五、小结

● 再详细的文档也需要一步一步实践，只有实践了，才是自己的知识，否则就是人云亦云
● 看文档一定要非常仔细，而且要对比对比在对比，如果你确定了和文档的一样没问题，做出来的依然错误，就去提issue吧

至此，本文结束
在下才疏学浅，有撒汤漏水的，请各位不吝赐教...