pods处于ContainerCreating状态

一、错误信息

创建 rc 之后 pods 起不来，一直处于 ContainerCreating 状态

[root@k8s-all-in-one pod-testweb]# kubectl get pods
NAME          READY     STATUS              RESTARTS   AGE
mysql-vmm72   0/1       ContainerCreating   0          2s

# 查看pods状态
[root@k8s-all-in-one pod-testweb]# kubectl describe pods mysql-vmm72
Name:           mysql-vmm72
Namespace:      default
Node:           127.0.0.1/127.0.0.1
Start Time:     Thu, 28 Feb 2019 21:36:08 -0500
······
Events:
  FirstSeen     LastSeen        Count   From                    SubObjectPath   Type            Reason          Message
  ---------     --------        -----   ----                    -------------   --------        ------          -------
  47s           47s             1       {default-scheduler }                    Normal          Scheduled       Successfully assigned mysql-vmm72 to 127.0.0.1
  23s           23s             1       {kubelet 127.0.0.1}                     Warning         FailedSync      Error syncing pod, skipping: failed to "StartContainer" for "POD" with ImagePullBackOff: "Back-off pulling image \"registry.access.redhat.com/rhel7/pod-infrastructure:latest\""

  47s   10s     3       {kubelet 127.0.0.1}             Warning FailedSync      Error syncing pod, skipping: failed to "StartContainer" for "POD" with ErrImagePull: "image pull failed for registry.access.redhat.com/rhel7/pod-infrastructure:latest, this may be because there are no credentials on this request.  details: (open /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt: no such file or directory)"

二、排查过程

1、查看到缺失的文件是个链接文件

[root@k8s-all-in-one pod-testweb]# ll /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt
lrwxrwxrwx 1 root root 27 Feb 28 21:32 /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt -> /etc/rhsm/ca/redhat-uep.pem

2、链接到的文件也处于缺失状态

[root@k8s-all-in-one pod-testweb]# ll /etc/rhsm/ca/redhat-uep.pem
ls: cannot access /etc/rhsm/ca/redhat-uep.pem: No such file or directory

三、生成证书

1、下载rpm包

wget http://mirror.centos.org/centos/7/os/x86_64/Packages/python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm

 2、生成证书

[root@k8s-all-in-one ~]# rpm2cpio python-rhsm-certificates-1.19.10-1.el7_4.x86_64.rpm | cpio -iv --to-stdout ./etc/rhsm/ca/redhat-uep.pem | tee /etc/rhsm/ca/redhat-uep.pem

3、查看文件

[root@k8s-all-in-one ~]# ll /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt
lrwxrwxrwx 1 root root 27 Feb 28 21:32 /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt -> /etc/rhsm/ca/redhat-uep.pem
[root@k8s-all-in-one ~]#
[root@k8s-all-in-one ~]# ll /etc/rhsm/ca/redhat-uep.pem
-rw-r--r-- 1 root root 7732 Feb 28 21:52 /etc/rhsm/ca/redhat-uep.pem

4、重建rc

[root@k8s-all-in-one pod-testweb]# kubectl delete  -f mysql-rc.yaml
replicationcontroller "mysql" deleted
[root@k8s-all-in-one pod-testweb]#
[root@k8s-all-in-one pod-testweb]# kubectl get pods
No resources found.
[root@k8s-all-in-one pod-testweb]#
[root@k8s-all-in-one pod-testweb]# kubectl create -f mysql-rc.yaml
replicationcontroller "mysql" created

5、状态成功

[root@k8s-all-in-one pod-testweb]# kubectl get pods
NAME          READY     STATUS              RESTARTS   AGE
mysql-t2l96   0/1       ContainerCreating   0          2s
[root@k8s-all-in-one pod-testweb]#
[root@k8s-all-in-one pod-testweb]# kubectl get pods
NAME          READY     STATUS    RESTARTS   AGE
mysql-t2l96   1/1       Running   0          9m

参考资料

1. K8S上mysql，无法创建rc解决，打不开/registry.access.redhat.com/redhat-ca.crt问题的解决