kubernetes安装和kube-dns:打开/run/flannel/subnet.env:没有这样的文件或目录

概观

kubeadm初始化和networking设置后, kube-dns无法启动(SetupNetworkError):

Error syncing pod, skipping: failed to "SetupNetwork" for "kube-dns-654381707-w4mpg_kube-system" with SetupNetworkError: "Failed to setup network for pod \"kube-dns-654381707-w4mpg_kube-system(8ffe3172-a739-11e6-871f-000c2912631c)\" using network plugins \"cni\": open /run/flannel/subnet.env: no such file or directory; Skipping pod" 

Kubernetes版本

 Client Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.4", GitCommit:"3b417cc4ccd1b8f38ff9ec96bb50a81ca0ea9d56", GitTreeState:"clean", BuildDate:"2016-10-21T02:48:38Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.4", GitCommit:"3b417cc4ccd1b8f38ff9ec96bb50a81ca0ea9d56", GitTreeState:"clean", BuildDate:"2016-10-21T02:42:39Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"} 

环境

适用于Mac的VMWare Fusion

OS

 NAME="Ubuntu" VERSION="16.04.1 LTS (Xenial Xerus)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 16.04.1 LTS" VERSION_ID="16.04" HOME_URL="http://www.ubuntu.com/" SUPPORT_URL="http://help.ubuntu.com/" BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/" VERSION_CODENAME=xenial UBUNTU_CODENAME=xenial 

内核(例如uname -a)

 Linux ubuntu-master 4.4.0-47-generic #68-Ubuntu SMP Wed Oct 26 19:39:52 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux 

问题是什么

 kube-system kube-dns-654381707-w4mpg 0/3 ContainerCreating 0 2m FirstSeen LastSeen Count From SubobjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 3m 3m 1 {default-scheduler } Normal Scheduled Successfully assigned kube-dns-654381707-w4mpg to ubuntu-master 2m 1s 177 {kubelet ubuntu-master} Warning FailedSync Error syncing pod, skipping: failed to "SetupNetwork" for "kube-dns-654381707-w4mpg_kube-system" with SetupNetworkError: "Failed to setup network for pod \"kube-dns-654381707-w4mpg_kube-system(8ffe3172-a739-11e6-871f-000c2912631c)\" using network plugins \"cni\": open /run/flannel/subnet.env: no such file or directory; Skipping pod" 

我预料会发生什么

kube-dns运行

如何重现它

 root@ubuntu-master:~# kubeadm init Running pre-flight checks <master/tokens> generated token: "247a8e.b7c8c1a7685bf204" <master/pki> generated Certificate Authority key and certificate: Issuer: CN=kubernetes | Subject: CN=kubernetes | CA: true Not before: 2016-11-10 11:40:21 +0000 UTC Not After: 2026-11-08 11:40:21 +0000 UTC Public: /etc/kubernetes/pki/ca-pub.pem Private: /etc/kubernetes/pki/ca-key.pem Cert: /etc/kubernetes/pki/ca.pem <master/pki> generated API Server key and certificate: Issuer: CN=kubernetes | Subject: CN=kube-apiserver | CA: false Not before: 2016-11-10 11:40:21 +0000 UTC Not After: 2017-11-10 11:40:21 +0000 UTC Alternate Names: [172.20.10.4 10.96.0.1 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] Public: /etc/kubernetes/pki/apiserver-pub.pem Private: /etc/kubernetes/pki/apiserver-key.pem Cert: /etc/kubernetes/pki/apiserver.pem <master/pki> generated Service Account Signing keys: Public: /etc/kubernetes/pki/sa-pub.pem Private: /etc/kubernetes/pki/sa-key.pem <master/pki> created keys and certificates in "/etc/kubernetes/pki" <util/kubeconfig> created "/etc/kubernetes/kubelet.conf" <util/kubeconfig> created "/etc/kubernetes/admin.conf" <master/apiclient> created API client configuration <master/apiclient> created API client, waiting for the control plane to become ready <master/apiclient> all control plane components are healthy after 14.053453 seconds <master/apiclient> waiting for at least one node to register and become ready <master/apiclient> first node is ready after 0.508561 seconds <master/apiclient> attempting a test deployment <master/apiclient> test deployment succeeded <master/discovery> created essential addon: kube-discovery, waiting for it to become ready <master/discovery> kube-discovery is ready after 1.503838 seconds <master/addons> created essential addon: kube-proxy <master/addons> created essential addon: kube-dns Kubernetes master initialised successfully! You can now join any number of machines by running the following on each node: kubeadm join --token=247a8e.b7c8c1a7685bf204 172.20.10.4 root@ubuntu-master:~# root@ubuntu-master:~# kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system dummy-2088944543-eo1ua 1/1 Running 0 47s kube-system etcd-ubuntu-master 1/1 Running 3 51s kube-system kube-apiserver-ubuntu-master 1/1 Running 0 49s kube-system kube-controller-manager-ubuntu-master 1/1 Running 3 51s kube-system kube-discovery-1150918428-qmu0b 1/1 Running 0 46s kube-system kube-dns-654381707-mv47d 0/3 ContainerCreating 0 44s kube-system kube-proxy-k0k9q 1/1 Running 0 44s kube-system kube-scheduler-ubuntu-master 1/1 Running 3 51s root@ubuntu-master:~# root@ubuntu-master:~# kubectl apply -f https://git.io/weave-kube daemonset "weave-net" created root@ubuntu-master:~# root@ubuntu-master:~# root@ubuntu-master:~# kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system dummy-2088944543-eo1ua 1/1 Running 0 47s kube-system etcd-ubuntu-master 1/1 Running 3 51s kube-system kube-apiserver-ubuntu-master 1/1 Running 0 49s kube-system kube-controller-manager-ubuntu-master 1/1 Running 3 51s kube-system kube-discovery-1150918428-qmu0b 1/1 Running 0 46s kube-system kube-dns-654381707-mv47d 0/3 ContainerCreating 0 44s kube-system kube-proxy-k0k9q 1/1 Running 0 44s kube-system kube-scheduler-ubuntu-master 1/1 Running 3 51s kube-system weave-net-ja736 2/2 Running 0 1h 

看起来你在运行kubeadm init之前configuration了flannel。 您可以尝试通过删除绒布来解决这个问题(删除configuration文件rm -f /etc/cni/net.d/*flannel*可能就足够了),但最好重新开始。