在重新启动kubernetes后,Mongodb容器的数据变成“只读”,glusterfs作为存储?

我的mongo作为kubernetes上的docker容器运行,glusterfs提供持久的卷。 当我重新启动kuberntes(机器关机并重新启动)后,所有的mongo pod都不能回来,他们的日志:

chown: changing ownership of `/data/db/user_management.ns': Read-only file system chown: changing ownership of `/data/db/storage.bson': Read-only file system chown: changing ownership of `/data/db/local.ns': Read-only file system chown: changing ownership of `/data/db/mongod.lock': Read-only file system 

这里/data/db/是安装的gluster音量,我可以确定它是rw模式!

 # kubectl get pod mongoxxx -o yaml apiVersion: v1 kind: Pod spec: containers: - image: mongo:3.0.5 imagePullPolicy: IfNotPresent name: mongo ports: - containerPort: 27017 protocol: TCP volumeMounts: - mountPath: /data/db name: mongo-storage volumes: - name: mongo-storage persistentVolumeClaim: claimName: auth-mongo-data # kubectl describe pod mongoxxx ... Volume Mounts: /data/db from mongo-storage (rw) /var/run/secrets/kubernetes.io/serviceaccount from default-token-wdrfp (ro) Environment Variables: <none> Conditions: Type Status Initialized True Ready False PodScheduled True Volumes: mongo-storage: Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace) ClaimName: auth-mongo-data ReadOnly: false ... # kubect get pv xxx apiVersion: v1 kind: PersistentVolume metadata: annotations: pv.kubernetes.io/bound-by-controller: "yes" name: auth-mongo-data resourceVersion: "215201" selfLink: /api/v1/persistentvolumes/auth-mongo-data uid: fb74a4b9-e0a3-11e6-b0d1-5254003b48ea spec: accessModes: - ReadWriteMany capacity: storage: 4Gi claimRef: apiVersion: v1 kind: PersistentVolumeClaim name: auth-mongo-data namespace: default glusterfs: endpoints: glusterfs-cluster path: infra-auth-mongo persistentVolumeReclaimPolicy: Retain status: phase: Bound 

当我在kubernetes节点上时:

 # ls -ls /var/lib/kubelet/pods/fc6c9ef3-e0a3-11e6-b0d1-5254003b48ea/volumes/kubernetes.io~glusterfs/auth-mongo-data/ total 163849 4 drwxr-xr-x. 2 mongo input 4096 1月 22 21:18 journal 65536 -rw-------. 1 mongo input 67108864 1月 22 21:16 local.0 16384 -rw-------. 1 mongo root 16777216 1月 23 17:15 local.ns 1 -rwxr-xr-x. 1 mongo root 2 1月 23 17:15 mongod.lock 1 -rw-r--r--. 1 mongo root 69 1月 23 17:15 storage.bson 4 drwxr-xr-x. 2 mongo input 4096 1月 22 21:18 _tmp 65536 -rw-------. 1 mongo input 67108864 1月 22 21:18 user_management.0 16384 -rw-------. 1 mongo root 16777216 1月 23 17:15 user_management.ns 

虽然音量挂载为rw但无法播放。

我的主机是CentOs 7.3 :Linux c4v160 3.10.0-514.el7.x86_64#1 SMP Tue Nov 11 22:42:41 UTC 2016 x86_64 x86_64 x86_64 GNU / Linux。

我想这是因为我提供的glusterfs卷是不干净的。 glusterfs卷infra-auth-mongo可能由脏目录组成。 一个解决scheme是删除这个卷,并创build另一个。

另一个解决scheme是在启动mongodb进程之前,先对mongodb dockerfile进行破解,强制更改/data/db的所有权。 像这样: https : //github.com/harryge00/mongo/commit/143bfc317e431692010f09b5c0d1f28395d2055b