AWS中的Docker卷 – 写入EBS,EFS,S3

如何在Swarm中编写Docker容器将数据写入任何文件系统挂载(如EBS,EFS),甚至是从S3中读取数据?

EFS,EBS,S3插件与AWS政策工作

https://github.com/codedellemc/rexray/issues/820#issuecomment-323624765的完整答案详情。

  • 我使用了RexRay Docker Volume插件来支持EBS,EFS和S3
    • 该卷允许您在不使用任何AWS API的情况下自动将其中一个作为文件系统装载到容器中。 也就是说,对容器透明!

这里是例子

docker service create --replicas 10 --name echo-to-efs --mount type=volume,source=echo_data,target=/test,volume-driver=rexray/efs alpine /bin/sh -c 'echo "writing from $HOSTNAME" >> \ /test/echo.log ; cat /test/echo.log ; sleep 30' iy4rzd4imq776abjlqdqf8l10 

EFS MOUNTS 在这里输入图像说明

EBS MOUNTS 在这里输入图像说明

安装卷插件

重复每个人,使用他们的docker命令行参数…

[root@ip-10-zz-yy-xx ec2-user]# sudo docker plugin install --grant-all-permissions rexray/s3fs latest: Pulling from rexray/s3fs 5a6437c91cc6: Download complete Digest: sha256:6ba105974d5298ff49def3c83e5547d41e573e0a3cb8604f1dcb1a0f90ee070b Status: Downloaded newer image for rexray/s3fs:latest Installed plugin rexray/s3fs

Docker插件ls

  • 设置UserData后安装3个插件,一切正常!

[ec2-user@ip-10-82-48-59 ~]$ docker plugin ls ID NAME DESCRIPTION ENABLED 0eb15d4a9594 rexray/s3fs:latest REX-Ray FUSE Driver for Amazon Simple Stor... true 6bae636cb377 rexray/ebs:latest REX-Ray for Amazon EBS true 9e231d9177c2 rexray/efs:latest REX-Ray for Amazon EFS true

Docker容器自动创build/安装卷

作为一个经验法则,我学到了以下几点:

  • EFS =并发写入(x副本scheme)
  • EBS =单个写入(1个复制scheme)

所以,下面的例子显示了以下内容:

  • 同时创build10个Docker容器,写入同一个文件

Last login: Mon Aug 21 00:34:46 2017 from 10.82.49.230 [ec2-user@ip-10-zz-yy-xx ~]$ docker service create --replicas 10 --name echo-to-efs
--mount type=volume,source=echo_data,target=/test,volume-driver=rexray/efs alpine /bin/sh -c 'echo "writing from $HOSTNAME" >> /test/echo.log ; cat /test/echo.log ; sleep 30' iy4rzd4imq776abjlqdqf8l10

  • docker ps显示其中一些写作和失败
    • 如果有人能解释为什么在这种情况下的一些失败,请让我知道…
    • 我有REXRAY_PREEMPT=true所以,不知道这是不相关的

[ec2-user@ip-10-82-48-59 ~]$ docker service ps echo-to-efs ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS 8c8lxqcoc3n5 echo-to-efs.1 alpine:latest ip-10-82-48-59.vpc.internal Running Running 8 seconds ago 23meciw5cvad echo-to-efs.2 alpine:latest ip-10-82-49-52.vpc.internal Running Running 2 seconds ago si7lg30kb13b \_ echo-to-efs.2 alpine:latest ip-10-82-48-93.vpc.internal Shutdown Failed about a minute ago "starting container failed: er…" rmj163xu97ai echo-to-efs.3 alpine:latest ip-10-82-48-96.vpc.internal Running Running 7 seconds ago 5jtao2yu5udt echo-to-efs.4 alpine:latest ip-10-82-48-95.vpc.internal Running Running 8 seconds ago wot9hd5nbjsz echo-to-efs.5 alpine:latest ip-10-82-49-71.vpc.internal Running Running 7 seconds ago hcbbgfiuga9c echo-to-efs.6 alpine:latest ip-10-82-48-140.vpc.internal Running Starting 49 seconds ago wrnnrsm16usr \_ echo-to-efs.6 alpine:latest ip-10-82-48-25.vpc.internal Shutdown Rejected 58 seconds ago "create echo_data: VolumeDrive…" w3v4x0pf9uri \_ echo-to-efs.6 alpine:latest ip-10-82-48-37.vpc.internal Shutdown Rejected about a minute ago "create echo_data: VolumeDrive…" okgda40hb4hy \_ echo-to-efs.6 alpine:latest ip-10-82-49-107.vpc.internal Shutdown Rejected about a minute ago "create echo_data: VolumeDrive…" bzoafno0gb4d echo-to-efs.7 alpine:latest ip-10-82-48-74.vpc.internal Running Running 7 seconds ago rk7uebz4cdql echo-to-efs.8 alpine:latest ip-10-82-49-111.vpc.internal Running Running 6 seconds ago hy1rfid4zitl echo-to-efs.9 alpine:latest ip-10-82-48-162.vpc.internal Running Running 7 seconds ago k5lmmmu44yh3 echo-to-efs.10 alpine:latest ip-10-82-48-187.vpc.internal Running Running 8 seconds ago

  • 现在我们可以validation文件是否正在写入…

[ec2-user@ip-10-82-48-59 ~]$ docker service logs echo-to-efs echo-to-efs.1.8c8lxqcoc3n5@ip-10-82-48-59.vpc.internal | writing from 3da21bc313f7 echo-to-efs.3.rmj163xu97ai@ip-10-82-48-96.vpc.internal | writing from 3da21bc313f7 echo-to-efs.1.8c8lxqcoc3n5@ip-10-82-48-59.vpc.internal | writing from 334b26ce3747 echo-to-efs.7.bzoafno0gb4d@ip-10-82-48-74.vpc.internal | writing from 3da21bc313f7 echo-to-efs.7.bzoafno0gb4d@ip-10-82-48-74.vpc.internal | writing from 334b26ce3747 echo-to-efs.3.rmj163xu97ai@ip-10-82-48-96.vpc.internal | writing from 334b26ce3747 echo-to-efs.1.8c8lxqcoc3n5@ip-10-82-48-59.vpc.internal | writing from fcefd6eb5035 echo-to-efs.1.8c8lxqcoc3n5@ip-10-82-48-59.vpc.internal | writing from 84ffd56fdcfa echo-to-efs.1.8c8lxqcoc3n5@ip-10-82-48-59.vpc.internal | writing from b7ecbd01debe echo-to-efs.1.8c8lxqcoc3n5@ip-10-82-48-59.vpc.internal | writing from e5ef2a2821a2 echo-to-efs.1.8c8lxqcoc3n5@ip-10-82-48-59.vpc.internal | writing from 5179f81f04f9 echo-to-efs.1.8c8lxqcoc3n5@ip-10-82-48-59.vpc.internal | writing from 5e859dce0c2f echo-to-efs.7.bzoafno0gb4d@ip-10-82-48-74.vpc.internal | writing from fcefd6eb5035 echo-to-efs.3.rmj163xu97ai@ip-10-82-48-96.vpc.internal | writing from fcefd6eb5035 echo-to-efs.7.bzoafno0gb4d@ip-10-82-48-74.vpc.internal | writing from 84ffd56fdcfa echo-to-efs.7.bzoafno0gb4d@ip-10-82-48-74.vpc.internal | writing from b7ecbd01debe echo-to-efs.3.rmj163xu97ai@ip-10-82-48-96.vpc.internal | writing from 84ffd56fdcfa echo-to-efs.3.rmj163xu97ai@ip-10-82-48-96.vpc.internal | writing from b7ecbd01debe echo-to-efs.3.rmj163xu97ai@ip-10-82-48-96.vpc.internal | writing from e5ef2a2821a2 echo-to-efs.4.5jtao2yu5udt@ip-10-82-48-95.vpc.internal | writing from 3da21bc313f7 echo-to-efs.3.rmj163xu97ai@ip-10-82-48-96.vpc.internal | writing from 5179f81f04f9 echo-to-efs.7.bzoafno0gb4d@ip-10-82-48-74.vpc.internal | writing from e5ef2a2821a2 echo-to-efs.3.rmj163xu97ai@ip-10-82-48-96.vpc.internal | writing from 5e859dce0c2f ... ...

  • 现在看到文件中的当前值。
    • 这里的命令会挂载这个卷,然后从EFS挂载中读取文件。
    • 它需要很长的时间,直到容器准备安装…像2分钟…不知道这是否是预期的。

[ec2-user@ip-10-82-48-59 ~]$ docker run -ti --volume-driver=rexray/efs -v echo_data:/test alpine cat /test/echo.log Unable to find image 'alpine:latest' locally latest: Pulling from library/alpine Digest: sha256:1072e499f3f655a032e88542330cf75b02e7bdf673278f701d7ba61629ee3ebe Status: Downloaded newer image for alpine:latest writing from 3da21bc313f7 writing from 334b26ce3747 writing from fcefd6eb5035 writing from 84ffd56fdcfa writing from b7ecbd01debe writing from e5ef2a2821a2 writing from 5179f81f04f9 writing from 5e859dce0c2f writing from af94a5a29b4a writing from a3f860caaf03 writing from b7b3d30d2f8e writing from cf65a52760c0 writing from a551b529e599 writing from 0533784ce9e5 writing from d78f781baa4d writing from 06b84229ec32 writing from 4e3974cfb8a9 writing from 007cd66062e7 writing from 665f76589ae2 writing from 68b04985559a writing from 9a2e19e110fb writing from 1ed7903abf86 writing from 763c7eea52fb writing from 529c7fff9c71 writing from 7172d5fb56d9 writing from c33294ce8d9d writing from b95a5b5f1bc1 writing from ff269d8a78ba writing from b1299af12f30 writing from 4f970a6e234d writing from a4a1f331b3c9 writing from bb3915d9d242 writing from 0fbafba12a0c writing from 017f70a769da writing from 8d657b23571f writing from 570e2fa76e16 writing from 45a92f3d19b4 writing from 3b02cb3675e4 writing from e2d49358df6f

Docker卷状态

  • Docker卷现在将所有的安装与插件映射
    • 这对我来说是一个重大胜利!

[ec2-user@ip-10-82-48-59 ~]$ docker volume ls DRIVER VOLUME NAME rexray/s3fs:latest 7820-3357-5600-us-west-2 rexray/s3fs:latest cf-templates-8hmsxuf2muaa-us-west-2 rexray/efs:latest echo_data rexray/s3fs:latest marcello-12345 rexray/s3fs:latest marcello-testing-bucket-1 rexray/s3fs:latest marcello-testing-bucket-2 rexray/s3fs:latest new-config-publisher rexray/s3fs:latest publisher.intuit-caas rexray/s3fs:latest publisher.intuit-caas-new-update rexray/ebs:latest quay-registry-configs rexray/ebs:latest quay-registry-images-datastore rexray/ebs:latest quay-registry-mysql-datastore rexray/ebs:latest registry_configs rexray/ebs:latest registry_mysql-data rexray/s3fs:latest swarmbucket-vpc-155e5072

从Swarm中的Docker容器访问EBS,EFS可以将它们挂载到Swarm节点(EC2实例)下面并作为卷装载到每个容器。

要访问S3,您需要使用具有所需权限的AWS开发工具包,CLI或REST API。 (对于访问权限,您可以使用由下面的SWARM节点承担的IAMangular色)。