为什么Docker Compose重新启动可以渲染容器输出越来越多?

我正在编写一个基于Docker Compose的Web应用程序,它有许多后台系统 – 一个HTTP API,一个HTTP代理和一个队列。 所有东西都在PHP的Alpine容器中,PHP 5.6或7.0。

我最初在API容器内的Supervisor中设置了队列,工作正常。 然而,Supervisor / Python使得容器比他们应该更加胖(80M而不是25M),所以我把队列移到了自己的容器中。 它生活了大约5分钟,为了重新启动而退出,我在Supervisor中使用了自动重启系统,所以我已经在Docker Compose中交换到了重启系统。 我正在使用Compose YAML格式的版本2。

当队列启动时,它会向stdout呈现一个简单的消息:

queue_instance | Starting queue watcher (path=/remote/queue, proxying to proximate-proxy:8081) 

当我最初做docker-compose up时,这很好。 但是,对于每次重新启动,我都会收到其中三条消息,然后是五条消息,以此类推,没有限制。 如果我做docker ps然后它表明只有一个队列运行:

 halfer@machine:~/proximate-app$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES a9c94558769d proximate-app "/tmp/container-st..." 2 hours ago Up 2 hours 0.0.0.0:8084->8084/tcp app_instance 7e48d6aec459 proximate-api "sh /tmp/bin/web-s..." 2 hours ago Up 2 hours 0.0.0.0:8080->8080/tcp api_instance 86c564becadf proximate-queue "sh /var/app/bin/c..." 2 hours ago Up About a minute queue_instance 20c2145f80e4 proximate-proxy "sh /var/proxy/con..." 2 hours ago Up 2 hours 0.0.0.0:8081->8081/tcp proxy_instance 

这是我的撰写文件:

 version: '2' services: proximate-app: container_name: "app_instance" image: proximate-app ports: - "8084:8084" links: - proximate-api # @todo Remove external ports once everything is happy proximate-api: container_name: "api_instance" image: proximate-api ports: - "8080:8080" links: - proximate-proxy - proximate-queue volumes: - proximate-volume:/remote/cache - proximate-q-volume:/remote/queue # Use time and TZ from the host, could alternatively use env vars and set it # manually in the container, see https://wiki.alpinelinux.org/wiki/Setting_the_timezone - /etc/localtime:/etc/localtime:ro - /etc/timezone:/etc/timezone:ro # Should perhaps pass this as a var to docker-compose so as not to hardwire it, # but it is fine for now environment: - PHP_TIMEZONE=Europe/London proximate-queue: container_name: "queue_instance" image: proximate-queue restart: always links: - proximate-proxy volumes: - proximate-volume:/remote/cache - proximate-q-volume:/remote/queue environment: - PROXY_ADDRESS=proximate-proxy:8081 # @todo Remove external ports once everything is happy proximate-proxy: container_name: "proxy_instance" image: proximate-proxy ports: - "8081:8081" volumes: - proximate-volume:/remote/cache environment: - PROXY_LOG_PATH=/remote/cache/proxy.log volumes: proximate-volume: proximate-q-volume: 

相关的容器是proximate-queue

我很确定,我的容器本身并不是这个怪事的责任。 我的Dockerfile进入如下:

 ENTRYPOINT ["sh", "/var/app/bin/container-start.sh"] 

这只是一个启动脚本:

 #!/bin/sh php \ /var/app/bin/queue.php \ --queue-path /remote/queue \ --proxy-address ${PROXY_ADDRESS} 

哪个运行队列进程:

 #!/usr/bin/env php <?php use Proximate\Service\File; use Proximate\Service\SiteFetcher as SiteFetcherService; use Proximate\Queue\Read as QueueReader; $root = realpath(__DIR__ . '/..'); require_once $root . '/vendor/autoload.php'; $actions = getopt('p:q:', ['proxy-address:', 'queue-path:']); $queuePath = isset($actions['queue-path']) ? $actions['queue-path'] : (isset($actions['q']) ? $actions['q'] : null); $proxyAddress = isset($actions['proxy-address']) ? $actions['proxy-address'] : (isset($actions['p']) ? $actions['p'] : null); if (!$queuePath || !$proxyAddress) { $command = __FILE__; die( sprintf("Syntax: %s --proxy-address <proxy:port> --queue-path <queue-path>\n", $command) ); } if (!file_exists($queuePath)) { die( sprintf("Error: the supplied queue path `%s` does not exist\n", $queuePath) ); } echo sprintf( "Starting queue watcher (path=%s, proxying to %s)\n", $queuePath, $proxyAddress ); $queue = new QueueReader($queuePath, new File()); $queue-> setFetcher(new SiteFetcherService($proxyAddress))-> process(); 

正如你所看到的, echo sprintf()是宣布启动的东西,没有什么可以做到这一点的循环。 这可能是Docker Compose中的一个错误吗? 我使用docker-compose version 1.9.0, build 2585387在Ubuntu 14.04上docker-compose version 1.9.0, build 2585387

作为参考,Docker Compose标准输出看起来像这样(队列中重复的行是可见的):

 halfer@machine:~/proximate-app$ docker-compose up Creating network "proximateapp_default" with the default driver Creating proxy_instance Creating queue_instance Creating api_instance Creating app_instance Attaching to proxy_instance, queue_instance, api_instance, app_instance proxy_instance | Teeing proxy logs also to /remote/cache/proxy.log proxy_instance | [2017-05-10 09:18:42] stdout.INFO: Setting up queue at `/remote/cache/data` [] [] proxy_instance | [2017-05-10 09:18:42] stdout.INFO: Starting proxy listener on 172.18.0.2:8081 [] [] queue_instance | Starting queue watcher (path=/remote/queue, proxying to proximate-proxy:8081) api_instance | PHP 7.0.16 Development Server started at Wed May 10 10:19:00 2017 app_instance | PHP 5.6.29 Development Server started at Wed May 10 09:19:10 2017 app_instance | PHP 5.6.29 Development Server started at Wed May 10 09:19:10 2017 queue_instance exited with code 0 queue_instance | Starting queue watcher (path=/remote/queue, proxying to proximate-proxy:8081) queue_instance | Starting queue watcher (path=/remote/queue, proxying to proximate-proxy:8081) queue_instance | Starting queue watcher (path=/remote/queue, proxying to proximate-proxy:8081) queue_instance exited with code 0 queue_instance | Starting queue watcher (path=/remote/queue, proxying to proximate-proxy:8081) queue_instance | Starting queue watcher (path=/remote/queue, proxying to proximate-proxy:8081) queue_instance | Starting queue watcher (path=/remote/queue, proxying to proximate-proxy:8081) queue_instance | Starting queue watcher (path=/remote/queue, proxying to proximate-proxy:8081) queue_instance | Starting queue watcher (path=/remote/queue, proxying to proximate-proxy:8081) 

我可以尝试的一件事就是让应用程序进入睡眠状态,不要做任何其他事情,以certificate某些奇怪的退出处理程序或某些东西不会造成严重破坏。 但是,我期望这将做同样的事情。

更新

我已经用打印时间信息的脚本replace了队列,然后睡了20秒。 这是我得到的:

 halfer@machine:~/proximate-app$ docker-compose up Creating network "proximateapp_default" with the default driver Creating proxy_instance Creating queue_instance Creating api_instance Creating app_instance Attaching to proxy_instance, queue_instance, api_instance, app_instance proxy_instance | Teeing proxy logs also to /remote/cache/proxy.log proxy_instance | [2017-05-10 11:51:17] stdout.INFO: Setting up queue at `/remote/cache/data` [] [] proxy_instance | [2017-05-10 11:51:17] stdout.INFO: Starting proxy listener on 172.18.0.2:8081 [] [] queue_instance | Hello everyone! Time=Wed, 10 May 2017 11:51:27 +0000. Microtime=1494417087.107185 api_instance | PHP 7.0.16 Development Server started at Wed May 10 12:51:37 2017 app_instance | PHP 5.6.29 Development Server started at Wed May 10 11:51:46 2017 app_instance | PHP 5.6.29 Development Server started at Wed May 10 11:51:46 2017 queue_instance exited with code 0 queue_instance | Hello everyone! Time=Wed, 10 May 2017 11:51:27 +0000. Microtime=1494417087.107185 queue_instance | Hello everyone! Time=Wed, 10 May 2017 11:51:55 +0000. Microtime=1494417115.178871 queue_instance | Hello everyone! Time=Wed, 10 May 2017 11:52:22 +0000. Microtime=1494417142.409513 queue_instance exited with code 0 queue_instance | Hello everyone! Time=Wed, 10 May 2017 11:51:27 +0000. Microtime=1494417087.107185 queue_instance | Hello everyone! Time=Wed, 10 May 2017 11:51:55 +0000. Microtime=1494417115.178871 queue_instance | Hello everyone! Time=Wed, 10 May 2017 11:52:22 +0000. Microtime=1494417142.409513 queue_instance | Hello everyone! Time=Wed, 10 May 2017 11:52:49 +0000. Microtime=1494417169.612523 queue_instance | Hello everyone! Time=Wed, 10 May 2017 11:53:17 +0000. Microtime=1494417197.826749 

这似乎表明:

  • 重新启动只报告每隔一次重新启动
  • 我的任务完全需要20秒,但是Compose却很慢地重启(接近每30秒)。 这并不会让我感到困扰,但可能是一个有用的观察
  • 线路正在重复,实际上是重新启动报告。

到底是怎么回事?

使用Docker重启function只是为了重新启动应该是一个服务(长时间运行的队列任务)是没有必要的,因为操作系统带有你需要的一切。

在使用docker restartfunction之前,您重新启动了自己的队列。 但是你写给你的是主pipe/ Python,因为你的眼睛太胖(〜55M)。

所以我读了它,你想保持足迹低。

我们来做吧 所以我的build议是,从最低限度的操作系统开始。 它应该拥有你的队列进程pipe理所需的所有东西,因为这对于操作系统是很常见的:调用和控制其他进程(保持列表简洁)。

对于使用Busybox的Alpine Linux PHP图像,减less了一组命令和shell(例如,没有Bash)也是如此。

所以我假设你想每5分钟踢一次PHP脚本,然后在我的例子中再次启动它(没有超时也是可能的,即使代码较less)。

Docker镜像附带了执行此操作的超时命令。 您可以在Dockerfile入口点sh脚本中使用它。 所以PHP被调用并在300秒(五分钟)后终止:

 # executed by sh command from BusyBox v1.24.2 (2017-01-18 14:13:46 GMT) multi-call binary # # usage: ENTRYPOINT ["sh", "docker-entrypoint.sh"] set -eu timed_queue() { set +e timeout \ -t 300 \ php -d max_execution_time=301 \ -f /queue.php -- \ --proxy-address "${PROXY_ADDRESS}" \ --queue-path /remote/queue \ ; retval=$? set -e } timed_queue while [ $retval -eq 143 -o $retval -eq 0 ]; do timed_queue done 

使用-t 300命令行参数进行设置的设置为timeout 。 在超时启动的情况下,退出状态默认为143(SIGTERM)

如果队列脚本成功结束,则返回0。 在这两种情况下,脚本再次启动(超时)。

这需要对您的队列脚本进行额外的小改动。 您可以通过以下方式使用死命令来指示错误:

 die( sprintf("Error: the supplied queue path `%s` does not exist\n", $queuePath) ); 

相反,将错误消息输出到标准错误输出,并以非零状态代码(例如1)退出:

 fprintf(STDERR, "Error: the supplied queue path `%s` does not exist\n", $queuePath); die(1); 

这将使脚本与操作系统兼容。 在命令行中,零(0)的退出状态表示OK(即使用带string消息的die("string") ,但是非零代码显示错误(对于初学者使用1)。

做到这一点,您将获得重新启动和超时脚本的function,几乎没有开销。 这既是文件大小,也是处理时间。 我已经把这个testing的超时时间为1秒,并且在重启时只显示几微秒的开销:

 queue_instance | info: 2017-05-19T14:00:24.34824700 queue.php started... queue_instance | info: 2017-05-19T14:00:25.35548200 queue.php started... queue_instance | info: 2017-05-19T14:00:26.34564400 queue.php started... queue_instance | info: 2017-05-19T14:00:27.35868700 queue.php started... queue_instance | info: 2017-05-19T14:00:28.34597300 queue.php started... queue_instance | info: 2017-05-19T14:00:29.34139800 queue.php started... queue_instance | info: 2017-05-19T14:00:30.26049500 queue.php started... queue_instance | info: 2017-05-19T14:00:31.26174500 queue.php started... queue_instance | info: 2017-05-19T14:00:32.26322800 queue.php started... queue_instance | info: 2017-05-19T14:00:33.26352800 queue.php started... queue_instance | info: 2017-05-19T14:00:34.26533300 queue.php started... queue_instance | info: 2017-05-19T14:00:35.26524300 queue.php started... queue_instance | info: 2017-05-19T14:00:36.26743300 queue.php started... queue_instance | info: 2017-05-19T14:00:37.26889500 queue.php started... queue_instance | info: 2017-05-19T14:00:38.27222300 queue.php started... queue_instance | info: 2017-05-19T14:00:39.27209000 queue.php started... queue_instance | info: 2017-05-19T14:00:40.27620500 queue.php started... queue_instance | info: 2017-05-19T14:00:41.27985300 queue.php started... queue_instance | info: 2017-05-19T14:00:42.28136100 queue.php started... queue_instance | info: 2017-05-19T14:00:43.28252200 queue.php started... queue_instance | info: 2017-05-19T14:00:44.28403600 queue.php started... queue_instance | info: 2017-05-19T14:00:45.28595300 queue.php started... queue_instance | info: 2017-05-19T14:00:46.28683900 queue.php started... queue_instance | info: 2017-05-19T14:00:47.28803800 queue.php started... 

如果您想在五分钟内运行该脚本,则在您的scheme中可能会忽略细微的差异。

如果进程崩溃(致命错误等)PHP将有一个127退出码,所以你会得到通知容器closures时,然后。 如果Dockerfile入口点脚本中存在错误,就像未定义的环境variables一样。

所以如果我理解你的话,这应该是你要找的。

根据通过CMDENTRYPOINT最终调用的ENTRYPOINT ,可能会应用更多的stream程处理。 或更多的处理可能会进一步申请 一些参考:

  • docker工人的替代supervisord
  • Docker的dumb-init有多重要?
  • su-exec(普通c) , gosu(在go,MBs more)
  • 运行部件(Busybox)

我总结了我们发现的。

累积的输出是由于容器在每次退出时没有被移除。 所以,docker-compose再次启动,显示docker logs -f来自同一个重新开始的容器。 这是按预期工作。

我没有find一个方法来获得docker组成 – 删除一个容器,然后再次启动它同一个docker-compose up

我喜欢你的基于队列的体系结构,但是我认为如果你使用一个长寿命的容器而不是一个容器来启动/退出/启动/退出会更好。你可以用一个简单的方法来完成它while true; do ./queue-app-blabla.php; sleep 20; done while true; do ./queue-app-blabla.php; sleep 20; done 在shell脚本中完成。

任何需要重现的人都可以使用下面的docker-compose.yml

 version: "2" services: ui: image: ubuntu command: tail -f /dev/null links: - queue queue: image: ubuntu command: sh -c "date; echo Sleeping...; sleep 20; echo exit." restart: always 

输出:

 queue_1 | Mon May 15 04:32:12 UTC 2017 queue_1 | Sleeping... queue_1 | exit. docker_queue_1 exited with code 0 queue_1 | Mon May 15 04:32:12 UTC 2017 queue_1 | Sleeping... queue_1 | exit. queue_1 | Mon May 15 04:32:33 UTC 2017 queue_1 | Sleeping... queue_1 | exit. queue_1 | Mon May 15 04:32:54 UTC 2017 queue_1 | Sleeping... queue_1 | exit. docker_queue_1 exited with code 0 queue_1 | Mon May 15 04:32:12 UTC 2017 queue_1 | Sleeping... queue_1 | exit. queue_1 | Mon May 15 04:32:33 UTC 2017 queue_1 | Sleeping... queue_1 | exit. queue_1 | Mon May 15 04:32:54 UTC 2017 queue_1 | Sleeping... queue_1 | exit. queue_1 | Mon May 15 04:33:17 UTC 2017 queue_1 | Sleeping... queue_1 | exit. queue_1 | Mon May 15 04:33:38 UTC 2017 queue_1 | Sleeping... queue_1 | exit. 

我build议的解决scheme

 version: "2" services: ui: image: ubuntu command: tail -f /dev/null links: - queue queue: image: ubuntu command: sh -c "while true; do date; echo Sleeping...; sleep 20; echo loop.; done" restart: always 

输出:

 Attaching to docker_queue_1, docker_ui_1 queue_1 | Mon May 15 04:36:16 UTC 2017 queue_1 | Sleeping... queue_1 | loop. queue_1 | Mon May 15 04:36:36 UTC 2017 queue_1 | Sleeping... queue_1 | loop. queue_1 | Mon May 15 04:36:56 UTC 2017 queue_1 | Sleeping... queue_1 | loop. queue_1 | Mon May 15 04:37:16 UTC 2017 queue_1 | Sleeping... 

国际海事组织,最好看到集装箱启动和工作,而不是拍打开始/出口集装箱。 对于具有这种行为的容器,Docker有一个特殊的对待,请参阅docker文档 。 在每次重新启动之前,都会增加一个不断增加的延迟(前一个延迟的两倍,从100毫秒开始),以防止服务器泛滥。 (但它不适用于你的情况,因为你的容器大于10秒钟。


编辑:

检查可能的错误并退出。 这可能是有用的:

 #!/bin/bash PERIOD=10 # seconds while true; do echo Starting process... date # the process STATUS=$? if [ $STATUS -ne 0 ]; then echo "Error found. Exiting" >&2 exit $STATUS fi echo End process. Sleeping ${PERIOD}s sleep $PERIOD done 

输出:

 ▶ ./script.sh Starting process... Mon May 15 11:43:06 ART 2017 End process. Sleeping 10s Starting process... Mon May 15 11:43:16 ART 2017 End process. Sleeping 10s Starting process... Mon May 15 11:43:26 ART 2017 End process. Sleeping 10s Starting process... Mon May 15 11:43:36 ART 2017 End process. Sleeping 10s Starting process... Mon May 15 11:43:46 ART 2017 End process. Sleeping 10s