芹菜在docker上不起作用
我有问题在docker工人使用芹菜。
我configuration了两个docker容器，web_server和celery_worker。 celery_worker包含rabbitmq-server。 web_server从芹菜工作者调用任务。
我用stream浪汉在VM中configuration了相同的东西。 它工作。 但是docker工人会说出如下的错误信息。
Traceback (most recent call last): File "/web_server/test/test_v1_data_description.py", line 58, in test_create_description headers=self.get_basic_header() ......... ......... File "../task_runner/__init__.py", line 31, in run_describe_task kwargs={}) File "/usr/local/lib/python3.4/dist-packages/celery/app/base.py", line 349, in send_task self.backend.on_task_call(P, task_id) File "/usr/local/lib/python3.4/dist-packages/celery/backends/rpc.py", line 32, in on_task_call maybe_declare(self.binding(producer.channel), retry=True) File "/usr/local/lib/python3.4/dist-packages/kombu/messaging.py", line 194, in _get_channel channel = self._channel = channel() File "/usr/local/lib/python3.4/dist-packages/kombu/utils/__init__.py", line 425, in __call__ value = self.__value__ = self.__contract__() File "/usr/local/lib/python3.4/dist-packages/kombu/messaging.py", line 209, in <lambda> channel = ChannelPromise(lambda: connection.default_channel) File "/usr/local/lib/python3.4/dist-packages/kombu/connection.py", line 756, in default_channel self.connection File "/usr/local/lib/python3.4/dist-packages/kombu/connection.py", line 741, in connection self._connection = self._establish_connection() File "/usr/local/lib/python3.4/dist-packages/kombu/connection.py", line 696, in _establish_connection conn = self.transport.establish_connection() File "/usr/local/lib/python3.4/dist-packages/kombu/transport/pyamqp.py", line 116, in establish_connection conn = self.Connection(**opts) File "/usr/local/lib/python3.4/dist-packages/amqp/connection.py", line 165, in __init__ self.transport = self.Transport(host, connect_timeout, ssl) File "/usr/local/lib/python3.4/dist-packages/amqp/connection.py", line 186, in Transport return create_transport(host, connect_timeout, ssl) File "/usr/local/lib/python3.4/dist-packages/amqp/transport.py", line 299, in create_transport return TCPTransport(host, connect_timeout) File "/usr/local/lib/python3.4/dist-packages/amqp/transport.py", line 95, in __init__ raise socket.error(last_err) nose.proxy.OSError: [Errno 111] Connection refused 这些是两个容器的Dockerfiles。
Dockerfile for web_server。
FROM ubuntu:14.04 MAINTAINER Jinho Yoo # Update packages. RUN apt-get clean RUN apt-get update # Create work folder. RUN mkdir /web_server WORKDIR /web_server # Setup web server and celery. ADD ./install_web_server_conf.sh ./install_web_server_conf.sh RUN chmod +x ./install_web_server_conf.sh RUN ./install_web_server_conf.sh #Reduce docker size. RUN rm -rf /var/lib/apt/lists/* # Run web server. CMD ["python3","web_server.py"] # Expose port. EXPOSE 5000
celery_worker的Dockerfile。
FROM ubuntu:14.04 MAINTAINER Jinho Yoo # Update packages. RUN apt-get clean RUN apt-get update RUN apt-get install -y wget build-essential ca-certificates-java # Setup python environment. ADD ./bootstrap/install_python_env.sh ./install_python_env.sh RUN chmod +x ./install_python_env.sh RUN ./install_python_env.sh # Install Python libraries including celery. RUN pip3 install -r ./core/requirements.txt # Add mlcore user for Celery worker RUN useradd --uid 1234 -M mlcore RUN usermod -L mlcore # Celery configuration for supervisor ADD celeryd.conf /etc/supervisor/conf.d/celeryd.conf RUN mkdir -p /var/log/celery # Reduce docker size. RUN rm -rf /var/lib/apt/lists/* # Run celery server by supervisor. CMD ["supervisord", "-c", "/ml_core/supervisord.conf"] # Expose port. EXPOSE 8080 EXPOSE 8081 EXPOSE 4040 EXPOSE 7070 EXPOSE 5672 EXPOSE 5671 EXPOSE 15672
Docker容器不能正常对话。 我的猜测是你的连接string是像localhost:<port> ？
有几种方法可以让你的容器能够沟通。
1：链接http://rominirani.com/2015/07/31/docker-tutorial-series-part-8-linking-containers/
本质上，在运行时，docker会在您的hosts文件中添加一个条目，指向同一个专用dockernetworking堆栈中的docker集装箱的内部IP地址。
2： docker run --net=host ：这个绑定你的容器到你的主机networking堆栈，因此，所有的容器看起来都是从localhost运行的，并且可以这样访问。 如果您正在运行绑定到相同外部端口的多个容器，则可能会遇到端口冲突问题，请注意这一点。
3：外部HAProxy：你可以将一个DNS条目绑定到一个HAProxy，并configuration代理redirectstream量，使用与DNS条目相匹配的主机头到主机：您的容器运行的端口，任何来自其他容器的调用将“离开“私人dockernetworking堆栈，打到DNS服务器，并回到HAProxy，这将直接到适当的容器。
我find了原因。 celery_worker中的Docker容器不运行rabbitmq-server 。 所以我在下面的celery_worker的celery_worker添加了两行。
# Run rabbitmq server and celery. ENTRYPOINT service rabbitmq-server start && supervisord -c /ml_core/supervisord.conf