卡夫卡docker工人 – 不能从docker集装箱外生产或消费
卡夫卡在docker集装箱工作正常。 我可以使用docker exec -it [container name] [kafkascript]并成功创build主题，生成/使用消息，但是当我使用本地kafka脚本从Docker容器之外进行尝试时，我只能创build和列出主题。 生成和使用消息会引发错误：
制作 ：
~/development/lib/kafka/kafka_2.11-0.10.0.0$ bin/kafka-console-producer.sh --broker-list $(docker-machine ip kafka):9092 --topic test asdf
[2016-09-18 10:13:48,999] ERROR Error when sending message to topic test with key: null, value: 4 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.errors.TimeoutException: Batch containing 1 record(s) expired due to timeout while requesting metadata from brokers for test-0
消费：
~/development/lib/kafka/kafka_2.11-0.10.0.0$ bin/kafka-console-consumer.sh --zookeeper $(docker-machine ip kafka):2181 --topic test --from-beginning
[2016-09-18 09:57:10,389] WARN Fetching topic metadata with correlation id 0 for topics [Set(test)] from broker [BrokerEndPoint(0,ba762186182f,9092)] failed (kafka.client.ClientUtils$) java.nio.channels.ClosedChannelException at kafka.network.BlockingChannel.send(BlockingChannel.scala:110) at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:80) at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:79) at kafka.producer.SyncProducer.send(SyncProducer.scala:124) at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59) at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:94) at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63) [2016-09-18 09:57:10,392] WARN [console-consumer-34526_3c15c2c24040-1474210630122-9404562b-leader-finder-thread], Failed to find leader for Set([test,0]) (kafka.consumer.ConsumerFetcherManager$LeaderFinderThread) kafka.common.KafkaException: fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(BrokerEndPoint(0,ba762186182f,9092))] failed at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:73) at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:94) at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:66) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63) Caused by: java.nio.channels.ClosedChannelException at kafka.network.BlockingChannel.send(BlockingChannel.scala:110) at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:80) at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:79) at kafka.producer.SyncProducer.send(SyncProducer.scala:124) at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59) ... 3 more
我使用的是spotify / docker-kafka，但我将其升级到了0.10.0.0，并使用了jshark的一些build议来设置advertised.listeners。 我在Mac上运行。 我创build了一个名为kafka的docker机器。 这是我的docker运行：
docker run -p 2181:2181 -p 9092:9092 --env ADVERTISED_HOST=`docker-machine ip kafka` --env ADVERTISED_PORT=9092 kafka
这是我的dockerfile：
# Kafka and Zookeeper FROM java:openjdk-8-jre ENV DEBIAN_FRONTEND noninteractive ENV SCALA_VERSION 2.11 ENV KAFKA_VERSION 0.10.0.0 ENV KAFKA_HOME /opt/kafka_"$SCALA_VERSION"-"$KAFKA_VERSION" # Install Kafka, Zookeeper and other needed things RUN apt-get update && \ apt-get install -y zookeeper wget supervisor dnsutils && \ rm -rf /var/lib/apt/lists/* && \ apt-get clean && \ wget -q http://apache.mirrors.spacedump.net/kafka/"$KAFKA_VERSION"/kafka_"$SCALA_VERSION"-"$KAFKA_VERSION".tgz -O /tmp/kafka_"$SCALA_VERSION"-"$KAFKA_VERSION".tgz && \ tar xfz /tmp/kafka_"$SCALA_VERSION"-"$KAFKA_VERSION".tgz -C /opt && \ rm /tmp/kafka_"$SCALA_VERSION"-"$KAFKA_VERSION".tgz ADD scripts/start-kafka.sh /usr/bin/start-kafka.sh # Supervisor config ADD supervisor/kafka.conf supervisor/zookeeper.conf /etc/supervisor/conf.d/ # 2181 is zookeeper, 9092 is kafka EXPOSE 2181 EXPOSE 9092 CMD ["supervisord", "-n"]
脚本/ start-kafka.sh
这对我工作： https : //stackoverflow.com/a/37655203/1839580
我的总结：spotify / kafka容器中的ADVERTISED_HOST环境variables需要根据服务在容器内部还是外部运行而改变。 我使用Mac的Docker，我有我的dockernetworking设置桥接。 在Docker之外，需要将ADVERTISED_HOST设置为docker内的localhost，将其设置为myproject_kafka_1，或者将其设置为系统中的任何内容。 为了解决这个问题，我在我的MacOS主机文件中添加并input了将127.0.0.1映射到myproject_kafka_1的文件。 我不喜欢弄乱我的主机文件，但它为我解决了这个问题。
127.0.0.1 localhost 127.0.0.1 myproject_kafka_1