Tag: apache zookeeper

无法将远程Hazelcast客户端连接到在Docker容器中运行的群集

我有一个运行在Docker容器中的Hazelcast集群，使用动物园pipe理员的发现API。 这一切都正常工作，并启动群集，并按预期工作。 我的问题是连接客户端从另一台服务器的集群。 群集正在将127.0.0.1和172.17.0.1返回给zookeeper，因为它是群集地址，这意味着客户端可以在同一台机器上正常运行，但即使将172.17.0.1映射到客户端主机文件中，也不会从远程计算机连接到Hz集群的服务器ip。 我试着用net = host和-h启动容器，让它返回一个地址，我可以在客户端的主机文件中映射，但似乎没有任何工作。 我错过了什么吗？ 以下是来自客户端的日志和堆栈跟踪 /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/bin/java -Didea.launcher.port=7535 "-Didea.launcher.bin.path=/Applications/IntelliJ IDEA.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath "/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/JObjC.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/htmlconverter.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/lib/javafx-doclet.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/lib/tools.jar:/Users/oliverbuckley-salmon/IdeaProjects/sandpit/datagrid/target/classes:/Users/oliverbuckley-salmon/IdeaProjects/sandpit/domain/target/classes:/Users/oliverbuckley-salmon/.m2/repository/com/google/code/gson/gson/2.6.2/gson-2.6.2.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/hbase/hbase-client/1.2.3/hbase-client-1.2.3.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/hbase/hbase-annotations/1.2.3/hbase-annotations-1.2.3.jar:/Users/oliverbuckley-salmon/.m2/repository/com/github/stephenc/findbugs/findbugs-annotations/1.3.9-1/findbugs-annotations-1.3.9-1.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/hbase/hbase-common/1.2.3/hbase-common-1.2.3.jar:/Users/oliverbuckley-salmon/.m2/repository/commons-collections/commons-collections/3.2.2/commons-collections-3.2.2.jar:/Users/oliverbuckley-salmon/.m2/repository/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/hbase/hbase-protocol/1.2.3/hbase-protocol-1.2.3.jar:/Users/oliverbuckley-salmon/.m2/repository/commons-codec/commons-codec/1.9/commons-codec-1.9.jar:/Users/oliverbuckley-salmon/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar:/Users/oliverbuckley-salmon/.m2/repository/commons-lang/commons-lang/2.6/commons-lang-2.6.jar:/Users/oliverbuckley-salmon/.m2/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar:/Users/oliverbuckley-salmon/.m2/repository/com/google/guava/guava/12.0.1/guava-12.0.1.jar:/Users/oliverbuckley-salmon/.m2/repository/com/google/protobuf/protobuf-java/2.5.0/protobuf-java-2.5.0.jar:/Users/oliverbuckley-salmon/.m2/repository/io/netty/netty-all/4.0.23.Final/netty-all-4.0.23.Final.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/zookeeper/zookeeper/3.4.6/zookeeper-3.4.6.jar:/Users/oliverbuckley-salmon/.m2/repository/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar:/Users/oliverbuckley-salmon/.m2/repository/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/htrace/htrace-core/3.1.0-incubating/htrace-core-3.1.0-incubating.jar:/Users/oliverbuckley-salmon/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.9.13/jackson-mapper-asl-1.9.13.jar:/Users/oliverbuckley-salmon/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.9.13/jackson-core-asl-1.9.13.jar:/Users/oliverbuckley-salmon/.m2/repository/org/jruby/jcodings/jcodings/1.0.8/jcodings-1.0.8.jar:/Users/oliverbuckley-salmon/.m2/repository/org/jruby/joni/joni/2.1.2/joni-2.1.2.jar:/Users/oliverbuckley-salmon/.m2/repository/com/yammer/metrics/metrics-core/2.2.0/metrics-core-2.2.0.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/hadoop/hadoop-auth/2.5.1/hadoop-auth-2.5.1.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/httpcomponents/httpclient/4.2.5/httpclient-4.2.5.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/httpcomponents/httpcore/4.2.4/httpcore-4.2.4.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/directory/server/apacheds-kerberos-codec/2.0.0-M15/apacheds-kerberos-codec-2.0.0-M15.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/directory/server/apacheds-i18n/2.0.0-M15/apacheds-i18n-2.0.0-M15.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/directory/api/api-asn1-api/1.0.0-M20/api-asn1-api-1.0.0-M20.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/directory/api/api-util/1.0.0-M20/api-util-1.0.0-M20.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/hadoop/hadoop-common/2.5.1/hadoop-common-2.5.1.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/hadoop/hadoop-annotations/2.5.1/hadoop-annotations-2.5.1.jar:/Users/oliverbuckley-salmon/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/commons/commons-math3/3.1.1/commons-math3-3.1.1.jar:/Users/oliverbuckley-salmon/.m2/repository/xmlenc/xmlenc/0.52/xmlenc-0.52.jar:/Users/oliverbuckley-salmon/.m2/repository/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar:/Users/oliverbuckley-salmon/.m2/repository/commons-net/commons-net/3.1/commons-net-3.1.jar:/Users/oliverbuckley-salmon/.m2/repository/commons-el/commons-el/1.0/commons-el-1.0.jar:/Users/oliverbuckley-salmon/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/Users/oliverbuckley-salmon/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar:/Users/oliverbuckley-salmon/.m2/repository/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar:/Users/oliverbuckley-salmon/.m2/repository/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/avro/avro/1.7.4/avro-1.7.4.jar:/Users/oliverbuckley-salmon/.m2/repository/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar:/Users/oliverbuckley-salmon/.m2/repository/org/xerial/snappy/snappy-java/1.0.4.1/snappy-java-1.0.4.1.jar:/Users/oliverbuckley-salmon/.m2/repository/com/jcraft/jsch/0.1.42/jsch-0.1.42.jar:/Users/oliverbuckley-salmon/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/commons/commons-compress/1.4.1/commons-compress-1.4.1.jar:/Users/oliverbuckley-salmon/.m2/repository/org/tukaani/xz/1.0/xz-1.0.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/2.5.1/hadoop-mapreduce-client-core-2.5.1.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/hadoop/hadoop-yarn-common/2.5.1/hadoop-yarn-common-2.5.1.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/hadoop/hadoop-yarn-api/2.5.1/hadoop-yarn-api-2.5.1.jar:/Users/oliverbuckley-salmon/.m2/repository/javax/xml/bind/jaxb-api/2.2.2/jaxb-api-2.2.2.jar:/Users/oliverbuckley-salmon/.m2/repository/javax/xml/stream/stax-api/1.0-2/stax-api-1.0-2.jar:/Users/oliverbuckley-salmon/.m2/repository/javax/activation/activation/1.1/activation-1.1.jar:/Users/oliverbuckley-salmon/.m2/repository/io/netty/netty/3.6.2.Final/netty-3.6.2.Final.jar:/Users/oliverbuckley-salmon/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/Users/oliverbuckley-salmon/.m2/repository/com/hazelcast/hazelcast-all/3.7.2/hazelcast-all-3.7.2.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/curator/curator-framework/2.10.0/curator-framework-2.10.0.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/curator/curator-client/2.10.0/curator-client-2.10.0.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/curator/curator-x-discovery/2.10.0/curator-x-discovery-2.10.0.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/curator/curator-recipes/2.10.0/curator-recipes-2.10.0.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/curator/curator-test/2.10.0/curator-test-2.10.0.jar:/Users/oliverbuckley-salmon/.m2/repository/org/javassist/javassist/3.18.1-GA/javassist-3.18.1-GA.jar:/Users/oliverbuckley-salmon/.m2/repository/org/apache/commons/commons-math/2.2/commons-math-2.2.jar:/Users/oliverbuckley-salmon/.m2/repository/com/hazelcast/hazelcast-zookeeper/3.6.1/hazelcast-zookeeper-3.6.1.jar:/Users/oliverbuckley-salmon/.m2/repository/joda-time/joda-time/2.9.4/joda-time-2.9.4.jar:/Applications/IntelliJ IDEA.app/Contents/lib/idea_rt.jar" com.intellij.rt.execution.application.AppMain com.example.datagrid.cachewarmer.CacheReader 2016-11-28 20:51:35 INFO TradeMapStore:64 – Trying to connect to HBase 2016-11-28 20:51:35 WARN NativeCodeLoader:62 – Unable to load native-hadoop library for your platform… using builtin-java classes where applicable 2016-11-28 20:51:37.473 java[25412:2402206] Unable to load realm info […]

Kubernetes多节点上的Kafka

所以我的目标是以分散的方式build立几个卡夫卡经纪人集群。 但我看不出让经纪人意识到对方的方法。 据我了解，每个经纪人都需要在他们的configuration中有一个单独的ID，如果我从kubernetes启动容器，我不能保证或configuration。 他们还需要有相同的advertised_host？ 有什么参数我错过了，需要改变的节点发现彼此？ 在Dockerfile的末尾用脚本做这样的configuration是否可行？ 和/或共享音量？ 我目前正在尝试在香草Kubernetes上使用具有预configuration的zookeeper + kafka组合的spotify / kafka-image。

运行docker映像时运行时错误“oci运行时错误

我有下面的动物园pipe理员安装Dockerfile，我已经build立了docker的形象。 FROM openjdk:8-jre-alpine # Install required packages RUN apk add –no-cache \ bash \ su-exec ENV ZOO_USER=zookeeper \ ZOO_CONF_DIR=/conf \ ZOO_DATA_DIR=/data \ ZOO_DATA_LOG_DIR=/datalog \ ZOO_PORT=2181 \ ZOO_TICK_TIME=2000 \ ZOO_INIT_LIMIT=5 \ ZOO_SYNC_LIMIT=2 # Add a user and make dirs RUN set -x \ && adduser -D "$ZOO_USER" \ && mkdir -p "$ZOO_DATA_LOG_DIR" "$ZOO_DATA_DIR" "$ZOO_CONF_DIR" \ && […]

如何在Docker群上设置zookeeper群集

环境 ：6个服务器docker swarm集群（2个master和4个worker） 要求 ：我们需要在现有的docker群上设置zookeeper群集。 阻止 ：要在群集中设置zookeeper，我们需要在每个服务器configuration中提供所有zk服务器，并在myid文件中提供唯一的ID。 问题 ：当我们在docker swarm中创buildzookeeper的副本时，我们如何为每个副本提供唯一的ID。 另外我们如何更新每个zookeeper容器的ID的zoo.cfgconfiguration文件。

如何安排监控CPU，内存，磁盘等工作

我的问题是我有一个专用的服务器，但资源仍然有限，即IO，内存，CPU等，我需要每天运行大量的工作。 一些工作是密集型的，有些工作是计算密集型的。 有没有办法来监控当前的状态，并决定何时从我的工作池开始一个新的工作。 例如，当它知道当前正在运行的工作是密集型的时，它可以在一个不能继续工作的工作中吃饭。 或者它可以select一个正在使用大量磁盘io的正在运行的作业，停止它，稍后重新安排它。 我提出了docker的解决scheme，因为它可以监视进程，但是我不知道在Docker之上构build这样的调度器。 谢谢