资源pipe理器没有节点

编辑:我看了YARN Resourcemanager不连接到nodemanager和解决scheme不适合我。 我附加了连接到资源pipe理器的节点pipe理器日志部分:

[main] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at /0.0.0.0:8031 2016-06-17 19:01:04,697 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:getNMContainerStatuses(429)) - Sending out 0 NM container statuses: [] 2016-06-17 19:01:04,701 INFO [main] nodemanager.NodeStatusUpdaterImpl (NodeStatusUpdaterImpl.java:registerWithRM(268)) - Registering with RM using containers :[] 2016-06-17 19:01:05,815 INFO [main] ipc.Client (Client.java:handleConnectionFailure(867)) - Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS) 2016-06-17 19:01:06,816 INFO [main] ipc.Client (Client.java:handleConnectionFailure(867)) - Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS) 

由于某种原因,它说它连接到0.0.0.0。 当我ssh进入一个数据节点和ping资源pipe理器时,我得到一个响应,所以它能够parsing主机名。

这导致我相信,我的yarn-site.xml中的选项是不正确的,因为我的节点试图连接到0.0.0.0:8031,而不是资源pipe理器:8031


我正在Docker上运行Cloudera hadoop集群,并且遇到了Yarn资源pipe理器能够看到其他节点的问题。 它们的设置方式如下:

Node1 – Namenode(hadoop-hdfs-namenode)

节点2 – 次要名称节点(hadoop-hdfs-secondarynamenode)

节点3 – 纱线资源pipe理器(hadoop-yarn-resourcemanager)

节点4 – 数据节点和节点pipe理器(hadoop-hdfs-datanode,hadoop-yarn-nodemanager)

节点5 – 数据节点和节点pipe理器(hadoop-hdfs-datanode,hadoop-yarn-nodemanager)

当我去namenode:50070我能看到两个节点。 但是,当我去资源pipe理器:8088它显示我有零节点。 我在每个节点上的yarn-site.xml文件如下:

 <configuration> <property> <name>yarn.resourcemanager.address</name> <value>resource-manager:8032</value> </property> <property> <name>yarn.resourcemanager.scheduler.address</name> <value>resource-manager:8030</value> </property> <property> <description>Classpath for typical applications.</description> <name>yarn.application.classpath</name> <value> $HADOOP_CONF_DIR, $HADOOP_COMMON_HOME/*,$HADOOP_COMMON_HOME/lib/*, $HADOOP_HDFS_HOME/*,$HADOOP_HDFS_HOME/lib/*, $HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/lib/*, $HADOOP_YARN_HOME/*,$HADOOP_YARN_HOME/lib/* </value> </property> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.nodemanager.local-dirs</name> <value>file:///data/1/yarn/local,file:///data/2/yarn/local,file:///data/3/yarn/local</value> </property> <property> <name>yarn.nodemanager.log-dirs</name> <value>file:///data/1/yarn/logs,file:///data/2/yarn/logs,file:///data/3/yarn/logs</value> </property> <property> <name>yarn.log.aggregation-enable</name> <value>true</value> </property> <property> <description>Where to aggregate logs</description> <name>yarn.nodemanager.remote-app-log-dir</name> <value>hdfs://namenode:8020/var/log/hadoop-yarn/apps</value> </property> <property> <name>yarn.resourcemanager.webapp.address</name> <value>resource-manager:8088</value> </property> <property> <name>yarn.resourcemanager.resource-tracker.address</name> <value>resource-manager:8031</value> </property> <property> <name>yarn.resourcemanager.admin.address</name> <value>resource-manager:8033</value> </property> <property> <description> Number of seconds after an application finishes before the nodemanager's DeletionService will delete the application's localized file directory and log directory. To diagnose Yarn application problems, set this property's value large enough (for example, to 600 = 10 minutes) to permit examination of these directories. After changing the property's value, you must restart the nodemanager in order for it to have an effect. The roots of Yarn applications' work directories is configurable with the yarn.nodemanager.local-dirs property (see below), and the roots of the Yarn applications' log directories is configurable with the yarn.nodemanager.log-dirs property (see also below). </description> <name>yarn.nodemanager.delete.debug-delay-sec</name> <value>600</value> </property> </configuration> 

有没有人有任何想法,为什么这是事实?

谢谢阅读。

详细说明:

 <property> <name>yarn.resourcemanager.hostname</name> <value>master-1</value> </property> 

如编辑中所示,似乎yarn-site.xml没有被拾取,只有默认情况发生。 我解决了这个问题,将yarn-site.xml文件作为root用户拷贝到机器上的每个目录中。 然后运行节点pipe理器,使其读取文件时出错,因为它不在用户root下运行。 日志指引我到期望在纱线特定的目录而不是一般的hadoop目录的文件的位置。