Cloudera Docker的图像…一切都出错了
我正在用El Capitan OS运行16 GB Macbook Pro。 我使用了安装cloudera docker镜像
docker pull cloudera/quickstart:latest docker run --privileged=true --hostname=quickstart.cloudera -t -i 9f3ab06c7554 /usr/bin/docker-quickstart 图像启动良好，我可以看到大多数服务启动
Started Hadoop historyserver: [ OK ] starting nodemanager, logging to /var/log/hadoop-yarn/yarn-yarn-nodemanager-quickstart.cloudera.out Started Hadoop nodemanager: [ OK ] starting resourcemanager, logging to /var/log/hadoop-yarn/yarn-yarn-resourcemanager-quickstart.cloudera.out Started Hadoop resourcemanager: [ OK ] starting master, logging to /var/log/hbase/hbase-hbase-master-quickstart.cloudera.out Started HBase master daemon (hbase-master): [ OK ] starting rest, logging to /var/log/hbase/hbase-hbase-rest-quickstart.cloudera.out Started HBase rest daemon (hbase-rest): [ OK ] starting thrift, logging to /var/log/hbase/hbase-hbase-thrift-quickstart.cloudera.out Started HBase thrift daemon (hbase-thrift): [ OK ] Starting Hive Metastore (hive-metastore): [ OK ] Started Hive Server2 (hive-server2): [ OK ] Starting Sqoop Server: [ OK ] Sqoop home directory: /usr/lib/sqoop2
一些失败也是如此
Failure to start Spark history-server (spark-history-server[FAILED]n value: 1 Starting Hadoop HBase regionserver daemon: starting regionserver, logging to /var/log/hbase/hbase-hbase-regionserver-quickstart.cloudera.out hbase-regionserver. Starting hue: [FAILED]
但一旦启动完成，如果我尝试运行任何东西，它会失败
例如试图运行火花shell
[root@quickstart /]# spark-shell Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000b0000000, 357892096, 0) failed; error='Cannot allocate memory' (errno=12) # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (malloc) failed to allocate 357892096 bytes for committing reserved memory. # An error report file with more information is saved as: # //hs_err_pid3113.log
或者试图运行configuration单元的shell
[root@quickstart /]# hive Unable to determine Hadoop version information. 'hadoop version' returned: Hadoop 2.6.0-cdh5.5.0 Subversion http://github.com/cloudera/hadoop -r fd21232cef7b8c1f536965897ce20f50b83ee7b2 Compiled by jenkins on 2015-11-09T20:37Z Compiled with protoc 2.5.0 From source with checksum 98e07176d1787150a6a9c087627562c This command was run using /usr/jars/hadoop-common-2.6.0-cdh5.5.0.jar [root@quickstart /]#
我的问题是我能做些什么，我可以成功运行spark-shell和hive shell？
由于您在Mac上运行Docker，因此Docker在VirtualBox下运行，而不是直接在Mac内存上运行。 （同样的事情会发生在Windows中）。
你可能不会在Linux主机上得到这些错误，因为Docker没有在那里虚拟化。
Cloudera quickstart vmbuild议使用8Gb的内存来运行所有服务，而docker vm只有512Mb。
解决方法是停止docker-machine实例，打开VirtualBox，并将“默认”虚拟机的内存大小增加到必要的数量。