neo4j-mazerunner，如何增加docker-compose.yml中的内存大小
在MacBook Pro（16GB内存）上使用docker-compose创buildkbastani / spark-neo4j ，我试图分析我的图的strongly_connected_components 。
我有一个约60,000个节点(n1:Node {id:1})-[r:NEXT {count:100}]->(n2:Node {id:2})的graphics。
使用neo4j浏览器，我已经设法将pagerank处理回我的节点。
但是，当我尝试运行像strong_connected_components这样更复杂的algorithm时，出现以下错误：
mazerunner_1 | 16/11/29 14:58:01 ERROR Utils: Uncaught exception in thread SparkListenerBus mazerunner_1 | java.lang.OutOfMemoryError: Java heap space mazerunner_1 | at org.apache.spark.ui.jobs.JobProgressListener$$anonfun$onJobStart$5$$anonfun$apply$9.apply(JobProgressListener.scala:200) mazerunner_1 | at org.apache.spark.ui.jobs.JobProgressListener$$anonfun$onJobStart$5$$anonfun$apply$9.apply(JobProgressListener.scala:200) mazerunner_1 | at scala.collection.mutable.MapLike$class.getOrElseUpdate(MapLike.scala:189) mazerunner_1 | at scala.collection.mutable.AbstractMap.getOrElseUpdate(Map.scala:91) mazerunner_1 | at org.apache.spark.ui.jobs.JobProgressListener$$anonfun$onJobStart$5.apply(JobProgressListener.scala:200) mazerunner_1 | at org.apache.spark.ui.jobs.JobProgressListener$$anonfun$onJobStart$5.apply(JobProgressListener.scala:198) mazerunner_1 | at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) mazerunner_1 | at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:34) mazerunner_1 | at org.apache.spark.ui.jobs.JobProgressListener.onJobStart(JobProgressListener.scala:198) mazerunner_1 | at org.apache.spark.scheduler.SparkListenerBus$class.onPostEvent(SparkListenerBus.scala:34) mazerunner_1 | at org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31) mazerunner_1 | at org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31) mazerunner_1 | at org.apache.spark.util.ListenerBus$class.postToAll(ListenerBus.scala:53) mazerunner_1 | at org.apache.spark.util.AsynchronousListenerBus.postToAll(AsynchronousListenerBus.scala:36) mazerunner_1 | at org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1.apply$mcV$sp(AsynchronousListenerBus.scala:76) mazerunner_1 | at org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1.apply(AsynchronousListenerBus.scala:61) mazerunner_1 | at org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1.apply(AsynchronousListenerBus.scala:61) mazerunner_1 | at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1618) mazerunner_1 | at org.apache.spark.util.AsynchronousListenerBus$$anon$1.run(AsynchronousListenerBus.scala:60) mazerunner_1 | Exception in thread "SparkListenerBus" java.lang.OutOfMemoryError: Java heap space mazerunner_1 | at org.apache.spark.ui.jobs.JobProgressListener$$anonfun$onJobStart$5$$anonfun$apply$9.apply(JobProgressListener.scala:200) mazerunner_1 | at org.apache.spark.ui.jobs.JobProgressListener$$anonfun$onJobStart$5$$anonfun$apply$9.apply(JobProgressListener.scala:200) mazerunner_1 | at scala.collection.mutable.MapLike$class.getOrElseUpdate(MapLike.scala:189) mazerunner_1 | at scala.collection.mutable.AbstractMap.getOrElseUpdate(Map.scala:91) mazerunner_1 | at org.apache.spark.ui.jobs.JobProgressListener$$anonfun$onJobStart$5.apply(JobProgressListener.scala:200) mazerunner_1 | at org.apache.spark.ui.jobs.JobProgressListener$$anonfun$onJobStart$5.apply(JobProgressListener.scala:198) mazerunner_1 | at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) mazerunner_1 | at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:34) mazerunner_1 | at org.apache.spark.ui.jobs.JobProgressListener.onJobStart(JobProgressListener.scala:198) mazerunner_1 | at org.apache.spark.scheduler.SparkListenerBus$class.onPostEvent(SparkListenerBus.scala:34) mazerunner_1 | at org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31) mazerunner_1 | at org.apache.spark.scheduler.LiveListenerBus.onPostEvent(LiveListenerBus.scala:31) mazerunner_1 | at org.apache.spark.util.ListenerBus$class.postToAll(ListenerBus.scala:53) mazerunner_1 | at org.apache.spark.util.AsynchronousListenerBus.postToAll(AsynchronousListenerBus.scala:36) mazerunner_1 | at org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1.apply$mcV$sp(AsynchronousListenerBus.scala:76) mazerunner_1 | at org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1.apply(AsynchronousListenerBus.scala:61) mazerunner_1 | at org.apache.spark.util.AsynchronousListenerBus$$anon$1$$anonfun$run$1.apply(AsynchronousListenerBus.scala:61) mazerunner_1 | at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1618) mazerunner_1 | at org.apache.spark.util.AsynchronousListenerBus$$anon$1.run(AsynchronousListenerBus.scala:60)
我试图修改我的docker-compose.yml文件，如下所示：
hdfs: environment: - "JAVA_OPTS=-Xmx5g" image: sequenceiq/hadoop-docker:2.4.1 command: /etc/bootstrap.sh -d -bash mazerunner: environment: - "JAVA_OPTS=-Xmx5g" image: kbastani/neo4j-graph-analytics:latest links: - hdfs graphdb: environment: - "JAVA_OPTS=-Xmx2g" image: kbastani/docker-neo4j:latest ports: - "7474:7474" - "1337:1337" volumes: - /opt/data links: - mazerunner - hdfs
没有成功。 如何configurationspark＆hdfs使用最大可用内存？
我的解决scheme是增加虚拟机的内存大小。 在我的Virtual Box UI上，我已经调整了“基本内存”大小滑块。
