Mahout trainlogistic encounter jvm 'out of memory' issue

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Mahout trainlogistic encounter jvm 'out of memory' issue

jamesyi
This post has NOT been accepted by the mailing list yet.
I am running mahout job but always come across an OOM issue like below. Could you please help me how to fix this issue? Thanks in advance !

Mahout and Hadoop Version:
Mahout version: mahout-0.9+cdh5.3.0+17-1.cdh5.3.0.p0.23.el6.noarch
Hadoop verison: hadoop-2.5.0+cdh5.3.0+781-1.cdh5.3.0.p0.54.el6.x86_64

Error logs:
Command  mahout trainlogistic --input /tmp/tmp.Zywm8wYo2X --output /tmp/tmp.V9bKupaSHe --target c_customer_sk --categories 2 --predictors college_education male label --types n n n --passes 20 --features 20 --rate 1 --lambda 0.5
tmp output: /tmp/tmp.V9bKupaSHe
-------------------------
MAHOUT_LOCAL is not set; adding HADOOP_CONF_DIR to classpath.
Running on hadoop, using /usr/lib/hadoop/bin/hadoop and HADOOP_CONF_DIR=/etc/hadoop/conf
MAHOUT-JOB: /usr/lib/mahout/mahout-examples-0.9-cdh5.3.0-job.jar
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.io.BufferedReader.<init>(BufferedReader.java:98)
        at java.io.BufferedReader.<init>(BufferedReader.java:109)
        at org.apache.commons.csv.ExtendedBufferedReader.<init>(ExtendedBufferedReader.java:58)
        at org.apache.commons.csv.CSVParser.<init>(CSVParser.java:166)
        at org.apache.commons.csv.CSVParser.<init>(CSVParser.java:156)
        at org.apache.commons.csv.CSVParser.<init>(CSVParser.java:139)
        at org.apache.commons.csv.CSVParser.<init>(CSVParser.java:125)
        at org.apache.commons.csv.CSVUtils.parseLine(CSVUtils.java:118)
        at org.apache.mahout.classifier.sgd.CsvRecordFactory.parseCsvLine(CsvRecordFactory.java:113)
        at org.apache.mahout.classifier.sgd.CsvRecordFactory.processLine(CsvRecordFactory.java:264)
        at org.apache.mahout.classifier.sgd.TrainLogistic.mainToOutput(TrainLogistic.java:85)
        at org.apache.mahout.classifier.sgd.TrainLogistic.main(TrainLogistic.java:65)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
        at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:145)
        at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:153)
        at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:195)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
15/07/03 12:25:30 WARN util.ShutdownHookManager: ShutdownHook '' failed, java.lang.OutOfMemoryError: GC overhead limit exceeded
java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.lang.StringCoding$StringEncoder.encode(StringCoding.java:300)
        at java.lang.StringCoding.encode(StringCoding.java:344)
        at java.lang.String.getBytes(String.java:916)
        at java.io.UnixFileSystem.getBooleanAttributes0(Native Method)
        at java.io.UnixFileSystem.getBooleanAttributes(UnixFileSystem.java:242)
        at java.io.File.isFile(File.java:876)
        at org.apache.hadoop.fs.FileUtil.fullyDeleteContents(FileUtil.java:221)
        at org.apache.hadoop.fs.FileUtil.fullyDelete(FileUtil.java:140)
        at org.apache.hadoop.fs.FileUtil.fullyDeleteContents(FileUtil.java:237)
        at org.apache.hadoop.fs.FileUtil.fullyDelete(FileUtil.java:140)
        at org.apache.hadoop.fs.FileUtil.fullyDeleteContents(FileUtil.java:237)
        at org.apache.hadoop.fs.FileUtil.fullyDelete(FileUtil.java:140)
        at org.apache.hadoop.fs.FileUtil.fullyDeleteContents(FileUtil.java:237)
        at org.apache.hadoop.fs.FileUtil.fullyDelete(FileUtil.java:140)
        at org.apache.hadoop.fs.FileUtil.fullyDeleteContents(FileUtil.java:237)
        at org.apache.hadoop.fs.FileUtil.fullyDelete(FileUtil.java:140)
        at org.apache.hadoop.fs.FileUtil.fullyDeleteContents(FileUtil.java:237)
        at org.apache.hadoop.fs.FileUtil.fullyDelete(FileUtil.java:140)
        at org.apache.hadoop.fs.FileUtil.fullyDeleteContents(FileUtil.java:237)
        at org.apache.hadoop.fs.FileUtil.fullyDelete(FileUtil.java:140)
        at org.apache.hadoop.fs.FileUtil.fullyDelete(FileUtil.java:111)
        at org.apache.hadoop.util.RunJar$1.run(RunJar.java:183)
        at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:54)