Memory problem while running Nutch

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Memory problem while running Nutch

Jayant Kumar Gandhi
Hi,

I installed Tomcat using cPanel/WHM as root. It downloaded JDK
along-with as well. I setup Java on the line of the popular 'Fedora
Core 4 Installation Notes'
http://stanton-finley.net/fedora_core_4_installation_notes.html#Java
as root.
As a regular user I downloaded nutch 0.7.2 and followed the tutorial
over at http://lucene.apache.org/nutch/tutorial.html .

The problem comes, when I run nutch to crawl I get the following error

run java in /usr/local/jdk1.5.0_05/jre
Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.

Tomcat is running fine.

The relevant environment variables are:
CLASSPATH=".:/usr/local/jdk/lib/classes.zip"
J2RE_HOME="/usr/local/jdk1.5.0_05/jre"
JAVA_HOME="/usr/local/jdk"
NUTCH_JAVA_HOME="/usr/local/jdk1.5.0_05/jre"
PATH="/usr/local/jdk/bin:/usr/kerberos/bin:/usr/local/jdk1.5.0_05/jre/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/usr/local/bin:/usr/X11R6/bin:/home/khojguru/bin"

Its a VPS. The server has 4 Xeon 2.4Ghz. OS: CentOS 3.7 with guranteed
RAM of 512Mb for me. Swap space is just 20% full. Rest is free. I have
total disk space of 5Gb of which 2Gb is free.

Please help.

Thanks and Regards,
Jayant Gandhi

--
www.jkg.in | http://www.jkg.in/contact-me/
Jayant Kr. Gandhi | +91-9871412929
M.Tech. Computer Tech. Class of 2007,
D-38, Aravali Hostel, IIT Delhi,
Hauz Khas, Delhi-110016
Reply | Threaded
Open this post in threaded view
|

Re: Memory problem while running Nutch

Nutch Newbie
Hi

Have a look at the bin/nutch script about JAVA HEAP SIZE adjust it to
your settings..you should see something a line like

JAVA_HEAP_MAX=-Xmx1000m

in bin/nutch script

rgds

On 6/15/06, Jayant Kumar Gandhi <[hidden email]> wrote:

> Hi,
>
> I installed Tomcat using cPanel/WHM as root. It downloaded JDK
> along-with as well. I setup Java on the line of the popular 'Fedora
> Core 4 Installation Notes'
> http://stanton-finley.net/fedora_core_4_installation_notes.html#Java
> as root.
> As a regular user I downloaded nutch 0.7.2 and followed the tutorial
> over at http://lucene.apache.org/nutch/tutorial.html .
>
> The problem comes, when I run nutch to crawl I get the following error
>
> run java in /usr/local/jdk1.5.0_05/jre
> Error occurred during initialization of VM
> Could not reserve enough space for object heap
> Could not create the Java virtual machine.
>
> Tomcat is running fine.
>
> The relevant environment variables are:
> CLASSPATH=".:/usr/local/jdk/lib/classes.zip"
> J2RE_HOME="/usr/local/jdk1.5.0_05/jre"
> JAVA_HOME="/usr/local/jdk"
> NUTCH_JAVA_HOME="/usr/local/jdk1.5.0_05/jre"
> PATH="/usr/local/jdk/bin:/usr/kerberos/bin:/usr/local/jdk1.5.0_05/jre/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/usr/local/bin:/usr/X11R6/bin:/home/khojguru/bin"
>
> Its a VPS. The server has 4 Xeon 2.4Ghz. OS: CentOS 3.7 with guranteed
> RAM of 512Mb for me. Swap space is just 20% full. Rest is free. I have
> total disk space of 5Gb of which 2Gb is free.
>
> Please help.
>
> Thanks and Regards,
> Jayant Gandhi
>
> --
> www.jkg.in | http://www.jkg.in/contact-me/
> Jayant Kr. Gandhi | +91-9871412929
> M.Tech. Computer Tech. Class of 2007,
> D-38, Aravali Hostel, IIT Delhi,
> Hauz Khas, Delhi-110016
>
Reply | Threaded
Open this post in threaded view
|

Re: Memory problem while running Nutch

Jayant Kumar Gandhi
I tried that, it didn't help. Thanks anyway.

I believe the problem is in Java and not just with Nutch.
Because I get the same error when i do java -version or java crashes.
Log of crash report is attached.
I read on one of the forums that it may be due to jdk1.5.0_05 and
jdk1.5.0_07 should work fine. I installed jre1.5.0_07 but still no
luck.

Can anyone help me install/configure JDK/JRE on CentOS?

Attached: error logs with both versions of java

Thanks in advance.
Regards,
Jayant.

On 6/15/06, Nutch Newbie <[hidden email]> wrote:

> Hi
>
> Have a look at the bin/nutch script about JAVA HEAP SIZE adjust it to
> your settings..you should see something a line like
>
> JAVA_HEAP_MAX=-Xmx1000m
>
> in bin/nutch script
>
> rgds
>
> On 6/15/06, Jayant Kumar Gandhi <[hidden email]> wrote:
> > Hi,
> >
> > I installed Tomcat using cPanel/WHM as root. It downloaded JDK
> > along-with as well. I setup Java on the line of the popular 'Fedora
> > Core 4 Installation Notes'
> > http://stanton-finley.net/fedora_core_4_installation_notes.html#Java
> > as root.
> > As a regular user I downloaded nutch 0.7.2 and followed the tutorial
> > over at http://lucene.apache.org/nutch/tutorial.html .
> >
> > The problem comes, when I run nutch to crawl I get the following error
> >
> > run java in /usr/local/jdk1.5.0_05/jre
> > Error occurred during initialization of VM
> > Could not reserve enough space for object heap
> > Could not create the Java virtual machine.
> >
> > Tomcat is running fine.
> >
> > The relevant environment variables are:
> > CLASSPATH=".:/usr/local/jdk/lib/classes.zip"
> > J2RE_HOME="/usr/local/jdk1.5.0_05/jre"
> > JAVA_HOME="/usr/local/jdk"
> > NUTCH_JAVA_HOME="/usr/local/jdk1.5.0_05/jre"
> > PATH="/usr/local/jdk/bin:/usr/kerberos/bin:/usr/local/jdk1.5.0_05/jre/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/usr/local/bin:/usr/X11R6/bin:/home/khojguru/bin"
> >
> > Its a VPS. The server has 4 Xeon 2.4Ghz. OS: CentOS 3.7 with guranteed
> > RAM of 512Mb for me. Swap space is just 20% full. Rest is free. I have
> > total disk space of 5Gb of which 2Gb is free.
> >
> > Please help.
> >
> > Thanks and Regards,
> > Jayant Gandhi
> >
> > --
> > www.jkg.in | http://www.jkg.in/contact-me/
> > Jayant Kr. Gandhi | +91-9871412929
> > M.Tech. Computer Tech. Class of 2007,
> > D-38, Aravali Hostel, IIT Delhi,
> > Hauz Khas, Delhi-110016
> >
>

--
www.jkg.in | http://www.jkg.in/contact-me/
Jayant Kr. Gandhi | +91-9871412929
M.Tech. Computer Tech. Class of 2007,
D-38, Aravali Hostel, IIT Delhi,
Hauz Khas, Delhi-110016
Reply | Threaded
Open this post in threaded view
|

Re: Memory problem while running Nutch

Thomas Delnoij-3
You can't run Nutch effectively on a VPS. You will either run into
memory / disk problems or into your hosting party :)

Rgrds, Thomas

On 6/15/06, Jayant Kumar Gandhi <[hidden email]> wrote:

> I tried that, it didn't help. Thanks anyway.
>
> I believe the problem is in Java and not just with Nutch.
> Because I get the same error when i do java -version or java crashes.
> Log of crash report is attached.
> I read on one of the forums that it may be due to jdk1.5.0_05 and
> jdk1.5.0_07 should work fine. I installed jre1.5.0_07 but still no
> luck.
>
> Can anyone help me install/configure JDK/JRE on CentOS?
>
> Attached: error logs with both versions of java
>
> Thanks in advance.
> Regards,
> Jayant.
>
> On 6/15/06, Nutch Newbie <[hidden email]> wrote:
> > Hi
> >
> > Have a look at the bin/nutch script about JAVA HEAP SIZE adjust it to
> > your settings..you should see something a line like
> >
> > JAVA_HEAP_MAX=-Xmx1000m
> >
> > in bin/nutch script
> >
> > rgds
> >
> > On 6/15/06, Jayant Kumar Gandhi <[hidden email]> wrote:
> > > Hi,
> > >
> > > I installed Tomcat using cPanel/WHM as root. It downloaded JDK
> > > along-with as well. I setup Java on the line of the popular 'Fedora
> > > Core 4 Installation Notes'
> > > http://stanton-finley.net/fedora_core_4_installation_notes.html#Java
> > > as root.
> > > As a regular user I downloaded nutch 0.7.2 and followed the tutorial
> > > over at http://lucene.apache.org/nutch/tutorial.html .
> > >
> > > The problem comes, when I run nutch to crawl I get the following error
> > >
> > > run java in /usr/local/jdk1.5.0_05/jre
> > > Error occurred during initialization of VM
> > > Could not reserve enough space for object heap
> > > Could not create the Java virtual machine.
> > >
> > > Tomcat is running fine.
> > >
> > > The relevant environment variables are:
> > > CLASSPATH=".:/usr/local/jdk/lib/classes.zip"
> > > J2RE_HOME="/usr/local/jdk1.5.0_05/jre"
> > > JAVA_HOME="/usr/local/jdk"
> > > NUTCH_JAVA_HOME="/usr/local/jdk1.5.0_05/jre"
> > > PATH="/usr/local/jdk/bin:/usr/kerberos/bin:/usr/local/jdk1.5.0_05/jre/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/usr/local/bin:/usr/X11R6/bin:/home/khojguru/bin"
> > >
> > > Its a VPS. The server has 4 Xeon 2.4Ghz. OS: CentOS 3.7 with guranteed
> > > RAM of 512Mb for me. Swap space is just 20% full. Rest is free. I have
> > > total disk space of 5Gb of which 2Gb is free.
> > >
> > > Please help.
> > >
> > > Thanks and Regards,
> > > Jayant Gandhi
> > >
> > > --
> > > www.jkg.in | http://www.jkg.in/contact-me/
> > > Jayant Kr. Gandhi | +91-9871412929
> > > M.Tech. Computer Tech. Class of 2007,
> > > D-38, Aravali Hostel, IIT Delhi,
> > > Hauz Khas, Delhi-110016
> > >
> >
>
>
> --
> www.jkg.in | http://www.jkg.in/contact-me/
> Jayant Kr. Gandhi | +91-9871412929
> M.Tech. Computer Tech. Class of 2007,
> D-38, Aravali Hostel, IIT Delhi,
> Hauz Khas, Delhi-110016
>
>