Nutch 1.5 Deploy Mode Doesn't Work like Nutch 1.4 Deploy Mode

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Nutch 1.5 Deploy Mode Doesn't Work like Nutch 1.4 Deploy Mode

sidbatra
Hi,

I'm excited to upgrade to Nutch 1.5 but something seems fundamentally different about the binaries generated in runtime/deploy

With Nutch 1.4 after downloading the source and running ant the runtime/deploy folder binary and job file was ready to work on hadoop and worked seamlessly.

With Nutch 1.5 after downloading the source and running ant the runtime/deploy folder still contains the job file and binary but upon running it I get this error -

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/util/Tool
        at java.lang.ClassLoader.defineClass1(Native Method)
        at java.lang.ClassLoader.defineClass(ClassLoader.java:634)
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:277)
        at java.net.URLClassLoader.access$000(URLClassLoader.java:73)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:212)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.Tool
        at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
        ... 11 more
Could not find the main class: org.apache.nutch.crawl.Crawl. Program will exit.


Can someone please help out and shed light on what has fundamentally changed with the deployment process in Nutch 1.5 ??

Thanks for your help.
Reply | Threaded
Open this post in threaded view
|

Re: Nutch 1.5 Deploy Mode Doesn't Work like Nutch 1.4 Deploy Mode

sidbatra
This turns out to be a genuine bug with an easy fix.

build.xml is configured to generate a job file titled "apache-nutch-1.5.job" but the deploy binary is still looking for "nutch-1.5.job"


Renaming "apache-nutch-1.5.job" to "nutch-1.5.job" fixes this bug in deploy mode.
Reply | Threaded
Open this post in threaded view
|

Re: Nutch 1.5 Deploy Mode Doesn't Work like Nutch 1.4 Deploy Mode

Julien Nioche-4
Good catch. Could you please open an issue on JIRA?

On 19 June 2012 00:09, sidbatra <[hidden email]> wrote:

> This turns out to be a genuine bug with an easy fix.
>
> build.xml is configured to generate a job file titled
> "apache-nutch-1.5.job"
> but the deploy binary is still looking for "nutch-1.5.job"
>
>
> Renaming "apache-nutch-1.5.job" to "nutch-1.5.job" fixes this bug in deploy
> mode.
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Nutch-1-5-Deploy-Mode-Doesn-t-Work-like-Nutch-1-4-Deploy-Mode-tp3990169p3990196.html
> Sent from the Nutch - User mailing list archive at Nabble.com.
>



--
*
*Open Source Solutions for Text Engineering

http://digitalpebble.blogspot.com/
http://www.digitalpebble.com
http://twitter.com/digitalpebble
Reply | Threaded
Open this post in threaded view
|

Re: Nutch 1.5 Deploy Mode Doesn't Work like Nutch 1.4 Deploy Mode

Julien Nioche-4
In reply to this post by sidbatra
Alternatively modify the bin/nutch script to make it more robust

*# NUTCH_JOB
if [ -f ${NUTCH_HOME}/*nutch*.job ]; then
    local=false
  for f in $NUTCH_HOME/*nutch*.job; do
    NUTCH_JOB=$f;
  done
fi*

On 19 June 2012 00:09, sidbatra <[hidden email]> wrote:

> This turns out to be a genuine bug with an easy fix.
>
> build.xml is configured to generate a job file titled
> "apache-nutch-1.5.job"
> but the deploy binary is still looking for "nutch-1.5.job"
>
>
> Renaming "apache-nutch-1.5.job" to "nutch-1.5.job" fixes this bug in deploy
> mode.
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Nutch-1-5-Deploy-Mode-Doesn-t-Work-like-Nutch-1-4-Deploy-Mode-tp3990169p3990196.html
> Sent from the Nutch - User mailing list archive at Nabble.com.
>



--
*
*Open Source Solutions for Text Engineering

http://digitalpebble.blogspot.com/
http://www.digitalpebble.com
http://twitter.com/digitalpebble
Reply | Threaded
Open this post in threaded view
|

Re: Nutch 1.5 Deploy Mode Doesn't Work like Nutch 1.4 Deploy Mode

Julien Nioche-4
In reply to this post by Julien Nioche-4
Have committed it. Thanks for reporting the issue Sidbatra

Julien

On 19 June 2012 08:52, Julien Nioche <[hidden email]> wrote:

> Good catch. Could you please open an issue on JIRA?
>
>
> On 19 June 2012 00:09, sidbatra <[hidden email]> wrote:
>
>> This turns out to be a genuine bug with an easy fix.
>>
>> build.xml is configured to generate a job file titled
>> "apache-nutch-1.5.job"
>> but the deploy binary is still looking for "nutch-1.5.job"
>>
>>
>> Renaming "apache-nutch-1.5.job" to "nutch-1.5.job" fixes this bug in
>> deploy
>> mode.
>>
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/Nutch-1-5-Deploy-Mode-Doesn-t-Work-like-Nutch-1-4-Deploy-Mode-tp3990169p3990196.html
>> Sent from the Nutch - User mailing list archive at Nabble.com.
>>
>
>
>
> --
> *
> *Open Source Solutions for Text Engineering
>
> http://digitalpebble.blogspot.com/
> http://www.digitalpebble.com
> http://twitter.com/digitalpebble
>
>


--
*
*Open Source Solutions for Text Engineering

http://digitalpebble.blogspot.com/
http://www.digitalpebble.com
http://twitter.com/digitalpebble