Adding new plugin and classloading issues

classic Classic list List threaded Threaded
12 messages Options
adb
Reply | Threaded
Open this post in threaded view
|

Adding new plugin and classloading issues

adb
I'm using the 0.9 plugin framework and am trying to add a new plugin and am
having classloading issues.

I want to use Javamail's mail.jar and BouncyCastle jars.  My plugin
implemnentation catches javax.mail.internet.ParseException and when it tries to
create the new instance of my Parser, it gives

java.lang.NoClassDefFoundError: javax/mail/internet/ParseException
         at java.lang.Class.getDeclaredConstructors0(Native Method)
         at java.lang.Class.privateGetDeclaredConstructors(Class.java:2389)
         at java.lang.Class.getConstructor0(Class.java:2699)
         at java.lang.Class.newInstance0(Class.java:326)
         at java.lang.Class.newInstance(Class.java:308)
         at
org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:160)
         at org.apache.nutch.parse.ParserFactory.getParsers(ParserFactory.java:130)

My plugin.xml has

    <runtime>
       <library name="parse-mime.jar">
          <export name="*"/>
       </library>
       <library name="bcmail-jdk16-141.jar"/>
       <library name="bcprov-jdk16-141.jar"/>
       <library name="mail.jar"/>
    </runtime>

I have tried it with the export for all libraries, also by having a lib-javamail
plugin, which does an export for mail.jar and with my plugin having the

    <requires>
       <import plugin="lib-javamail"/>
    </requires>

but it just doesn't work.  It seems identical to the standard PDF parsing or the
POI based parsers, so I'm at a loss...

It's a stand alone app, no Tomcat or anything else.

Any ideas?
Antony

Reply | Threaded
Open this post in threaded view
|

Re: Adding new plugin and classloading issues

Otis Gospodnetic-2-2
Step one is to identify the exact jar where this class lives.  Are you sure it's in mail.jar?  Maybe it's in activate.jar?

Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch



----- Original Message ----

> From: Antony Bowesman <[hidden email]>
> To: [hidden email]
> Sent: Friday, January 23, 2009 2:49:48 AM
> Subject: Adding new plugin and classloading issues
>
> I'm using the 0.9 plugin framework and am trying to add a new plugin and am
> having classloading issues.
>
> I want to use Javamail's mail.jar and BouncyCastle jars.  My plugin
> implemnentation catches javax.mail.internet.ParseException and when it tries to
> create the new instance of my Parser, it gives
>
> java.lang.NoClassDefFoundError: javax/mail/internet/ParseException
>         at java.lang.Class.getDeclaredConstructors0(Native Method)
>         at java.lang.Class.privateGetDeclaredConstructors(Class.java:2389)
>         at java.lang.Class.getConstructor0(Class.java:2699)
>         at java.lang.Class.newInstance0(Class.java:326)
>         at java.lang.Class.newInstance(Class.java:308)
>         at
> org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:160)
>         at
> org.apache.nutch.parse.ParserFactory.getParsers(ParserFactory.java:130)
>
> My plugin.xml has
>
>  
>      
>        
>      
>      
>      
>      
>  
>
> I have tried it with the export for all libraries, also by having a lib-javamail
> plugin, which does an export for mail.jar and with my plugin having the
>
>  
>      
>  
>
> but it just doesn't work.  It seems identical to the standard PDF parsing or the
> POI based parsers, so I'm at a loss...
>
> It's a stand alone app, no Tomcat or anything else.
>
> Any ideas?
> Antony

adb
Reply | Threaded
Open this post in threaded view
|

Re: Adding new plugin and classloading issues

adb
Otis Gospodnetic wrote:
> Step one is to identify the exact jar where this class lives.  Are you sure it's in mail.jar?  Maybe it's in activate.jar?

$ jar tvf mail.jar | grep ParseException
    385 Wed Oct 17 11:05:30 EST 2007 javax/mail/internet/ParseException.class

It's javamail 1.4.1.  If I put mail.jar into my application's lib directory,
where all jars are put on the classpath it finds the class.  However, then it
fails on another class from the BouncyCastle jars, also part of the plugin, so
it's some Nutch issue.

In the debugger, it shows the plugin descriptor's classloader having the 4
libraries in the loader.ucp.path ArrayList.

ExtensionClass.getExtensionInstance() does the following

         Class extensionClazz = loader.loadClass(getClazz());
         // lazy loading of Plugin in case there is no instance of the plugin
         // already.
         this.pluginRepository.getPluginInstance(getDescriptor());
         Object object = extensionClazz.newInstance();

and it is the call to newInstance() which gives the CNF exception.  The final
entry in the stack is

Launcher$AppClassLoader(ClassLoader).loadClassInternal(String), where the arg is
javax.mail.internet.ParseException.

I've not managed to figure out what part of the object creation the Nutch
classload takes part in, but it seems it doesn't have a lot to do with the
classloading for the plugins.

Antony


>
> Otis
> --
> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
>
>
>
> ----- Original Message ----
>> From: Antony Bowesman <[hidden email]>
>> To: [hidden email]
>> Sent: Friday, January 23, 2009 2:49:48 AM
>> Subject: Adding new plugin and classloading issues
>>
>> I'm using the 0.9 plugin framework and am trying to add a new plugin and am
>> having classloading issues.
>>
>> I want to use Javamail's mail.jar and BouncyCastle jars.  My plugin
>> implemnentation catches javax.mail.internet.ParseException and when it tries to
>> create the new instance of my Parser, it gives
>>
>> java.lang.NoClassDefFoundError: javax/mail/internet/ParseException
>>         at java.lang.Class.getDeclaredConstructors0(Native Method)
>>         at java.lang.Class.privateGetDeclaredConstructors(Class.java:2389)
>>         at java.lang.Class.getConstructor0(Class.java:2699)
>>         at java.lang.Class.newInstance0(Class.java:326)
>>         at java.lang.Class.newInstance(Class.java:308)
>>         at
>> org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:160)
>>         at
>> org.apache.nutch.parse.ParserFactory.getParsers(ParserFactory.java:130)
>>
>> My plugin.xml has
>>
>>  
>>      
>>        
>>      
>>      
>>      
>>      
>>  
>>
>> I have tried it with the export for all libraries, also by having a lib-javamail
>> plugin, which does an export for mail.jar and with my plugin having the
>>
>>  
>>      
>>  
>>
>> but it just doesn't work.  It seems identical to the standard PDF parsing or the
>> POI based parsers, so I'm at a loss...
>>
>> It's a stand alone app, no Tomcat or anything else.
>>
>> Any ideas?
>> Antony
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Adding new plugin and classloading issues

Doğacan Güney-3
Can you check build/plugins/<your plugin> to see if mail.jar is there?

dogacan-mbp:~/Documents/kod/nutch-svn$ ls build/plugins/parse-pdf/
FontBox-0.1.0-dev.jar parse-pdf.jar
PDFBox-0.7.3.jar plugin.xml

It should be there, based on the files you sent, but maybe there is a bug there
in nutch we are missing.

On Sun, Jan 25, 2009 at 11:47 PM, Antony Bowesman <[hidden email]> wrote:

> Otis Gospodnetic wrote:
>>
>> Step one is to identify the exact jar where this class lives.  Are you
>> sure it's in mail.jar?  Maybe it's in activate.jar?
>
> $ jar tvf mail.jar | grep ParseException
>   385 Wed Oct 17 11:05:30 EST 2007 javax/mail/internet/ParseException.class
>
> It's javamail 1.4.1.  If I put mail.jar into my application's lib directory,
> where all jars are put on the classpath it finds the class.  However, then
> it fails on another class from the BouncyCastle jars, also part of the
> plugin, so it's some Nutch issue.
>
> In the debugger, it shows the plugin descriptor's classloader having the 4
> libraries in the loader.ucp.path ArrayList.
>
> ExtensionClass.getExtensionInstance() does the following
>
>        Class extensionClazz = loader.loadClass(getClazz());
>        // lazy loading of Plugin in case there is no instance of the plugin
>        // already.
>        this.pluginRepository.getPluginInstance(getDescriptor());
>        Object object = extensionClazz.newInstance();
>
> and it is the call to newInstance() which gives the CNF exception.  The
> final entry in the stack is
>
> Launcher$AppClassLoader(ClassLoader).loadClassInternal(String), where the
> arg is javax.mail.internet.ParseException.
>
> I've not managed to figure out what part of the object creation the Nutch
> classload takes part in, but it seems it doesn't have a lot to do with the
> classloading for the plugins.
>
> Antony
>
>
>>
>> Otis
>> --
>> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
>>
>>
>>
>> ----- Original Message ----
>>>
>>> From: Antony Bowesman <[hidden email]>
>>> To: [hidden email]
>>> Sent: Friday, January 23, 2009 2:49:48 AM
>>> Subject: Adding new plugin and classloading issues
>>>
>>> I'm using the 0.9 plugin framework and am trying to add a new plugin and
>>> am having classloading issues.
>>>
>>> I want to use Javamail's mail.jar and BouncyCastle jars.  My plugin
>>> implemnentation catches javax.mail.internet.ParseException and when it tries
>>> to create the new instance of my Parser, it gives
>>>
>>> java.lang.NoClassDefFoundError: javax/mail/internet/ParseException
>>>        at java.lang.Class.getDeclaredConstructors0(Native Method)
>>>        at java.lang.Class.privateGetDeclaredConstructors(Class.java:2389)
>>>        at java.lang.Class.getConstructor0(Class.java:2699)
>>>        at java.lang.Class.newInstance0(Class.java:326)
>>>        at java.lang.Class.newInstance(Class.java:308)
>>>        at
>>> org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:160)
>>>        at
>>> org.apache.nutch.parse.ParserFactory.getParsers(ParserFactory.java:130)
>>>
>>> My plugin.xml has
>>>
>>>
>>> I have tried it with the export for all libraries, also by having a
>>> lib-javamail plugin, which does an export for mail.jar and with my plugin
>>> having the
>>>
>>>
>>> but it just doesn't work.  It seems identical to the standard PDF parsing
>>> or the POI based parsers, so I'm at a loss...
>>>
>>> It's a stand alone app, no Tomcat or anything else.
>>>
>>> Any ideas?
>>> Antony
>>
>>
>
>



--
Doğacan Güney
adb
Reply | Threaded
Open this post in threaded view
|

Re: Adding new plugin and classloading issues

adb
D:\Toss-1.3.3x\plugins>ls -l parse-mime
total 2120
-rwxrwxrwx 1 adb None  217843 Jan 23 15:42 bcmail-jdk16-141.jar
-rwxrwxrwx 1 adb None 1680122 Jan 23 15:42 bcprov-jdk16-141.jar
-rwxrwxrwx 1 adb None  371264 May 28  2008 mail.jar
-rwxrwxrwx 1 adb None    5648 Jan 23 15:44 parse-mime.jar
-rwxrwxrwx 1 adb None    1057 Jan 23 18:30 plugin.xml

Rgds
Antony

Doğacan Güney wrote:

> Can you check build/plugins/<your plugin> to see if mail.jar is there?
>
> dogacan-mbp:~/Documents/kod/nutch-svn$ ls build/plugins/parse-pdf/
> FontBox-0.1.0-dev.jar parse-pdf.jar
> PDFBox-0.7.3.jar plugin.xml
>
> It should be there, based on the files you sent, but maybe there is a bug there
> in nutch we are missing.
>
> On Sun, Jan 25, 2009 at 11:47 PM, Antony Bowesman <[hidden email]> wrote:
>> Otis Gospodnetic wrote:
>>> Step one is to identify the exact jar where this class lives.  Are you
>>> sure it's in mail.jar?  Maybe it's in activate.jar?
>> $ jar tvf mail.jar | grep ParseException
>>   385 Wed Oct 17 11:05:30 EST 2007 javax/mail/internet/ParseException.class
>>
>> It's javamail 1.4.1.  If I put mail.jar into my application's lib directory,
>> where all jars are put on the classpath it finds the class.  However, then
>> it fails on another class from the BouncyCastle jars, also part of the
>> plugin, so it's some Nutch issue.
>>
>> In the debugger, it shows the plugin descriptor's classloader having the 4
>> libraries in the loader.ucp.path ArrayList.
>>
>> ExtensionClass.getExtensionInstance() does the following
>>
>>        Class extensionClazz = loader.loadClass(getClazz());
>>        // lazy loading of Plugin in case there is no instance of the plugin
>>        // already.
>>        this.pluginRepository.getPluginInstance(getDescriptor());
>>        Object object = extensionClazz.newInstance();
>>
>> and it is the call to newInstance() which gives the CNF exception.  The
>> final entry in the stack is
>>
>> Launcher$AppClassLoader(ClassLoader).loadClassInternal(String), where the
>> arg is javax.mail.internet.ParseException.
>>
>> I've not managed to figure out what part of the object creation the Nutch
>> classload takes part in, but it seems it doesn't have a lot to do with the
>> classloading for the plugins.
>>
>> Antony
>>
>>
>>> Otis
>>> --
>>> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
>>>
>>>
>>>
>>> ----- Original Message ----
>>>> From: Antony Bowesman <[hidden email]>
>>>> To: [hidden email]
>>>> Sent: Friday, January 23, 2009 2:49:48 AM
>>>> Subject: Adding new plugin and classloading issues
>>>>
>>>> I'm using the 0.9 plugin framework and am trying to add a new plugin and
>>>> am having classloading issues.
>>>>
>>>> I want to use Javamail's mail.jar and BouncyCastle jars.  My plugin
>>>> implemnentation catches javax.mail.internet.ParseException and when it tries
>>>> to create the new instance of my Parser, it gives
>>>>
>>>> java.lang.NoClassDefFoundError: javax/mail/internet/ParseException
>>>>        at java.lang.Class.getDeclaredConstructors0(Native Method)
>>>>        at java.lang.Class.privateGetDeclaredConstructors(Class.java:2389)
>>>>        at java.lang.Class.getConstructor0(Class.java:2699)
>>>>        at java.lang.Class.newInstance0(Class.java:326)
>>>>        at java.lang.Class.newInstance(Class.java:308)
>>>>        at
>>>> org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:160)
>>>>        at
>>>> org.apache.nutch.parse.ParserFactory.getParsers(ParserFactory.java:130)
>>>>
>>>> My plugin.xml has
>>>>
>>>>
>>>> I have tried it with the export for all libraries, also by having a
>>>> lib-javamail plugin, which does an export for mail.jar and with my plugin
>>>> having the
>>>>
>>>>
>>>> but it just doesn't work.  It seems identical to the standard PDF parsing
>>>> or the POI based parsers, so I'm at a loss...
>>>>
>>>> It's a stand alone app, no Tomcat or anything else.
>>>>
>>>> Any ideas?
>>>> Antony
>>>
>>
>
>
>

adb
Reply | Threaded
Open this post in threaded view
|

Re: Adding new plugin and classloading issues

adb
In reply to this post by Doğacan Güney-3
Argh, sorry for the noise!  After lots of digging, I found that my plugin
classes were also in my app jar file, so it was loaded by the parent class
loader, not the plugin class loader.

Regards
Antony


adb
Reply | Threaded
Open this post in threaded view
|

FYI: Re: Adding new plugin and classloading issues

adb
One useful piece of info about using mail.jar as a Nutch plugin:

If mail.jar is installed as a plugin, then the META-INF/mailcap file may not be
found from mail.jar.

javax.activation.MailcapCommandMap searches mailcap from a number of places, but
it will call SecuritySupport.getContextClassLoader() to load mailcap from
'resources'.  This returns Thread.currentThread().getContextClassLoader() to get
the classloader, which is not the PluginClassLoader, so does not find it.

Antony




Antony Bowesman wrote:
> Argh, sorry for the noise!  After lots of digging, I found that my
> plugin classes were also in my app jar file, so it was loaded by the
> parent class loader, not the plugin class loader.
>
> Regards
> Antony
>
>
>

Reply | Threaded
Open this post in threaded view
|

Error in eclipse when crawl

Rolando Bermudez Peña
Hello all,
 I am new in Nutch, already create a new project in Eclipse 3.4 width the latest nutch in trunk and got the following error in the java console when trying to crawl some intranet site.

Exception in thread "main" java.io.IOException: Failed to get the current user's information.
        at org.apache.hadoop.mapred.JobClient.getUGI(JobClient.java:717)
        at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:592)
        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:774)
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1127)
        at org.apache.nutch.crawl.Injector.inject(Injector.java:160)
        at org.apache.nutch.crawl.Crawl.main(Crawl.java:112)
Caused by: javax.security.auth.login.LoginException: Login failed: Cannot run program "whoami": CreateProcess error=2, The system cannot find the file specified
        at org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:250)
        at org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:275)
        at org.apache.hadoop.mapred.JobClient.getUGI(JobClient.java:715)
        ... 5 more

Someone can help with any idea of what means 'whoami' or whats is the problem?

Thanks in advance.
Rolando.

Reply | Threaded
Open this post in threaded view
|

Re: Error in eclipse when crawl

Doğacan Güney-3
Are you using windows? "whoami" is a unix program that returns the
name of current user,
nutch (actually hadoop) needs it for security purposes.

On Mon, Jan 26, 2009 at 6:51 AM, Rolando Bermudez Peña <[hidden email]> wrote:

> Hello all,
>  I am new in Nutch, already create a new project in Eclipse 3.4 width the latest nutch in trunk and got the following error in the java console when trying to crawl some intranet site.
>
> Exception in thread "main" java.io.IOException: Failed to get the current user's information.
>        at org.apache.hadoop.mapred.JobClient.getUGI(JobClient.java:717)
>        at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:592)
>        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:774)
>        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1127)
>        at org.apache.nutch.crawl.Injector.inject(Injector.java:160)
>        at org.apache.nutch.crawl.Crawl.main(Crawl.java:112)
> Caused by: javax.security.auth.login.LoginException: Login failed: Cannot run program "whoami": CreateProcess error=2, The system cannot find the file specified
>        at org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:250)
>        at org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:275)
>        at org.apache.hadoop.mapred.JobClient.getUGI(JobClient.java:715)
>        ... 5 more
>
> Someone can help with any idea of what means 'whoami' or whats is the problem?
>
> Thanks in advance.
> Rolando.
>
>



--
Doğacan Güney
Reply | Threaded
Open this post in threaded view
|

RE: Error in eclipse when crawl

Rolando Bermudez Peña
Yes, I am using windows, nutch can’t run in windows?

-----Original Message-----
From: Doğacan Güney [mailto:[hidden email]]
Sent: Monday, January 26, 2009 3:04 AM
To: [hidden email]
Subject: Re: Error in eclipse when crawl

Are you using windows? "whoami" is a unix program that returns the
name of current user,
nutch (actually hadoop) needs it for security purposes.

On Mon, Jan 26, 2009 at 6:51 AM, Rolando Bermudez Peña <[hidden email]> wrote:

> Hello all,
>  I am new in Nutch, already create a new project in Eclipse 3.4 width the latest nutch in trunk and got the following error in the java console when trying to crawl some intranet site.
>
> Exception in thread "main" java.io.IOException: Failed to get the current user's information.
>        at org.apache.hadoop.mapred.JobClient.getUGI(JobClient.java:717)
>        at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:592)
>        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:774)
>        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1127)
>        at org.apache.nutch.crawl.Injector.inject(Injector.java:160)
>        at org.apache.nutch.crawl.Crawl.main(Crawl.java:112)
> Caused by: javax.security.auth.login.LoginException: Login failed: Cannot run program "whoami": CreateProcess error=2, The system cannot find the file specified
>        at org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:250)
>        at org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:275)
>        at org.apache.hadoop.mapred.JobClient.getUGI(JobClient.java:715)
>        ... 5 more
>
> Someone can help with any idea of what means 'whoami' or whats is the problem?
>
> Thanks in advance.
> Rolando.
>
>



--
Doğacan Güney
Reply | Threaded
Open this post in threaded view
|

AW: Error in eclipse when crawl

Koch Martina
Hi Rolando,

you have to install cygwin to be able to run Nutch in Windows. Have a look at this tutorial for more information: http://wiki.apache.org/nutch/GettingNutchRunningWithWindows
Besides there are several posts on the mailing list regarding this error message.

Kind regards,
Martina

-----Ursprüngliche Nachricht-----
Von: Rolando Bermudez Peña [mailto:[hidden email]]
Gesendet: Montag, 26. Januar 2009 09:04
An: [hidden email]
Betreff: RE: Error in eclipse when crawl

Yes, I am using windows, nutch can’t run in windows?

-----Original Message-----
From: Doğacan Güney [mailto:[hidden email]]
Sent: Monday, January 26, 2009 3:04 AM
To: [hidden email]
Subject: Re: Error in eclipse when crawl

Are you using windows? "whoami" is a unix program that returns the
name of current user,
nutch (actually hadoop) needs it for security purposes.

On Mon, Jan 26, 2009 at 6:51 AM, Rolando Bermudez Peña <[hidden email]> wrote:

> Hello all,
>  I am new in Nutch, already create a new project in Eclipse 3.4 width the latest nutch in trunk and got the following error in the java console when trying to crawl some intranet site.
>
> Exception in thread "main" java.io.IOException: Failed to get the current user's information.
>        at org.apache.hadoop.mapred.JobClient.getUGI(JobClient.java:717)
>        at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:592)
>        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:774)
>        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1127)
>        at org.apache.nutch.crawl.Injector.inject(Injector.java:160)
>        at org.apache.nutch.crawl.Crawl.main(Crawl.java:112)
> Caused by: javax.security.auth.login.LoginException: Login failed: Cannot run program "whoami": CreateProcess error=2, The system cannot find the file specified
>        at org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:250)
>        at org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:275)
>        at org.apache.hadoop.mapred.JobClient.getUGI(JobClient.java:715)
>        ... 5 more
>
> Someone can help with any idea of what means 'whoami' or whats is the problem?
>
> Thanks in advance.
> Rolando.
>
>



--
Doğacan Güney
Reply | Threaded
Open this post in threaded view
|

RE: Error in eclipse when crawl

Rolando Bermudez Peña
Thanks Martina, that works with cygwin, now I am getting this error when crawling an intranet, and the virtual memory of java gets up to 1gb and the crawl stops with no signs an no more errors than the following

Error parsing: http://intranet/pdf/ronda_pupo.pdf: failed(2,0): Can't be
 handled as pdf document. java.io.IOException: Error: expected the end of a dict
ionary.

The crawl stop there and no matter how long I wait, do nothing from there.

Any ideas?

Best regards,
Rolando

-----Original Message-----
From: Koch Martina [mailto:[hidden email]]
Sent: Monday, January 26, 2009 3:21 AM
To: [hidden email]
Subject: AW: Error in eclipse when crawl

Hi Rolando,

you have to install cygwin to be able to run Nutch in Windows. Have a look at this tutorial for more information: http://wiki.apache.org/nutch/GettingNutchRunningWithWindows
Besides there are several posts on the mailing list regarding this error message.

Kind regards,
Martina

-----Ursprüngliche Nachricht-----
Von: Rolando Bermudez Peña [mailto:[hidden email]]
Gesendet: Montag, 26. Januar 2009 09:04
An: [hidden email]
Betreff: RE: Error in eclipse when crawl

Yes, I am using windows, nutch can’t run in windows?

-----Original Message-----
From: Doğacan Güney [mailto:[hidden email]]
Sent: Monday, January 26, 2009 3:04 AM
To: [hidden email]
Subject: Re: Error in eclipse when crawl

Are you using windows? "whoami" is a unix program that returns the
name of current user,
nutch (actually hadoop) needs it for security purposes.

On Mon, Jan 26, 2009 at 6:51 AM, Rolando Bermudez Peña <[hidden email]> wrote:

> Hello all,
>  I am new in Nutch, already create a new project in Eclipse 3.4 width the latest nutch in trunk and got the following error in the java console when trying to crawl some intranet site.
>
> Exception in thread "main" java.io.IOException: Failed to get the current user's information.
>        at org.apache.hadoop.mapred.JobClient.getUGI(JobClient.java:717)
>        at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:592)
>        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:774)
>        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1127)
>        at org.apache.nutch.crawl.Injector.inject(Injector.java:160)
>        at org.apache.nutch.crawl.Crawl.main(Crawl.java:112)
> Caused by: javax.security.auth.login.LoginException: Login failed: Cannot run program "whoami": CreateProcess error=2, The system cannot find the file specified
>        at org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:250)
>        at org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:275)
>        at org.apache.hadoop.mapred.JobClient.getUGI(JobClient.java:715)
>        ... 5 more
>
> Someone can help with any idea of what means 'whoami' or whats is the problem?
>
> Thanks in advance.
> Rolando.
>
>



--
Doğacan Güney