Debugging Nutch in Netbeans

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Debugging Nutch in Netbeans

Kevin MacDonald-3
Does anyone have an ant script that would allow debugging into Nutch running
on Netbeans?

Thanks

Kevin
Reply | Threaded
Open this post in threaded view
|

Re: Debugging Nutch in Netbeans

Kevin MacDonald-3
I figured out how to debug into Nutch on Netbeans. This is a really good
how-to on the subject if anyone's interested.
http://www.netbeans.org/kb/articles/freeform-import.html

Something I am unable to do however, is debug very far into it. I am running
on Vista, and when the code executes at some point it attempts to invoke
"bash" and fails. Everything works fine at the command line using Cygwin,
but when debugging, Netbeans is not running in the Cygwin shell, so an
exception happens. Before I resign myself to printf debugging I am hoping
that there is a way of configuring Nutch (or possibly hadoop) to not do
whatever it is currently doing that causes the failure. If anyone has
experience getting Nutch working under Windows/Vista without using Cygwin I
would very much appreciate any hints you can give me. Below is the
stacktrace I get when debugging.

javax.security.auth.login.LoginException: Login failed: Cannot run program
"bash": CreateProcess error=2, The system cannot find the file specified
        at
org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:250)
        at
org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:275)
        at
org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:257)
        at
org.apache.hadoop.security.UserGroupInformation.login(UserGroupInformation.java:67)
        at
org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:1353)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1289)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:203)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:108)
        at org.apache.nutch.crawl.Crawl.main(Crawl.java:89)


Thanks

Kevin

On Mon, Sep 8, 2008 at 10:12 AM, Kevin MacDonald <[hidden email]>wrote:

> Does anyone have an ant script that would allow debugging into Nutch
> running on Netbeans?
>
> Thanks
>
> Kevin
>
Reply | Threaded
Open this post in threaded view
|

Re: Debugging Nutch in Netbeans

Andrzej Białecki-2
Kevin MacDonald wrote:

> I figured out how to debug into Nutch on Netbeans. This is a really good
> how-to on the subject if anyone's interested.
> http://www.netbeans.org/kb/articles/freeform-import.html
>
> Something I am unable to do however, is debug very far into it. I am running
> on Vista, and when the code executes at some point it attempts to invoke
> "bash" and fails. Everything works fine at the command line using Cygwin,
> but when debugging, Netbeans is not running in the Cygwin shell, so an
> exception happens. Before I resign myself to printf debugging I am hoping
> that there is a way of configuring Nutch (or possibly hadoop) to not do
> whatever it is currently doing that causes the failure. If anyone has
> experience getting Nutch working under Windows/Vista without using Cygwin I
> would very much appreciate any hints you can give me.

First, the bad news: currently there is NO WAY to run Hadoop (and
consequently Nutch) under Windows without Cygwin. Hadoop code makes
explicit calls to unix-like utilities, such as whoami, df, ls, chmod,
chgrp, bash, etc, etc ... and if they are not found on the path then bad
things will happen, as you witnessed yourself.

If there is enough pressure from Windows users, Hadoop developers might
work out the support for native Windows equivalents of these utilities,
and include them in libhadoop.dll (or provide workarounds, or return
fake values, or ...). At the moment, there is not enough demand (in
terms of dedicated users) to try to provide such support, so it's easier
just to install Cygwin and be done with it. :)


  Below is the
> stacktrace I get when debugging.
>
> javax.security.auth.login.LoginException: Login failed: Cannot run program
> "bash": CreateProcess error=2, The system cannot find the file specified

After you have installed the Cygwin, please set the PATH, which Netbeans
uses, so that it includes the Cygwin .../bin and .../usr/bin. I believe
the Cygwin setup already does this for you.

--
Best regards,
Andrzej Bialecki     <><
  ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com