Installing nutch

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Installing nutch

Bernd Fehling
Hi list,
I came across nutch while looking for search engines
and that nutch with its NDFS is very interesting to me.

A basic question:
Is it possible to install nutch with NDFS on a single machine
or do I need at least two maschines?

I followed the instructions from Stefan Groschupf which helped
a lot but still makes some trouble.
The installation and setup instructions are OK.
Before installing the user interface I tried to create a searchable
index. As far as I can see the "admin" command has been removed
from nutch version 0.8?
So I tried "quick tutorial for nutch 0.8" but this does not work.
Using "bin/nutch ndfs -mkdir urls" makes no directory.

How do I get a searchable index?

Best regards,
Bernd

Reply | Threaded
Open this post in threaded view
|

Re: Installing nutch

Owen O'Malley-5

On Feb 5, 2006, at 9:35 AM, Bernd Fehling wrote:
>
> A basic question:
> Is it possible to install nutch with NDFS on a single machine
> or do I need at least two maschines?

Yes,  it is possible. I just ran a Hadoop map/reduce example on a
single machine using Hadoop DFS.  On a single node, I ran one instance
of all 4 servers (namenode, datanode, jobtracker, and tasktracker). I
was able to run a map/reduce application with reading the inputs from
DFS and writing the output to DFS.

Note that effectively, this configuration is only useful for testing
because you are wasting time using the distributed framework for a
single node. As a test, it was very useful. *smile*

Note that you do want to change the value of dfs.replication to 1.

I've never run the indexing part of Nutch, so I can't help you on that
side. The Hadoop framework works fine in that configuration.

-- Owen