Optimizing nutch crawl for fastest performance

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Optimizing nutch crawl for fastest performance

Tranquil
Hi,

I've been testing nutch for a while now, with the purpose of using it only
as a crawler (fetcher) of URL's. (mainly for d/l content from pages.)
meaning that i have no need of the index/search capabilities of nutch.

My question is, how can i optimize nutch (by configuration /  code or
hardware ) to fetch pages with topmost speed available.

thank you.

--
Eyal Edri