Nutch and HTTrack Crawler

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Nutch and HTTrack Crawler

sudhendra seshachala
Is there a way I could use HTTrack for crawling and nutch for just searching?
 Has any body done this before andcomparision between crawlers.
 
 How easy or tough is it to customize nutch crawler for a specific vertical ?
 I know a crude way of writing a crawler, but was wondering if any one has actually done a custom crawler..

Thanks
Sudhi




  Sudhi Seshachala
  http://sudhilogs.blogspot.com/
   


                       
---------------------------------
 Yahoo! Mail
 Use Photomail to share photos without annoying attachments.
Reply | Threaded
Open this post in threaded view
|

Re: Nutch and HTTrack Crawler

Stefan Groschupf-2

Am 23.02.2006 um 01:55 schrieb sudhendra seshachala:

> Is there a way I could use HTTrack for crawling and nutch for just  
> searching?
>  Has any body done this before andcomparision between crawlers.
I suggest take a look to lucene, since I guess it is more work  
changing nutch to your requirements than adding lucene to httrack.

>
>  How easy or tough is it to customize nutch crawler for a specific  
> vertical ?
This is a may be  to general  question but a my general answer would  
be, yes you can. :)

Stefan