How can I modify the crawler?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

How can I modify the crawler?

suxiaoke04
   I want to realize a topic-based search engine through modifing the nutch. For example I define a computer topic so I hope that I only find some information about computer. I can't find the appropriate point where I can insert myself sentence in Fetcher.java. Please tell me how can I modify the Fetcher and the parser? thanks.
 
 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How can I modify the crawler?

Ernesto De Santis-4
Hi suxiaoke

I do something similar, I did tell it category.

Do you need do it in two steps:

- index your topic.
- search filtering by topic.

In my approach, I build a pluing to index the category. In it plugin,
the category is resolved by a rules applied to the url. In your case,
you know how to decide the topic's values.

Then, search over it is very easy.
In your search code, add something like this:

query.addRequiredTerm(aTopicValue, "topic");

Good luck
Ernesto.


suxiaoke04 escribió:

>    I want to realize a topic-based search engine through modifing the nutch. For example I define a computer topic so I hope that I only find some information about computer. I can't find the appropriate point where I can insert myself sentence in Fetcher.java. Please tell me how can I modify the Fetcher and the parser? thanks.
>  
>  
>  
> ------------------------------------------------------------------------
>
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.1.405 / Virus Database: 268.12.3/446 - Release Date: 12/09/2006
>  

       
       
               
__________________________________________________
Pregunt�. Respond�. Descubr�.
Todo lo que quer�as saber, y lo que ni imaginabas,
est� en Yahoo! Respuestas (Beta).
�Probalo ya!
http://www.yahoo.com.ar/respuestas


Loading...