Commom words

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Commom words

ocramp
Hi,

 Is there a way to Nutch ignore commom words while searching?
 For example,  while searching for "the boy and the girl" it would only look
for "boy girl".

Thanks,
Marco
Reply | Threaded
Open this post in threaded view
|

RE: Commom words

Bogdan Kecman

>  Is there a way to Nutch ignore commom words while searching?
>  For example,  while searching for "the boy and the girl" it
> would only look for "boy girl".

Yes,
In nutch conf dir there is a file common-terms.utf8
Copy that file also in your java container

Hope this helps
Bogdan

Reply | Threaded
Open this post in threaded view
|

RE: Commom words

Bogdan Kecman
In reply to this post by ocramp
 
>  Is there a way to Nutch ignore commom words while searching?
>  For example,  while searching for "the boy and the girl" it
> would only look for "boy girl".

Small addition from wiki:
http://wiki.apache.org/nutch/FAQ#head-12f4fd64f03fc3cd0a3063b9283ed829963ed4
88
You can tweak your conf/common-terms.utf8 file after creating an index
through the following command:

bin/nutch org.apache.nutch.indexer.HighFreqTerms -count 10 -nofreqs index

Regards
Bogdan