Disable term frequency for some fields in solr

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Disable term frequency for some fields in solr

Amit Jha
Hi,

In my case I would like to disable term frequency for some fields. These
field should return a constant term frequency irrespective of how many
times a query term occur in those fields.

Regards
Shanu
Reply | Threaded
Open this post in threaded view
|

RE: Disable term frequency for some fields in solr

Markus Jelsma-2
Hi,

You can either use omitTermFreqAndPositions on that field or set a custom similarity for that field that returns 1 for tf > 0.

http://wiki.apache.org/solr/SchemaXml#Common_field_options
http://wiki.apache.org/solr/SchemaXml#Similarity
 
 
-----Original message-----

> From:Amit Jha <[hidden email]>
> Sent: Thu 06-Dec-2012 08:13
> To: [hidden email]
> Subject: Disable term frequency for some fields in solr
>
> Hi,
>
> In my case I would like to disable term frequency for some fields. These
> field should return a constant term frequency irrespective of how many
> times a query term occur in those fields.
>
> Regards
> Shanu
>
Reply | Threaded
Open this post in threaded view
|

Re: Disable term frequency for some fields in solr

Amit Jha
Hi,

How can I do this in solr4.

Amit
On Thu, Dec 6, 2012 at 1:40 PM, Markus Jelsma <[hidden email]>wrote:

> custom similarity for that field that returns 1 for
Reply | Threaded
Open this post in threaded view
|

Re: Disable term frequency for some fields in solr

Malcolm Upayavira Holmes
This involves taking a subclass of the DefaultSimilarity class, in Java,
and adding that to your Solr setup. For someone versed in Java, this is
relatively straight-forward. For others it is non-trivial.

Upayavira

On Wed, Jan 16, 2013, at 10:57 AM, Amit Jha wrote:
> Hi,
>
> How can I do this in solr4.
>
> Amit
> On Thu, Dec 6, 2012 at 1:40 PM, Markus Jelsma
> <[hidden email]>wrote:
>
> > custom similarity for that field that returns 1 for
Reply | Threaded
Open this post in threaded view
|

Re: Disable term frequency for some fields in solr

Amit Jha
Done same thing in solr3.6 and working but in sorl3.6 filed level of
similarity is not available. And Solr4 has Similarity Factories. So I was
not getting how do I do it on solr4. Which class do i need to extend and
move ahead.


On Wed, Jan 16, 2013 at 4:44 PM, Upayavira <[hidden email]> wrote:

> For someone versed in Jav
Reply | Threaded
Open this post in threaded view
|

Re: Disable term frequency for some fields in solr

Malcolm Upayavira Holmes
There's gonna be two ways to do this - for yourself or for everyone.

For yourself, you'll want to subclass
org.apache.lucene.search.similarities.DefaultSimilarity and
org.apache.solr.search.similarities.DefaultSimilarityFactory.

Alternatively, patch those two files to allow setting the TF or the IDF
via a configuration parameter, and post a patch to JIRA. I'm sure there
are other folks that would want the feature, and would hope it would be
accepted easily.

E.g. disableIDF=true or disableTF=true would make those functions just
return 1.

My thoughts anyhow.

Upayavira

On Wed, Jan 16, 2013, at 11:37 AM, Amit Jha wrote:
> Done same thing in solr3.6 and working but in sorl3.6 filed level of
> similarity is not available. And Solr4 has Similarity Factories. So I was
> not getting how do I do it on solr4. Which class do i need to extend and
> move ahead.
>
>
> On Wed, Jan 16, 2013 at 4:44 PM, Upayavira <[hidden email]> wrote:
>
> > For someone versed in Jav
Reply | Threaded
Open this post in threaded view
|

RE: Disable term frequency for some fields in solr

Markus Jelsma-2
I would prefer to use SchemaSimilarityFactory as a global similarity and configure a per-field similarity of which some use a flat TF impl. Much simples and no need to patch anything, just build a custom sim.

-----Original message-----

> From:Upayavira <[hidden email]>
> Sent: Wed 16-Jan-2013 21:22
> To: [hidden email]
> Subject: Re: Disable term frequency for some fields in solr
>
> There's gonna be two ways to do this - for yourself or for everyone.
>
> For yourself, you'll want to subclass
> org.apache.lucene.search.similarities.DefaultSimilarity and
> org.apache.solr.search.similarities.DefaultSimilarityFactory.
>
> Alternatively, patch those two files to allow setting the TF or the IDF
> via a configuration parameter, and post a patch to JIRA. I'm sure there
> are other folks that would want the feature, and would hope it would be
> accepted easily.
>
> E.g. disableIDF=true or disableTF=true would make those functions just
> return 1.
>
> My thoughts anyhow.
>
> Upayavira
>
> On Wed, Jan 16, 2013, at 11:37 AM, Amit Jha wrote:
> > Done same thing in solr3.6 and working but in sorl3.6 filed level of
> > similarity is not available. And Solr4 has Similarity Factories. So I was
> > not getting how do I do it on solr4. Which class do i need to extend and
> > move ahead.
> >
> >
> > On Wed, Jan 16, 2013 at 4:44 PM, Upayavira <[hidden email]> wrote:
> >
> > > For someone versed in Jav
>
Reply | Threaded
Open this post in threaded view
|

Re: Disable term frequency for some fields in solr

Amit Jha
Please correct my understanding,

Use one of the factory as global similarity.

And extends org.apache.lucene.search.similarities.DefaultSimilarity to create custom sim.

And add a similarity tag in field type definition for required fields.

Or there is some other way to do that?

Rgds
AJ

On 17-Jan-2013, at 4:08, Markus Jelsma <[hidden email]> wrote:

> org.apache.lucene.search.similarities.DefaultSimilarity
Reply | Threaded
Open this post in threaded view
|

Re: Disable term frequency for some fields in solr

Chris Hostetter-3

: Or there is some other way to do that?

I'm late to this thread, but what was wrong with the simple suggestion of
omitTermFreqAndPositions="true" ?


-Hoss
Reply | Threaded
Open this post in threaded view
|

Re: Disable term frequency for some fields in solr

Amit Jha
It will affect the phrase queries. That is why I am not using suggest
configuration.

On Thu, Jan 17, 2013 at 7:20 AM, Chris Hostetter
<[hidden email]>wrote:

>
> : Or there is some other way to do that?
>
> I'm late to this thread, but what was wrong with the simple suggestion of
> omitTermFreqAndPositions="true" ?
>
>
> -Hoss
>