document boost and omitted norms

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

document boost and omitted norms

Karl Wettin
Is it so that document and field boosts are omitted together with
Field#setOmitNorms? By setting lengthNorm to 1f in the Similarity for
these fields and not omitting norms would fix it?


     karl

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: document boost and omitted norms

Michael McCandless-2

Yes and yes.

Mike

Karl Wettin wrote:

> Is it so that document and field boosts are omitted together with  
> Field#setOmitNorms? By setting lengthNorm to 1f in the Similarity  
> for these fields and not omitting norms would fix it?
>
>
>     karl
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: document boost and omitted norms

Karl Wettin
Michael McCandless skrev:
> Karl Wettin wrote:
>> Is it so that document and field boosts are omitted together with
>> Field#setOmitNorms? By setting lengthNorm to 1f in the Similarity for
>> these fields and not omitting norms would fix it?
> >
> Yes and yes.

I'm not sure if I do these things backwards or what, but in this case
the resolution of the 8 bit float of norms is not enough for me.

So I patched the norms float table in Similarity to handle the boost
values I assign my documents. Discretized 100f-250f in 60 bags and
cought up with the default table in 16 more bags.

But to me it seem as this only works if I also turn off length
normalization for all the other fields in my index, as they will affect
the value of the norm?

I could store it in the payloads of my tokens too, but I really don't
want payloads on all of them.

Now I'm thinking it would be nice to store a full float in an optional
document boost file. Perhaps with variable bytesize. But that would take
a lot of effort to update all related code.

Is there an easier way out here? Perhaps variable byte size per norms
and field? Document payloads? I don't know. Help much appreciated.



     karl


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: document boost and omitted norms

Karl Wettin
Karl Wettin skrev:

> Michael McCandless skrev:
>> Karl Wettin wrote:
>>> Is it so that document and field boosts are omitted together with
>>> Field#setOmitNorms? By setting lengthNorm to 1f in the Similarity for
>>> these fields and not omitting norms would fix it?
>> >
>> Yes and yes.
>
> I'm not sure if I do these things backwards or what, but in this case
> the resolution of the 8 bit float of norms is not enough for me.
>
> So I patched the norms float table in Similarity to handle the boost
> values I assign my documents. Discretized 100f-250f in 60 bags and
> cought up with the default table in 16 more bags.
>
> But to me it seem as this only works if I also turn off length
> normalization for all the other fields in my index, as they will affect
> the value of the norm?
>
> I could store it in the payloads of my tokens too, but I really don't
> want payloads on all of them.
>
> Now I'm thinking it would be nice to store a full float in an optional
> document boost file. Perhaps with variable bytesize. But that would take
> a lot of effort to update all related code.
>
> Is there an easier way out here? Perhaps variable byte size per norms
> and field? Document payloads? I don't know. Help much appreciated.
>

PS

What I really try to do is to add a weight to each document based on a
number of values in the domain object it represents. As I only want to
add this weight to the documents that match the query I don't can't add
a query with boosts on these values as that would match all documents.


     karl



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]