How to get boosted field and values?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

How to get boosted field and values?

Taisuke Miyazaki
I'm using Solr 7.5.0.
I want to get boosted field and values per documents.

e.g.
documents:
  id: 1, features: [1]
  id: 2, features: [1,2]
  id: 3, features: [1,2,3]

query:
  bq: features:2^1.0 AND features:3^1.0

I expect results like below.
boosted:
  - id: 2
    - field: features, value: 2
  - id: 3
    - field: features, value: 2
    - field: features, value: 3

I have an idea that set boost score like bit-flag, but it's not good I
think because I must send query twice.

bit-flag:
  bq: features:2^2.0 AND features:3^4.0
  docs:
    - id: 1, score: 1.0(0x001)
    - id: 2, score: 3.0(0x011) # have feature:2(2nd bit is 1)
    - id: 3, score: 7.0(0x111) # have feature:2 and feature:3(2nd and 3rd
bit are 1)
check score value then I can get boosted field.

Is there a better way?
Reply | Threaded
Open this post in threaded view
|

Re: How to get boosted field and values?

Alessandro Benedetti
Hi Taisuke,
there are various ways of approaching boosting and scoring in Apache Solr.
First of all you must decide if you are interested in multiplicative or
additive boost.
Multiplicative will multiply the score of your search result by a certain
factor while the additive will just add the factor to the final score.

Using advanced query parsers such as the dismax and edismax you can use the
:
*boost* parameter - multiplicative - takes function in input -
https://lucene.apache.org/solr/guide/6_6/the-extended-dismax-query-parser.html#TheExtendedDisMaxQueryParser-TheboostParameter
*bq*(boost query) - additive -
https://lucene.apache.org/solr/guide/6_6/the-dismax-query-parser.html#TheDisMaxQueryParser-Thebq_BoostQuery_Parameter
*bf*(boost function) - additive -
https://lucene.apache.org/solr/guide/6_6/the-dismax-query-parser.html#TheDisMaxQueryParser-Thebf_BoostFunctions_Parameter

This blog post is old but should help :
https://nolanlawson.com/2012/06/02/comparing-boost-methods-in-solr/

Then you can boost fields or even specific query clauses:

 1)
https://lucene.apache.org/solr/guide/6_6/the-dismax-query-parser.html#TheDisMaxQueryParser-Theqf_QueryFields_Parameter

2) q= features:2^1.0 AND features:3^5.0

1.0 is the default, you are multiplying the score contribution of the term
by 1.0, so no effect.
features:3^5.0 means that the score contribution of a match for the term
'3' in the field 'features' will be multiplied by 5.0 (you can also see
that enabling debug=results

Finally you can force the score contribution of a term to be a constant,
it's not recommended unless you are truly confident you don't need other
types of scoring:
q= features:2^=1.0 AND features:3^=5.0

in this example your document  id: 3 will have a score of 6.0

Not sure if this answers your question, if not feel free to elaborate more.

Cheers

--------------------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
www.sease.io


On Thu, 19 Mar 2020 at 11:18, Taisuke Miyazaki <[hidden email]>
wrote:

> I'm using Solr 7.5.0.
> I want to get boosted field and values per documents.
>
> e.g.
> documents:
>   id: 1, features: [1]
>   id: 2, features: [1,2]
>   id: 3, features: [1,2,3]
>
> query:
>   bq: features:2^1.0 AND features:3^1.0
>
> I expect results like below.
> boosted:
>   - id: 2
>     - field: features, value: 2
>   - id: 3
>     - field: features, value: 2
>     - field: features, value: 3
>
> I have an idea that set boost score like bit-flag, but it's not good I
> think because I must send query twice.
>
> bit-flag:
>   bq: features:2^2.0 AND features:3^4.0
>   docs:
>     - id: 1, score: 1.0(0x001)
>     - id: 2, score: 3.0(0x011) # have feature:2(2nd bit is 1)
>     - id: 3, score: 7.0(0x111) # have feature:2 and feature:3(2nd and 3rd
> bit are 1)
> check score value then I can get boosted field.
>
> Is there a better way?
>
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
Reply | Threaded
Open this post in threaded view
|

Re: How to get boosted field and values?

Taisuke Miyazaki
The blog looks like it's going to be useful from now on, so I'll take a
look.Thank you.

What I wanted, however, was a way to know what field was boosted as a
result.
But I couldn't find a way to do that, so instead I tried to get the field
and value out of the resulting score by putting a binary bit on the
field/value pair.
It doesn't really matter to me whether you do it additively or
multiplicatively, as it's good to know the field boosted as a result.

Do you see what I mean?


2020年3月20日(金) 18:56 Alessandro Benedetti <[hidden email]>:

> Hi Taisuke,
> there are various ways of approaching boosting and scoring in Apache Solr.
> First of all you must decide if you are interested in multiplicative or
> additive boost.
> Multiplicative will multiply the score of your search result by a certain
> factor while the additive will just add the factor to the final score.
>
> Using advanced query parsers such as the dismax and edismax you can use the
> :
> *boost* parameter - multiplicative - takes function in input -
>
> https://lucene.apache.org/solr/guide/6_6/the-extended-dismax-query-parser.html#TheExtendedDisMaxQueryParser-TheboostParameter
> *bq*(boost query) - additive -
>
> https://lucene.apache.org/solr/guide/6_6/the-dismax-query-parser.html#TheDisMaxQueryParser-Thebq_BoostQuery_Parameter
> *bf*(boost function) - additive -
>
> https://lucene.apache.org/solr/guide/6_6/the-dismax-query-parser.html#TheDisMaxQueryParser-Thebf_BoostFunctions_Parameter
>
> This blog post is old but should help :
> https://nolanlawson.com/2012/06/02/comparing-boost-methods-in-solr/
>
> Then you can boost fields or even specific query clauses:
>
>  1)
>
> https://lucene.apache.org/solr/guide/6_6/the-dismax-query-parser.html#TheDisMaxQueryParser-Theqf_QueryFields_Parameter
>
> 2) q= features:2^1.0 AND features:3^5.0
>
> 1.0 is the default, you are multiplying the score contribution of the term
> by 1.0, so no effect.
> features:3^5.0 means that the score contribution of a match for the term
> '3' in the field 'features' will be multiplied by 5.0 (you can also see
> that enabling debug=results
>
> Finally you can force the score contribution of a term to be a constant,
> it's not recommended unless you are truly confident you don't need other
> types of scoring:
> q= features:2^=1.0 AND features:3^=5.0
>
> in this example your document  id: 3 will have a score of 6.0
>
> Not sure if this answers your question, if not feel free to elaborate more.
>
> Cheers
>
> --------------------------
> Alessandro Benedetti
> Search Consultant, R&D Software Engineer, Director
> www.sease.io
>
>
> On Thu, 19 Mar 2020 at 11:18, Taisuke Miyazaki <[hidden email]
> >
> wrote:
>
> > I'm using Solr 7.5.0.
> > I want to get boosted field and values per documents.
> >
> > e.g.
> > documents:
> >   id: 1, features: [1]
> >   id: 2, features: [1,2]
> >   id: 3, features: [1,2,3]
> >
> > query:
> >   bq: features:2^1.0 AND features:3^1.0
> >
> > I expect results like below.
> > boosted:
> >   - id: 2
> >     - field: features, value: 2
> >   - id: 3
> >     - field: features, value: 2
> >     - field: features, value: 3
> >
> > I have an idea that set boost score like bit-flag, but it's not good I
> > think because I must send query twice.
> >
> > bit-flag:
> >   bq: features:2^2.0 AND features:3^4.0
> >   docs:
> >     - id: 1, score: 1.0(0x001)
> >     - id: 2, score: 3.0(0x011) # have feature:2(2nd bit is 1)
> >     - id: 3, score: 7.0(0x111) # have feature:2 and feature:3(2nd and 3rd
> > bit are 1)
> > check score value then I can get boosted field.
> >
> > Is there a better way?
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: How to get boosted field and values?

Yasufumi Mizoguchi
Hi,

I think "debug" query parameter or "explain" document transformer will help
you
to know which fields and query conditions are boosted.

https://lucene.apache.org/solr/guide/7_5/common-query-parameters.html
https://lucene.apache.org/solr/guide/7_5/transforming-result-documents.html

Thanks,
Yasufumi

2020年3月23日(月) 10:27 Taisuke Miyazaki <[hidden email]>:

> The blog looks like it's going to be useful from now on, so I'll take a
> look.Thank you.
>
> What I wanted, however, was a way to know what field was boosted as a
> result.
> But I couldn't find a way to do that, so instead I tried to get the field
> and value out of the resulting score by putting a binary bit on the
> field/value pair.
> It doesn't really matter to me whether you do it additively or
> multiplicatively, as it's good to know the field boosted as a result.
>
> Do you see what I mean?
>
>
> 2020年3月20日(金) 18:56 Alessandro Benedetti <[hidden email]>:
>
> > Hi Taisuke,
> > there are various ways of approaching boosting and scoring in Apache
> Solr.
> > First of all you must decide if you are interested in multiplicative or
> > additive boost.
> > Multiplicative will multiply the score of your search result by a certain
> > factor while the additive will just add the factor to the final score.
> >
> > Using advanced query parsers such as the dismax and edismax you can use
> the
> > :
> > *boost* parameter - multiplicative - takes function in input -
> >
> >
> https://lucene.apache.org/solr/guide/6_6/the-extended-dismax-query-parser.html#TheExtendedDisMaxQueryParser-TheboostParameter
> > *bq*(boost query) - additive -
> >
> >
> https://lucene.apache.org/solr/guide/6_6/the-dismax-query-parser.html#TheDisMaxQueryParser-Thebq_BoostQuery_Parameter
> > *bf*(boost function) - additive -
> >
> >
> https://lucene.apache.org/solr/guide/6_6/the-dismax-query-parser.html#TheDisMaxQueryParser-Thebf_BoostFunctions_Parameter
> >
> > This blog post is old but should help :
> > https://nolanlawson.com/2012/06/02/comparing-boost-methods-in-solr/
> >
> > Then you can boost fields or even specific query clauses:
> >
> >  1)
> >
> >
> https://lucene.apache.org/solr/guide/6_6/the-dismax-query-parser.html#TheDisMaxQueryParser-Theqf_QueryFields_Parameter
> >
> > 2) q= features:2^1.0 AND features:3^5.0
> >
> > 1.0 is the default, you are multiplying the score contribution of the
> term
> > by 1.0, so no effect.
> > features:3^5.0 means that the score contribution of a match for the term
> > '3' in the field 'features' will be multiplied by 5.0 (you can also see
> > that enabling debug=results
> >
> > Finally you can force the score contribution of a term to be a constant,
> > it's not recommended unless you are truly confident you don't need other
> > types of scoring:
> > q= features:2^=1.0 AND features:3^=5.0
> >
> > in this example your document  id: 3 will have a score of 6.0
> >
> > Not sure if this answers your question, if not feel free to elaborate
> more.
> >
> > Cheers
> >
> > --------------------------
> > Alessandro Benedetti
> > Search Consultant, R&D Software Engineer, Director
> > www.sease.io
> >
> >
> > On Thu, 19 Mar 2020 at 11:18, Taisuke Miyazaki <
> [hidden email]
> > >
> > wrote:
> >
> > > I'm using Solr 7.5.0.
> > > I want to get boosted field and values per documents.
> > >
> > > e.g.
> > > documents:
> > >   id: 1, features: [1]
> > >   id: 2, features: [1,2]
> > >   id: 3, features: [1,2,3]
> > >
> > > query:
> > >   bq: features:2^1.0 AND features:3^1.0
> > >
> > > I expect results like below.
> > > boosted:
> > >   - id: 2
> > >     - field: features, value: 2
> > >   - id: 3
> > >     - field: features, value: 2
> > >     - field: features, value: 3
> > >
> > > I have an idea that set boost score like bit-flag, but it's not good I
> > > think because I must send query twice.
> > >
> > > bit-flag:
> > >   bq: features:2^2.0 AND features:3^4.0
> > >   docs:
> > >     - id: 1, score: 1.0(0x001)
> > >     - id: 2, score: 3.0(0x011) # have feature:2(2nd bit is 1)
> > >     - id: 3, score: 7.0(0x111) # have feature:2 and feature:3(2nd and
> 3rd
> > > bit are 1)
> > > check score value then I can get boosted field.
> > >
> > > Is there a better way?
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: How to get boosted field and values?

Charlie Hull-3
In reply to this post by Taisuke Miyazaki
Try splainer.io - it parses the Debug output to show in detail how the
scores are calculated (disclaimer, I work for OSC who created it - but
it's free & open source of course ).

Charlie

On 23/03/2020 01:26, Taisuke Miyazaki wrote:

> The blog looks like it's going to be useful from now on, so I'll take a
> look.Thank you.
>
> What I wanted, however, was a way to know what field was boosted as a
> result.
> But I couldn't find a way to do that, so instead I tried to get the field
> and value out of the resulting score by putting a binary bit on the
> field/value pair.
> It doesn't really matter to me whether you do it additively or
> multiplicatively, as it's good to know the field boosted as a result.
>
> Do you see what I mean?
>
>
> 2020年3月20日(金) 18:56 Alessandro Benedetti <[hidden email]>:
>
>> Hi Taisuke,
>> there are various ways of approaching boosting and scoring in Apache Solr.
>> First of all you must decide if you are interested in multiplicative or
>> additive boost.
>> Multiplicative will multiply the score of your search result by a certain
>> factor while the additive will just add the factor to the final score.
>>
>> Using advanced query parsers such as the dismax and edismax you can use the
>> :
>> *boost* parameter - multiplicative - takes function in input -
>>
>> https://lucene.apache.org/solr/guide/6_6/the-extended-dismax-query-parser.html#TheExtendedDisMaxQueryParser-TheboostParameter
>> *bq*(boost query) - additive -
>>
>> https://lucene.apache.org/solr/guide/6_6/the-dismax-query-parser.html#TheDisMaxQueryParser-Thebq_BoostQuery_Parameter
>> *bf*(boost function) - additive -
>>
>> https://lucene.apache.org/solr/guide/6_6/the-dismax-query-parser.html#TheDisMaxQueryParser-Thebf_BoostFunctions_Parameter
>>
>> This blog post is old but should help :
>> https://nolanlawson.com/2012/06/02/comparing-boost-methods-in-solr/
>>
>> Then you can boost fields or even specific query clauses:
>>
>>   1)
>>
>> https://lucene.apache.org/solr/guide/6_6/the-dismax-query-parser.html#TheDisMaxQueryParser-Theqf_QueryFields_Parameter
>>
>> 2) q= features:2^1.0 AND features:3^5.0
>>
>> 1.0 is the default, you are multiplying the score contribution of the term
>> by 1.0, so no effect.
>> features:3^5.0 means that the score contribution of a match for the term
>> '3' in the field 'features' will be multiplied by 5.0 (you can also see
>> that enabling debug=results
>>
>> Finally you can force the score contribution of a term to be a constant,
>> it's not recommended unless you are truly confident you don't need other
>> types of scoring:
>> q= features:2^=1.0 AND features:3^=5.0
>>
>> in this example your document  id: 3 will have a score of 6.0
>>
>> Not sure if this answers your question, if not feel free to elaborate more.
>>
>> Cheers
>>
>> --------------------------
>> Alessandro Benedetti
>> Search Consultant, R&D Software Engineer, Director
>> www.sease.io
>>
>>
>> On Thu, 19 Mar 2020 at 11:18, Taisuke Miyazaki <[hidden email]
>> wrote:
>>
>>> I'm using Solr 7.5.0.
>>> I want to get boosted field and values per documents.
>>>
>>> e.g.
>>> documents:
>>>    id: 1, features: [1]
>>>    id: 2, features: [1,2]
>>>    id: 3, features: [1,2,3]
>>>
>>> query:
>>>    bq: features:2^1.0 AND features:3^1.0
>>>
>>> I expect results like below.
>>> boosted:
>>>    - id: 2
>>>      - field: features, value: 2
>>>    - id: 3
>>>      - field: features, value: 2
>>>      - field: features, value: 3
>>>
>>> I have an idea that set boost score like bit-flag, but it's not good I
>>> think because I must send query twice.
>>>
>>> bit-flag:
>>>    bq: features:2^2.0 AND features:3^4.0
>>>    docs:
>>>      - id: 1, score: 1.0(0x001)
>>>      - id: 2, score: 3.0(0x011) # have feature:2(2nd bit is 1)
>>>      - id: 3, score: 7.0(0x111) # have feature:2 and feature:3(2nd and 3rd
>>> bit are 1)
>>> check score value then I can get boosted field.
>>>
>>> Is there a better way?
>>>

--
Charlie Hull
OpenSource Connections, previously Flax

tel/fax: +44 (0)8700 118334
mobile:  +44 (0)7767 825828
web: www.o19s.com

Reply | Threaded
Open this post in threaded view
|

Re: How to get boosted field and values?

Taisuke Miyazaki
In reply to this post by Yasufumi Mizoguchi
Hi,

I knew I could get that information by using the "debug" query parameter,
but I want to use it in "production".

If I want to use it in "production", I have to write a plugin?


2020年3月25日(水) 9:03 Yasufumi Mizoguchi <[hidden email]>:

> Hi,
>
> I think "debug" query parameter or "explain" document transformer will help
> you
> to know which fields and query conditions are boosted.
>
> https://lucene.apache.org/solr/guide/7_5/common-query-parameters.html
> https://lucene.apache.org/solr/guide/7_5/transforming-result-documents.html
>
> Thanks,
> Yasufumi
>
> 2020年3月23日(月) 10:27 Taisuke Miyazaki <[hidden email]>:
>
> > The blog looks like it's going to be useful from now on, so I'll take a
> > look.Thank you.
> >
> > What I wanted, however, was a way to know what field was boosted as a
> > result.
> > But I couldn't find a way to do that, so instead I tried to get the field
> > and value out of the resulting score by putting a binary bit on the
> > field/value pair.
> > It doesn't really matter to me whether you do it additively or
> > multiplicatively, as it's good to know the field boosted as a result.
> >
> > Do you see what I mean?
> >
> >
> > 2020年3月20日(金) 18:56 Alessandro Benedetti <[hidden email]>:
> >
> > > Hi Taisuke,
> > > there are various ways of approaching boosting and scoring in Apache
> > Solr.
> > > First of all you must decide if you are interested in multiplicative or
> > > additive boost.
> > > Multiplicative will multiply the score of your search result by a
> certain
> > > factor while the additive will just add the factor to the final score.
> > >
> > > Using advanced query parsers such as the dismax and edismax you can use
> > the
> > > :
> > > *boost* parameter - multiplicative - takes function in input -
> > >
> > >
> >
> https://lucene.apache.org/solr/guide/6_6/the-extended-dismax-query-parser.html#TheExtendedDisMaxQueryParser-TheboostParameter
> > > *bq*(boost query) - additive -
> > >
> > >
> >
> https://lucene.apache.org/solr/guide/6_6/the-dismax-query-parser.html#TheDisMaxQueryParser-Thebq_BoostQuery_Parameter
> > > *bf*(boost function) - additive -
> > >
> > >
> >
> https://lucene.apache.org/solr/guide/6_6/the-dismax-query-parser.html#TheDisMaxQueryParser-Thebf_BoostFunctions_Parameter
> > >
> > > This blog post is old but should help :
> > > https://nolanlawson.com/2012/06/02/comparing-boost-methods-in-solr/
> > >
> > > Then you can boost fields or even specific query clauses:
> > >
> > >  1)
> > >
> > >
> >
> https://lucene.apache.org/solr/guide/6_6/the-dismax-query-parser.html#TheDisMaxQueryParser-Theqf_QueryFields_Parameter
> > >
> > > 2) q= features:2^1.0 AND features:3^5.0
> > >
> > > 1.0 is the default, you are multiplying the score contribution of the
> > term
> > > by 1.0, so no effect.
> > > features:3^5.0 means that the score contribution of a match for the
> term
> > > '3' in the field 'features' will be multiplied by 5.0 (you can also see
> > > that enabling debug=results
> > >
> > > Finally you can force the score contribution of a term to be a
> constant,
> > > it's not recommended unless you are truly confident you don't need
> other
> > > types of scoring:
> > > q= features:2^=1.0 AND features:3^=5.0
> > >
> > > in this example your document  id: 3 will have a score of 6.0
> > >
> > > Not sure if this answers your question, if not feel free to elaborate
> > more.
> > >
> > > Cheers
> > >
> > > --------------------------
> > > Alessandro Benedetti
> > > Search Consultant, R&D Software Engineer, Director
> > > www.sease.io
> > >
> > >
> > > On Thu, 19 Mar 2020 at 11:18, Taisuke Miyazaki <
> > [hidden email]
> > > >
> > > wrote:
> > >
> > > > I'm using Solr 7.5.0.
> > > > I want to get boosted field and values per documents.
> > > >
> > > > e.g.
> > > > documents:
> > > >   id: 1, features: [1]
> > > >   id: 2, features: [1,2]
> > > >   id: 3, features: [1,2,3]
> > > >
> > > > query:
> > > >   bq: features:2^1.0 AND features:3^1.0
> > > >
> > > > I expect results like below.
> > > > boosted:
> > > >   - id: 2
> > > >     - field: features, value: 2
> > > >   - id: 3
> > > >     - field: features, value: 2
> > > >     - field: features, value: 3
> > > >
> > > > I have an idea that set boost score like bit-flag, but it's not good
> I
> > > > think because I must send query twice.
> > > >
> > > > bit-flag:
> > > >   bq: features:2^2.0 AND features:3^4.0
> > > >   docs:
> > > >     - id: 1, score: 1.0(0x001)
> > > >     - id: 2, score: 3.0(0x011) # have feature:2(2nd bit is 1)
> > > >     - id: 3, score: 7.0(0x111) # have feature:2 and feature:3(2nd and
> > 3rd
> > > > bit are 1)
> > > > check score value then I can get boosted field.
> > > >
> > > > Is there a better way?
> > > >
> > >
> >
>