Boosting only top n results that match a criteria

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Boosting only top n results that match a criteria

Nitin Arora-2
Hello, I have a complex solr query with various boosts applied that
returns, say a few hundred results. Out of these hundreds of results I want
to further boost, say the top 5 results that satisfy a particular criteria
- e.g. class=A1. So I want the top 5 results from class A1 in my existing
results set to come further higher, so that I can show them on the first
page of my final results. How do I achieve this?
I am new to SOLR and this community so apologies if this is trivial/repeat.

Thanks,
Nitin
Reply | Threaded
Open this post in threaded view
|

Re: Boosting only top n results that match a criteria

Emir Arnautović
Hi Nitin,
Can you simply filter and return top 5:

        ….&fq=class:A1&rows=5

Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/



> On 27 Dec 2019, at 13:55, Nitin Arora <[hidden email]> wrote:
>
> Hello, I have a complex solr query with various boosts applied that
> returns, say a few hundred results. Out of these hundreds of results I want
> to further boost, say the top 5 results that satisfy a particular criteria
> - e.g. class=A1. So I want the top 5 results from class A1 in my existing
> results set to come further higher, so that I can show them on the first
> page of my final results. How do I achieve this?
> I am new to SOLR and this community so apologies if this is trivial/repeat.
>
> Thanks,
> Nitin

Reply | Threaded
Open this post in threaded view
|

Re: Boosting only top n results that match a criteria

Erick Erickson
Have you seen RerankQParserPlugin?

Best,
Erick

> On Dec 27, 2019, at 8:49 AM, Emir Arnautović <[hidden email]> wrote:
>
> Hi Nitin,
> Can you simply filter and return top 5:
>
> ….&fq=class:A1&rows=5
>
> Emir
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>
>
>
>> On 27 Dec 2019, at 13:55, Nitin Arora <[hidden email]> wrote:
>>
>> Hello, I have a complex solr query with various boosts applied that
>> returns, say a few hundred results. Out of these hundreds of results I want
>> to further boost, say the top 5 results that satisfy a particular criteria
>> - e.g. class=A1. So I want the top 5 results from class A1 in my existing
>> results set to come further higher, so that I can show them on the first
>> page of my final results. How do I achieve this?
>> I am new to SOLR and this community so apologies if this is trivial/repeat.
>>
>> Thanks,
>> Nitin
>

Reply | Threaded
Open this post in threaded view
|

Re: Boosting only top n results that match a criteria

Nitin Arora-2
In reply to this post by Emir Arnautović
Hi Emir, This will filter my results to only include top 5 results from
class A1. I want the other results to stay as is while boosting only the
top 5 results of class A1.

On Fri, 27 Dec 2019 at 19:19, Emir Arnautović <[hidden email]>
wrote:

> Hi Nitin,
> Can you simply filter and return top 5:
>
>         ….&fq=class:A1&rows=5
>
> Emir
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>
>
>
> > On 27 Dec 2019, at 13:55, Nitin Arora <[hidden email]> wrote:
> >
> > Hello, I have a complex solr query with various boosts applied that
> > returns, say a few hundred results. Out of these hundreds of results I
> want
> > to further boost, say the top 5 results that satisfy a particular
> criteria
> > - e.g. class=A1. So I want the top 5 results from class A1 in my existing
> > results set to come further higher, so that I can show them on the first
> > page of my final results. How do I achieve this?
> > I am new to SOLR and this community so apologies if this is
> trivial/repeat.
> >
> > Thanks,
> > Nitin
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Boosting only top n results that match a criteria

Nitin Arora-2
In reply to this post by Erick Erickson
Hi Erick, I was not able to figure how exactly I will use
RerankQParserPlugin to achieve the desired reranking. I see that I can
rerank all the top RERANK_DOCS results - it is possible that they contain a
hundred results of class A1 or none. But the desired behaviour I want is to
pick (only) the top 5 results of class A1 from my potentially 100s of
results. Then boost them to first page.
Do you think this(or near this) behaviour is possible
using RerankQParserPlugin? Please shed more light how.

On Fri, 27 Dec 2019 at 19:48, Erick Erickson <[hidden email]>
wrote:

> Have you seen RerankQParserPlugin?
>
> Best,
> Erick
>
> > On Dec 27, 2019, at 8:49 AM, Emir Arnautović <
> [hidden email]> wrote:
> >
> > Hi Nitin,
> > Can you simply filter and return top 5:
> >
> >       ….&fq=class:A1&rows=5
> >
> > Emir
> > --
> > Monitoring - Log Management - Alerting - Anomaly Detection
> > Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> >
> >
> >
> >> On 27 Dec 2019, at 13:55, Nitin Arora <[hidden email]> wrote:
> >>
> >> Hello, I have a complex solr query with various boosts applied that
> >> returns, say a few hundred results. Out of these hundreds of results I
> want
> >> to further boost, say the top 5 results that satisfy a particular
> criteria
> >> - e.g. class=A1. So I want the top 5 results from class A1 in my
> existing
> >> results set to come further higher, so that I can show them on the first
> >> page of my final results. How do I achieve this?
> >> I am new to SOLR and this community so apologies if this is
> trivial/repeat.
> >>
> >> Thanks,
> >> Nitin
> >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Boosting only top n results that match a criteria

Walter Underwood
You could use two queries. Do the first with rows=5, then for the second use
an fq that filters out the IDs of those five. You’ll need to do the first query again
to do the second and further page of results statelessly, but that should still
be pretty fast.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)

> On Dec 27, 2019, at 9:57 AM, Nitin Arora <[hidden email]> wrote:
>
> Hi Erick, I was not able to figure how exactly I will use
> RerankQParserPlugin to achieve the desired reranking. I see that I can
> rerank all the top RERANK_DOCS results - it is possible that they contain a
> hundred results of class A1 or none. But the desired behaviour I want is to
> pick (only) the top 5 results of class A1 from my potentially 100s of
> results. Then boost them to first page.
> Do you think this(or near this) behaviour is possible
> using RerankQParserPlugin? Please shed more light how.
>
> On Fri, 27 Dec 2019 at 19:48, Erick Erickson <[hidden email]>
> wrote:
>
>> Have you seen RerankQParserPlugin?
>>
>> Best,
>> Erick
>>
>>> On Dec 27, 2019, at 8:49 AM, Emir Arnautović <
>> [hidden email]> wrote:
>>>
>>> Hi Nitin,
>>> Can you simply filter and return top 5:
>>>
>>>      ….&fq=class:A1&rows=5
>>>
>>> Emir
>>> --
>>> Monitoring - Log Management - Alerting - Anomaly Detection
>>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>>
>>>
>>>
>>>> On 27 Dec 2019, at 13:55, Nitin Arora <[hidden email]> wrote:
>>>>
>>>> Hello, I have a complex solr query with various boosts applied that
>>>> returns, say a few hundred results. Out of these hundreds of results I
>> want
>>>> to further boost, say the top 5 results that satisfy a particular
>> criteria
>>>> - e.g. class=A1. So I want the top 5 results from class A1 in my
>> existing
>>>> results set to come further higher, so that I can show them on the first
>>>> page of my final results. How do I achieve this?
>>>> I am new to SOLR and this community so apologies if this is
>> trivial/repeat.
>>>>
>>>> Thanks,
>>>> Nitin
>>>
>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Boosting only top n results that match a criteria

Erick Erickson
In reply to this post by Nitin Arora-2
Yes. Rerank essentially takes the top N results of one query and re-scores them through another query. So just boost the secondary query.

But you may not even have to do that. Just add a boost clause to a single query and boost your class A1 quite high. See “boost” and/or “bq”.

Best,
Erick

> On Dec 27, 2019, at 10:57 AM, Nitin Arora <[hidden email]> wrote:
>
> Hi Erick, I was not able to figure how exactly I will use
> RerankQParserPlugin to achieve the desired reranking. I see that I can
> rerank all the top RERANK_DOCS results - it is possible that they contain a
> hundred results of class A1 or none. But the desired behaviour I want is to
> pick (only) the top 5 results of class A1 from my potentially 100s of
> results. Then boost them to first page.
> Do you think this(or near this) behaviour is possible
> using RerankQParserPlugin? Please shed more light how.
>
> On Fri, 27 Dec 2019 at 19:48, Erick Erickson <[hidden email]>
> wrote:
>
>> Have you seen RerankQParserPlugin?
>>
>> Best,
>> Erick
>>
>>> On Dec 27, 2019, at 8:49 AM, Emir Arnautović <
>> [hidden email]> wrote:
>>>
>>> Hi Nitin,
>>> Can you simply filter and return top 5:
>>>
>>>      ….&fq=class:A1&rows=5
>>>
>>> Emir
>>> --
>>> Monitoring - Log Management - Alerting - Anomaly Detection
>>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>>
>>>
>>>
>>>> On 27 Dec 2019, at 13:55, Nitin Arora <[hidden email]> wrote:
>>>>
>>>> Hello, I have a complex solr query with various boosts applied that
>>>> returns, say a few hundred results. Out of these hundreds of results I
>> want
>>>> to further boost, say the top 5 results that satisfy a particular
>> criteria
>>>> - e.g. class=A1. So I want the top 5 results from class A1 in my
>> existing
>>>> results set to come further higher, so that I can show them on the first
>>>> page of my final results. How do I achieve this?
>>>> I am new to SOLR and this community so apologies if this is
>> trivial/repeat.
>>>>
>>>> Thanks,
>>>> Nitin
>>>
>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Boosting only top n results that match a criteria

Nitin Arora-2
In reply to this post by Walter Underwood
It sounds like this will work. Can we do it in a single query?
Something like:
<Original Query> OR (<Original Query> AND class=A1 AND rows=5)^high_boost
I am not good with SOLR syntax and need you to validate if such a query
will serve the purpose.

On Fri, 27 Dec 2019 at 22:10, Walter Underwood <[hidden email]>
wrote:

> You could use two queries. Do the first with rows=5, then for the second
> use
> an fq that filters out the IDs of those five. You’ll need to do the first
> query again
> to do the second and further page of results statelessly, but that should
> still
> be pretty fast.
>
> wunder
> Walter Underwood
> [hidden email]
> http://observer.wunderwood.org/  (my blog)
>
> > On Dec 27, 2019, at 9:57 AM, Nitin Arora <[hidden email]> wrote:
> >
> > Hi Erick, I was not able to figure how exactly I will use
> > RerankQParserPlugin to achieve the desired reranking. I see that I can
> > rerank all the top RERANK_DOCS results - it is possible that they
> contain a
> > hundred results of class A1 or none. But the desired behaviour I want is
> to
> > pick (only) the top 5 results of class A1 from my potentially 100s of
> > results. Then boost them to first page.
> > Do you think this(or near this) behaviour is possible
> > using RerankQParserPlugin? Please shed more light how.
> >
> > On Fri, 27 Dec 2019 at 19:48, Erick Erickson <[hidden email]>
> > wrote:
> >
> >> Have you seen RerankQParserPlugin?
> >>
> >> Best,
> >> Erick
> >>
> >>> On Dec 27, 2019, at 8:49 AM, Emir Arnautović <
> >> [hidden email]> wrote:
> >>>
> >>> Hi Nitin,
> >>> Can you simply filter and return top 5:
> >>>
> >>>      ….&fq=class:A1&rows=5
> >>>
> >>> Emir
> >>> --
> >>> Monitoring - Log Management - Alerting - Anomaly Detection
> >>> Solr & Elasticsearch Consulting Support Training -
> http://sematext.com/
> >>>
> >>>
> >>>
> >>>> On 27 Dec 2019, at 13:55, Nitin Arora <[hidden email]> wrote:
> >>>>
> >>>> Hello, I have a complex solr query with various boosts applied that
> >>>> returns, say a few hundred results. Out of these hundreds of results I
> >> want
> >>>> to further boost, say the top 5 results that satisfy a particular
> >> criteria
> >>>> - e.g. class=A1. So I want the top 5 results from class A1 in my
> >> existing
> >>>> results set to come further higher, so that I can show them on the
> first
> >>>> page of my final results. How do I achieve this?
> >>>> I am new to SOLR and this community so apologies if this is
> >> trivial/repeat.
> >>>>
> >>>> Thanks,
> >>>> Nitin
> >>>
> >>
> >>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Boosting only top n results that match a criteria

Nitin Arora-2
In reply to this post by Erick Erickson
Simply boosting on class A1 won't work since there may be many documents
from that class, all getting equal boost. I want only top 5 docs of that
class to get the boost.

On Fri, 27 Dec 2019 at 22:42, Erick Erickson <[hidden email]>
wrote:

> Yes. Rerank essentially takes the top N results of one query and re-scores
> them through another query. So just boost the secondary query.
>
> But you may not even have to do that. Just add a boost clause to a single
> query and boost your class A1 quite high. See “boost” and/or “bq”.
>
> Best,
> Erick
>
> > On Dec 27, 2019, at 10:57 AM, Nitin Arora <[hidden email]>
> wrote:
> >
> > Hi Erick, I was not able to figure how exactly I will use
> > RerankQParserPlugin to achieve the desired reranking. I see that I can
> > rerank all the top RERANK_DOCS results - it is possible that they
> contain a
> > hundred results of class A1 or none. But the desired behaviour I want is
> to
> > pick (only) the top 5 results of class A1 from my potentially 100s of
> > results. Then boost them to first page.
> > Do you think this(or near this) behaviour is possible
> > using RerankQParserPlugin? Please shed more light how.
> >
> > On Fri, 27 Dec 2019 at 19:48, Erick Erickson <[hidden email]>
> > wrote:
> >
> >> Have you seen RerankQParserPlugin?
> >>
> >> Best,
> >> Erick
> >>
> >>> On Dec 27, 2019, at 8:49 AM, Emir Arnautović <
> >> [hidden email]> wrote:
> >>>
> >>> Hi Nitin,
> >>> Can you simply filter and return top 5:
> >>>
> >>>      ….&fq=class:A1&rows=5
> >>>
> >>> Emir
> >>> --
> >>> Monitoring - Log Management - Alerting - Anomaly Detection
> >>> Solr & Elasticsearch Consulting Support Training -
> http://sematext.com/
> >>>
> >>>
> >>>
> >>>> On 27 Dec 2019, at 13:55, Nitin Arora <[hidden email]> wrote:
> >>>>
> >>>> Hello, I have a complex solr query with various boosts applied that
> >>>> returns, say a few hundred results. Out of these hundreds of results I
> >> want
> >>>> to further boost, say the top 5 results that satisfy a particular
> >> criteria
> >>>> - e.g. class=A1. So I want the top 5 results from class A1 in my
> >> existing
> >>>> results set to come further higher, so that I can show them on the
> first
> >>>> page of my final results. How do I achieve this?
> >>>> I am new to SOLR and this community so apologies if this is
> >> trivial/repeat.
> >>>>
> >>>> Thanks,
> >>>> Nitin
> >>>
> >>
> >>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Boosting only top n results that match a criteria

Emir Arnautović
You could try and see if field collapsing can help you. That could let you return top 5 from each class if that is acceptable. Otherwise, you’ll have to go with two queries.

HTH,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/



> On 27 Dec 2019, at 19:08, Nitin Arora <[hidden email]> wrote:
>
> Simply boosting on class A1 won't work since there may be many documents
> from that class, all getting equal boost. I want only top 5 docs of that
> class to get the boost.
>
> On Fri, 27 Dec 2019 at 22:42, Erick Erickson <[hidden email]>
> wrote:
>
>> Yes. Rerank essentially takes the top N results of one query and re-scores
>> them through another query. So just boost the secondary query.
>>
>> But you may not even have to do that. Just add a boost clause to a single
>> query and boost your class A1 quite high. See “boost” and/or “bq”.
>>
>> Best,
>> Erick
>>
>>> On Dec 27, 2019, at 10:57 AM, Nitin Arora <[hidden email]>
>> wrote:
>>>
>>> Hi Erick, I was not able to figure how exactly I will use
>>> RerankQParserPlugin to achieve the desired reranking. I see that I can
>>> rerank all the top RERANK_DOCS results - it is possible that they
>> contain a
>>> hundred results of class A1 or none. But the desired behaviour I want is
>> to
>>> pick (only) the top 5 results of class A1 from my potentially 100s of
>>> results. Then boost them to first page.
>>> Do you think this(or near this) behaviour is possible
>>> using RerankQParserPlugin? Please shed more light how.
>>>
>>> On Fri, 27 Dec 2019 at 19:48, Erick Erickson <[hidden email]>
>>> wrote:
>>>
>>>> Have you seen RerankQParserPlugin?
>>>>
>>>> Best,
>>>> Erick
>>>>
>>>>> On Dec 27, 2019, at 8:49 AM, Emir Arnautović <
>>>> [hidden email]> wrote:
>>>>>
>>>>> Hi Nitin,
>>>>> Can you simply filter and return top 5:
>>>>>
>>>>>     ….&fq=class:A1&rows=5
>>>>>
>>>>> Emir
>>>>> --
>>>>> Monitoring - Log Management - Alerting - Anomaly Detection
>>>>> Solr & Elasticsearch Consulting Support Training -
>> http://sematext.com/
>>>>>
>>>>>
>>>>>
>>>>>> On 27 Dec 2019, at 13:55, Nitin Arora <[hidden email]> wrote:
>>>>>>
>>>>>> Hello, I have a complex solr query with various boosts applied that
>>>>>> returns, say a few hundred results. Out of these hundreds of results I
>>>> want
>>>>>> to further boost, say the top 5 results that satisfy a particular
>>>> criteria
>>>>>> - e.g. class=A1. So I want the top 5 results from class A1 in my
>>>> existing
>>>>>> results set to come further higher, so that I can show them on the
>> first
>>>>>> page of my final results. How do I achieve this?
>>>>>> I am new to SOLR and this community so apologies if this is
>>>> trivial/repeat.
>>>>>>
>>>>>> Thanks,
>>>>>> Nitin
>>>>>
>>>>
>>>>
>>
>>