Is it possible to skip scoring completely?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Is it possible to skip scoring completely?

Ashwin Ramesh
Hi everybody,

I was wondering if there is a way we can tell solr (7.3+) to run none of
it's scoring logic. We would like to simply add a set of filter queries and
order on a specific docValue field.

e.g. "Give me all fq=color:red documents ORDER on popularityScore DESC"

Thanks in advance,

Ash

--
*P.S. We've launched a new blog to share the latest ideas and case studies
from our team. Check it out here: product.canva.com
<https://product.canva.com/>. ***
** <https://www.canva.com/>Empowering the
world to design
Also, we're hiring. Apply here!
<https://about.canva.com/careers/>
 <https://twitter.com/canva>
<https://facebook.com/canva> <https://au.linkedin.com/company/canva>
<https://twitter.com/canva>  <https://facebook.com/canva
<https://au.linkedin.com/company/canva>  <https://instagram.com/canva>






Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to skip scoring completely?

Shawn Heisey-2
On 9/11/2019 5:27 PM, Ashwin Ramesh wrote:
> I was wondering if there is a way we can tell solr (7.3+) to run none of
> it's scoring logic. We would like to simply add a set of filter queries and
> order on a specific docValue field.
>
> e.g. "Give me all fq=color:red documents ORDER on popularityScore DESC"

I don't know whether doing so actually tells Solr to completely skip the
scoring calculations, but you can easily make those calculations
irrelevant simply by including a sort parameter with your query that
references something other than score.

I would not expect skipping scoring to provide much in the way of a
performance enhancement even if specifying a sort does disable those
calculations.  Java can do math calculations VERY quickly, and as I
understand it, scoring is just math.

Thanks,
Shawn
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to skip scoring completely?

Emir Arnautović
In reply to this post by Ashwin Ramesh
Hi Ash,
I did not check the code, so not sure if your question is based on something that you find in the codebase or you are just assuming that scoring is called? I would assume differently: if you use only fq, then Solr does not have anything to score. Also, if you order by something other than score and do not request score to be returned, I would also assume that Solr will not calculate score. Again, didn’t have time to check the code, so these are just assumptions.

Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/



> On 12 Sep 2019, at 01:27, Ashwin Ramesh <[hidden email]> wrote:
>
> Hi everybody,
>
> I was wondering if there is a way we can tell solr (7.3+) to run none of
> it's scoring logic. We would like to simply add a set of filter queries and
> order on a specific docValue field.
>
> e.g. "Give me all fq=color:red documents ORDER on popularityScore DESC"
>
> Thanks in advance,
>
> Ash
>
> --
> *P.S. We've launched a new blog to share the latest ideas and case studies
> from our team. Check it out here: product.canva.com
> <https://product.canva.com/>. ***
> ** <https://www.canva.com/>Empowering the
> world to design
> Also, we're hiring. Apply here!
> <https://about.canva.com/careers/>
> <https://twitter.com/canva>
> <https://facebook.com/canva> <https://au.linkedin.com/company/canva>
> <https://twitter.com/canva>  <https://facebook.com/canva>  
> <https://au.linkedin.com/company/canva>  <https://instagram.com/canva>
>
>
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to skip scoring completely?

Ashwin Ramesh
Thanks Shawn & Emir,

I just tried a * query with filters with fl=id,score. I noticed that all
scores were 1.0. Which I assume means no scoring was done. When I added a
sort after that test, scores were still 1.0.

I guess all I have to do is set q=* & set a sort.

Appreciate your help,

Ash

On Thu, Sep 12, 2019 at 4:40 PM Emir Arnautović <
[hidden email]> wrote:

> Hi Ash,
> I did not check the code, so not sure if your question is based on
> something that you find in the codebase or you are just assuming that
> scoring is called? I would assume differently: if you use only fq, then
> Solr does not have anything to score. Also, if you order by something other
> than score and do not request score to be returned, I would also assume
> that Solr will not calculate score. Again, didn’t have time to check the
> code, so these are just assumptions.
>
> Emir
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>
>
>
> > On 12 Sep 2019, at 01:27, Ashwin Ramesh <[hidden email]> wrote:
> >
> > Hi everybody,
> >
> > I was wondering if there is a way we can tell solr (7.3+) to run none of
> > it's scoring logic. We would like to simply add a set of filter queries
> and
> > order on a specific docValue field.
> >
> > e.g. "Give me all fq=color:red documents ORDER on popularityScore DESC"
> >
> > Thanks in advance,
> >
> > Ash
> >
> > --
> > *P.S. We've launched a new blog to share the latest ideas and case
> studies
> > from our team. Check it out here: product.canva.com
> > <https://product.canva.com/>. ***
> > ** <https://www.canva.com/>Empowering the
> > world to design
> > Also, we're hiring. Apply here!
> > <https://about.canva.com/careers/>
> > <https://twitter.com/canva>
> > <https://facebook.com/canva> <https://au.linkedin.com/company/canva>
> > <https://twitter.com/canva>  <https://facebook.com/canva>
> > <https://au.linkedin.com/company/canva>  <https://instagram.com/canva>
> >
> >
> >
> >
> >
> >
>
>

--
*P.S. We've launched a new blog to share the latest ideas and case studies
from our team. Check it out here: product.canva.com
<https://product.canva.com/>. ***
** <https://www.canva.com/>Empowering the
world to design
Also, we're hiring. Apply here!
<https://about.canva.com/careers/>
 <https://twitter.com/canva>
<https://facebook.com/canva> <https://au.linkedin.com/company/canva>
<https://twitter.com/canva>  <https://facebook.com/canva
<https://au.linkedin.com/company/canva>  <https://instagram.com/canva>






Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to skip scoring completely?

Paras Lehana
Exactly, Ashwin, as Emir said, if you are not using any map or boosting
function, your scores should be same. But, in order to answer your question
to the point, here you can do to manipulate scoring:

q.alt=*:**^0*


In simple words, I have just added *^0* to your * query that you are
presently using. Well, this doesn't "tell solr not to run scoring" but
actually makes sure that whatever score solr has calculated should map to 0
for all documents. By doing this, you would be able to see the scores as 0
now.

On Thu, 12 Sep 2019 at 12:14, Ashwin Ramesh <[hidden email]> wrote:

> Thanks Shawn & Emir,
>
> I just tried a * query with filters with fl=id,score. I noticed that all
> scores were 1.0. Which I assume means no scoring was done. When I added a
> sort after that test, scores were still 1.0.
>
> I guess all I have to do is set q=* & set a sort.
>
> Appreciate your help,
>
> Ash
>
> On Thu, Sep 12, 2019 at 4:40 PM Emir Arnautović <
> [hidden email]> wrote:
>
> > Hi Ash,
> > I did not check the code, so not sure if your question is based on
> > something that you find in the codebase or you are just assuming that
> > scoring is called? I would assume differently: if you use only fq, then
> > Solr does not have anything to score. Also, if you order by something
> other
> > than score and do not request score to be returned, I would also assume
> > that Solr will not calculate score. Again, didn’t have time to check the
> > code, so these are just assumptions.
> >
> > Emir
> > --
> > Monitoring - Log Management - Alerting - Anomaly Detection
> > Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> >
> >
> >
> > > On 12 Sep 2019, at 01:27, Ashwin Ramesh <[hidden email]> wrote:
> > >
> > > Hi everybody,
> > >
> > > I was wondering if there is a way we can tell solr (7.3+) to run none
> of
> > > it's scoring logic. We would like to simply add a set of filter queries
> > and
> > > order on a specific docValue field.
> > >
> > > e.g. "Give me all fq=color:red documents ORDER on popularityScore DESC"
> > >
> > > Thanks in advance,
> > >
> > > Ash
> > >
> > > --
> > > *P.S. We've launched a new blog to share the latest ideas and case
> > studies
> > > from our team. Check it out here: product.canva.com
> > > <https://product.canva.com/>. ***
> > > ** <https://www.canva.com/>Empowering the
> > > world to design
> > > Also, we're hiring. Apply here!
> > > <https://about.canva.com/careers/>
> > > <https://twitter.com/canva>
> > > <https://facebook.com/canva> <https://au.linkedin.com/company/canva>
> > > <https://twitter.com/canva>  <https://facebook.com/canva>
> > > <https://au.linkedin.com/company/canva>  <https://instagram.com/canva>
> > >
> > >
> > >
> > >
> > >
> > >
> >
> >
>
> --
> *P.S. We've launched a new blog to share the latest ideas and case studies
> from our team. Check it out here: product.canva.com
> <https://product.canva.com/>. ***
> ** <https://www.canva.com/>Empowering the
> world to design
> Also, we're hiring. Apply here!
> <https://about.canva.com/careers/>
>  <https://twitter.com/canva>
> <https://facebook.com/canva> <https://au.linkedin.com/company/canva>
> <https://twitter.com/canva>  <https://facebook.com/canva>
> <https://au.linkedin.com/company/canva>  <https://instagram.com/canva>
>
>
>
>
>
>
>

--
--
Regards,

*Paras Lehana* [65871]
Software Programmer, Auto-Suggest,
IndiaMART Intermesh Ltd.

8th Floor, Tower A, Advant-Navis Business Park, Sector 142,
Noida, UP, IN - 201303

Mob.: +91-9560911996
Work: 01203916600 | Extn:  *8173*

--
IMPORTANT: 
NEVER share your IndiaMART OTP/ Password with anyone.
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to skip scoring completely?

Shawn Heisey
In reply to this post by Ashwin Ramesh
On 9/12/2019 12:43 AM, Ashwin Ramesh wrote:
> I just tried a * query with filters with fl=id,score. I noticed that all
> scores were 1.0. Which I assume means no scoring was done. When I added a
> sort after that test, scores were still 1.0.
>
> I guess all I have to do is set q=* & set a sort.

Don't use q=* for your query. This is a wildcard query.  What that means
is that if the field you're querying contains 10 million different
values, then your actual query will be built with all 10 million of
those values.  It will be huge, and VERY slow.

Use q=*:* if you mean all documents.  This is special syntax that Lucene
and Solr understand and translate into a very fast "all documents
query".  That query will probably also generate 1.0 for scores, though I
haven't checked.

Thanks,
Shawn
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to skip scoring completely?

Ashwin Ramesh
Ah! Thanks so much!

On Thu., 12 Sep. 2019, 11:56 pm Shawn Heisey, <[hidden email]> wrote:

> On 9/12/2019 12:43 AM, Ashwin Ramesh wrote:
> > I just tried a * query with filters with fl=id,score. I noticed that all
> > scores were 1.0. Which I assume means no scoring was done. When I added a
> > sort after that test, scores were still 1.0.
> >
> > I guess all I have to do is set q=* & set a sort.
>
> Don't use q=* for your query. This is a wildcard query.  What that means
> is that if the field you're querying contains 10 million different
> values, then your actual query will be built with all 10 million of
> those values.  It will be huge, and VERY slow.
>
> Use q=*:* if you mean all documents.  This is special syntax that Lucene
> and Solr understand and translate into a very fast "all documents
> query".  That query will probably also generate 1.0 for scores, though I
> haven't checked.
>
> Thanks,
> Shawn
>

--
*P.S. We've launched a new blog to share the latest ideas and case studies
from our team. Check it out here: product.canva.com
<https://product.canva.com/>. ***
** <https://www.canva.com/>Empowering the
world to design
Also, we're hiring. Apply here!
<https://about.canva.com/careers/>
 <https://twitter.com/canva>
<https://facebook.com/canva> <https://au.linkedin.com/company/canva>
<https://twitter.com/canva>  <https://facebook.com/canva
<https://au.linkedin.com/company/canva>  <https://instagram.com/canva>