Document Score seen in debug section and in main results section dont match

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Document Score seen in debug section and in main results section dont match

Baloo
HI All,

Currently I am migrating  Solr 6.4.2 to Solr 7.4.  We pass multiple boost
queries (multiplicative boost queries, Solr's 'boost' parameter) to Solr
with each query. We have migrated all our custom components and solr
configurations to Solr 7.4.2 but during verification we have seen different
score in debug section which is not matching with the score that we see when
we return score field using fl parameter.

After spending half of my day analysing issue I found similar issue is
already open. Below issue is exactly same as what I am also observing with
Solr 7.4.  

https://issues.apache.org/jira/browse/SOLR-13126

Is there and solution/workaround available for this issue?
or going back to Solr 7.2.1 is the only solution - As per comments in above
issue (https://issues.apache.org/jira/browse/LUCENE-8099 these changes are
not there in 7.2.1)

Any help will be appreciated.




--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Document Score seen in debug section and in main results section dont match

Erick Erickson
No workarounds that I know of, but I have to ask:
"Why do you care?". This feels like an XY problem.
You are saying that "X" doesn't work, in this case the
scores are different in the debug section. But this
implies that there is a problem "Y" that you're having.

What is "Y"? What is the problem that these differing
scores is germane to? Perhaps there's another approach
that'll work to solve "Y" other than scores.

Best,
Erick

On Mon, Feb 11, 2019 at 6:04 AM Baloo <[hidden email]> wrote:

>
> HI All,
>
> Currently I am migrating  Solr 6.4.2 to Solr 7.4.  We pass multiple boost
> queries (multiplicative boost queries, Solr's 'boost' parameter) to Solr
> with each query. We have migrated all our custom components and solr
> configurations to Solr 7.4.2 but during verification we have seen different
> score in debug section which is not matching with the score that we see when
> we return score field using fl parameter.
>
> After spending half of my day analysing issue I found similar issue is
> already open. Below issue is exactly same as what I am also observing with
> Solr 7.4.
>
> https://issues.apache.org/jira/browse/SOLR-13126
>
> Is there and solution/workaround available for this issue?
> or going back to Solr 7.2.1 is the only solution - As per comments in above
> issue (https://issues.apache.org/jira/browse/LUCENE-8099 these changes are
> not there in 7.2.1)
>
> Any help will be appreciated.
>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Document Score seen in debug section and in main results section dont match

Baloo
Thanks Erick  to answer your question "What is "Y"?"
Score that we see in debug section actually looks correct and if we order
documents by that score we can get similar ranking of results that we were
getting for solr 6.4.2.

But With the score field that we get with each record it looks like boost
queries are getting ignored  (if there is more than one boost parameter to
solr, for single boost parameter it works fine, is something broken here
regarding multiple boost queries to solr)





--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Document Score seen in debug section and in main results section dont match

Baloo
I will try to provide more data about this issue,

If you see attached query response , It shows
>>  In 1st, 2nd  document only query matched and boost queries did not match
>>  In 3rd document boost query matched and total score of 3rd document
>> became 400.04 which is higher than 1st and 2nd document.

So I am wondering why Solr ignored score which is coming from boost query
section?  

Solr Response :
https://gist.github.com/baloo-solr/73add39132c8f3688a20d4f64fe93279




--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Document Score seen in debug section and in main results section dont match

Erick Erickson
Sounds like: https://issues.apache.org/jira/browse/SOLR-13126

On Tue, Feb 12, 2019 at 12:12 AM Baloo <[hidden email]> wrote:

>
> I will try to provide more data about this issue,
>
> If you see attached query response , It shows
> >>  In 1st, 2nd  document only query matched and boost queries did not match
> >>  In 3rd document boost query matched and total score of 3rd document
> >> became 400.04 which is higher than 1st and 2nd document.
>
> So I am wondering why Solr ignored score which is coming from boost query
> section?
>
> Solr Response :
> https://gist.github.com/baloo-solr/73add39132c8f3688a20d4f64fe93279
>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Document Score seen in debug section and in main results section dont match

Baloo
Thanks Erick,
We will stick to Solr 7.2.1 which works fine with multiple boost queries.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

RE: Document Score seen in debug section and in main results section dont match

Tobias Ibounig
In reply to this post by Erick Erickson
Hi Erick,

> You are saying that "X" doesn't work, in this case the scores are different in the debug section. But this implies that there is a problem "Y" that you're having.

The issue is not that they don't match, the issue is that the calculated score is wrong. The debug score shows everything as it should be, however the calculated score is only equal if none or all of the multiplicative boost queries match.

The jira issue has a misleading name. There is a testcase attached to reproduce the issue.

Best regards
Reply | Threaded
Open this post in threaded view
|

RE: Document Score seen in debug section and in main results section dont match

Tobias Ibounig
In reply to this post by Baloo
Hi Baloo,

> Is there and solution/workaround available for this issue?
> or going back to Solr 7.2.1 is the only solution - As per comments in above
> issue (https://issues.apache.org/jira/browse/LUCENE-8099 these changes are
> not there in 7.2.1)

The only workaround, I know of, is building SOLR yourself with the attached patch. Which we tested with 7.5 also in production.
https://issues.apache.org/jira/browse/SOLR-13126
"0001-use-deprecated-classes-to-fix-regression-introduced-.patch"

I was not able to rewrite the multiplicative boost queries in a way that did not trigger the regression.

Best regards