Difference in fieldLengh and avgFieldLength in Solr 6.6 vs Solr 7.1

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Difference in fieldLengh and avgFieldLength in Solr 6.6 vs Solr 7.1

rupali pol
Hi all,

We are doing upgrade from Solr 6.6 to Solr 7.1, we are seeing lot of
differneces in raking and scores of Solr 6.6 and Solr7.1 results.

The major differences we observed are in fieldLengh and avgFieldLength
parameters which are calculated per field, per document, per search term.

*Calculation of tfNorm in Solr 7.1.0 -*
tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - b + b *
fieldLength / avgFieldLength)) from:
fieldLength *53272.000000000* 4087877% 4087877%
avgFieldLength *7284.331000000* 558970% 558970%
termFreq=10.0 10.000000000 767% 767%
parameter k1 1.200000000 92% 92%
parameter b 0.750000000 58% 58%

*Calculation of tfNorm for same in Solr 6.6.0 -*
tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - b + b *
fieldLength / avgFieldLength)) from:
fieldLength *65536.000000000* 5480182% 5480182%
avgFieldLength *7284.830600000* 609164% 609164%
termFreq=10.0 10.000000000 836% 836%
parameter k1 1.200000000 100% 100%
parameter b 0.750000000 63% 63%


Can someone please elaborate on what differences are brought in Solr7.1 for
the fieldLength calculation?

Thanks in advance.

Best,
Rups
Reply | Threaded
Open this post in threaded view
|

Re: Difference in fieldLengh and avgFieldLength in Solr 6.6 vs Solr 7.1

Alessandro Benedetti
Shoot in the dark, I have not double checked in details but :

With Solr 7.x
"Index-time boosts have been removed from Lucene, and are no longer
available from Solr. If any boosts are provided, they will be ignored by the
indexing chain. As a replacement, index-time scoring factors should be
indexed in a separate field and combined with the query score using a
function query. See the section Function Queries for more information."

Are you using index time boost by any chance ?
If I remember correctly the Norms stored in the segment were affected by the
field length and index time boost.

Cheers



-----
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io