[jira] [Commented] (LUCENE-8950) FieldComparators Should Not Maintain Implicit PQs

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[jira] [Commented] (LUCENE-8950) FieldComparators Should Not Maintain Implicit PQs

Shalin Shekhar Mangar (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-8950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16907075#comment-16907075 ]

Atri Sharma commented on LUCENE-8950:

{quote}This looks like a duplicate of LUCENE-8878?
Not necessarily – 8878 targets refactoring the API to be simpler, whereas this Jira only targets removing the necessary condition that FieldComparators maintain their own priority queues. I believe this Jira compliments 8878.
{quote}I think all of us agree on the fact that it would be nice to have a simpler FieldComparator API. The challenge is that we don't want to trade too much efficiency. For instance the API you are proposing wouldn't work well with geo-distance sorting since it would require computing the actual distance for every new document, while the current implementation tries to be smart to first check a bounding box, and then compute a sort key that compares like the actual distance but is much cheaper to compute
Agreed, that is precisely why I suggested deprecating compare (slot, slot) instead of removing it completely. The idea is that comparators that require access to an internal PQ for whatever reasons are free to do so, but it should not be mandatory, and future comparators should not take on this dependency without understanding the tradeoffs

> FieldComparators Should Not Maintain Implicit PQs
> -------------------------------------------------
>                 Key: LUCENE-8950
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8950
>             Project: Lucene - Core
>          Issue Type: Improvement
>            Reporter: Atri Sharma
>            Priority: Major
> While doing some perf tests, I realised that FieldComparators inherently maintain implicit priority queues for maintaining the sorted order of documents for the given sort order. This is wasteful especially in the case of a multi feature sort order and a large number of hits requested.
> We should change this to have FieldComparators maintain only the top and bottom values, and use them as barriers to compare

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]