Performance Improvement for Search using PriorityQueue

classic Classic list List threaded Threaded
41 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: Performance Improvement for Search using PriorityQueue

Shai Erera
Done (PriorityQueue-2.patch)

Shai

On Dec 12, 2007 1:46 PM, Michael McCandless <[hidden email]>
wrote:

>
> I think it's fine to include those changes with this issue.
>
> Mike
>
> Shai Erera wrote:
>
> > Hi
> >
> > I created https://issues.apache.org/jira/browse/LUCENE-1089 and
> > added a
> > patch.
> > I noticed that we can replace the calls to insert() with
> > insertWithOverflow() in several other places, like
> > QualityQueriesFinder,
> > FuzzyQuery and TopFieldDocCollector. I wasn't sure if that should
> > be handled
> > as part of this issue, or a different one.
> >
> > On Dec 11, 2007 8:32 PM, Yonik Seeley <[hidden email]> wrote:
> >
> >> On Dec 11, 2007 1:21 PM, Timo Nentwig <[hidden email]> wrote:
> >>> On Tuesday 11 December 2007 14:32:12 Shai Erera wrote:
> >>>> For (1) - I can't explain it but I've run into documents with
> >>>> 0.0fscores.
> >>>> For (2) - this is a simple logic - if the lowest score in the
> >>>> queue is
> >> 'x'
> >>>> and you want to top docs only, then there's no point in
> >>>> attempting to
> >>>> insert a document with score lower than 'x' (it will not be added).
> >>>
> >>> Sure. I didn't notice that score is passed as parameter and was
> >> surprised that
> >>> subsequent calls to collect() are supposed to be guaranteed to
> >>> have a
> >> lower
> >>> score.
> >>
> >> One is not guaranteed this... collect() generally goes in docid
> >> order,
> >> and scores are unordered.
> >>
> >> If you are only gathering the top 10 docs by score, you can compare
> >> the current score to the lowest of the top 10 you currently have to
> >> determine if you should bother inserting into the queue.
> >>
> >> -Yonik
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [hidden email]
> >> For additional commands, e-mail: [hidden email]
> >>
> >>
> >
> >
> > --
> > Regards,
> >
> > Shai Erera
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>


--
Regards,

Shai Erera
123