Custom Solr Collector

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Custom Solr Collector

Kyle Maxwell
Hi,
Looking to see if there's any appetite for either:

1. Allowing custom collectors as Solr Plugins, or
2. Taking a patch on TimeLimitedCollector to allow it to be doc-limited as well.

Motivation:

TimeLimitedCollector Patch:

Which approach might people prefer?  I'm happy to do the legwork, but wanted to check in first.

Thanks,
Kyle
Reply | Threaded
Open this post in threaded view
|

Re: Custom Solr Collector

Tomás Fernández Löbbe
Hi Kyle,
For #2, I understand you need this because you want "min-visited-docs", right? Because, for max you could use EarlyTerminatingSortingCollector? (or Lucene's "HitsThresholdChecker", but I don't know if Solr has support for this yet). The "min-visited" would override the "timeAllowed", so even if the collection should expire based on time, you'd let it continue until something hits, is that the idea?

On Thu, Feb 13, 2020 at 9:29 AM Kyle Maxwell <[hidden email]> wrote:
Hi,
Looking to see if there's any appetite for either:

1. Allowing custom collectors as Solr Plugins, or
2. Taking a patch on TimeLimitedCollector to allow it to be doc-limited as well.

Motivation:

TimeLimitedCollector Patch:

Which approach might people prefer?  I'm happy to do the legwork, but wanted to check in first.

Thanks,
Kyle
Reply | Threaded
Open this post in threaded view
|

Re: Custom Solr Collector

Kyle Maxwell
You understand the min-visited part.

I don’t think the EarlyTerminatingSortingCollector is exactly what we want because we don’t want to sort the query results, just scan the index in roughly sorted order and score normally. 

Thankfully, a bunch of the features I’ve written custom collectors for over the years have made it into Solr by now, so maybe the full arbitrary collector configuration is overkill.

On Feb 13, 2020, at 11:42 AM, Tomás Fernández Löbbe <[hidden email]> wrote:


Hi Kyle,
For #2, I understand you need this because you want "min-visited-docs", right? Because, for max you could use EarlyTerminatingSortingCollector? (or Lucene's "HitsThresholdChecker", but I don't know if Solr has support for this yet). The "min-visited" would override the "timeAllowed", so even if the collection should expire based on time, you'd let it continue until something hits, is that the idea?

On Thu, Feb 13, 2020 at 9:29 AM Kyle Maxwell <[hidden email]> wrote:
Hi,
Looking to see if there's any appetite for either:

1. Allowing custom collectors as Solr Plugins, or
2. Taking a patch on TimeLimitedCollector to allow it to be doc-limited as well.

Motivation:

TimeLimitedCollector Patch:

Which approach might people prefer?  I'm happy to do the legwork, but wanted to check in first.

Thanks,
Kyle