Parallelizing post filter for better performance

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Parallelizing post filter for better performance

catchsandy
Hello,

Is there a way to execute the post filter in a parallel mode so that
multiple query results can be filtered in parallel?

Right now, in our code, the post filter is becoming kind of bottleneck as
we had to do some post processing on every returned result, and it runs
serially in a single thread.

Thanks
Sundeep
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Parallelizing post filter for better performance

Joel Bernstein
You'll probably get better results by trying to get more performance out of
your single threaded postfilter. If you can post the code in you collect()
method you may get some ideas on how to improve the performance.

Joel Bernstein
http://joelsolr.blogspot.com/

On Fri, Mar 17, 2017 at 2:13 PM, Sundeep T <[hidden email]> wrote:

> Hello,
>
> Is there a way to execute the post filter in a parallel mode so that
> multiple query results can be filtered in parallel?
>
> Right now, in our code, the post filter is becoming kind of bottleneck as
> we had to do some post processing on every returned result, and it runs
> serially in a single thread.
>
> Thanks
> Sundeep
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Parallelizing post filter for better performance

Mikhail Khludnev-2
In reply to this post by catchsandy
Lucene can search segments in parallel. In Solr, you can break it to
multiple shards/cores and "distributeSearch" even on single Solr instance
(even without SolrCloud).

On Fri, Mar 17, 2017 at 9:13 PM, Sundeep T <[hidden email]> wrote:

> Hello,
>
> Is there a way to execute the post filter in a parallel mode so that
> multiple query results can be filtered in parallel?
>
> Right now, in our code, the post filter is becoming kind of bottleneck as
> we had to do some post processing on every returned result, and it runs
> serially in a single thread.
>
> Thanks
> Sundeep
>



--
Sincerely yours
Mikhail Khludnev
Loading...