Lucene 5.5 with multi segment index

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Lucene 5.5 with multi segment index

Amit Phadke
Hey Guys,

I have a question on Query.rewrite() in Lucene 5.5
I am noticing that Query.rewrite() is actually more expensive than the actual search on a particular index with multiple segments (8 in my case).
Most of the rewrite cost seems to come from FuzzyQuery’s that need to be rewritten.

Tracing the code a bit, it looks like collectTerms() in TermCollectingRewrite seems to be going through all segments sequentially.
https://github.com/apache/lucene-solr/blob/releases/lucene-solr/5.5.0/lucene/core/src/java/org/apache/lucene/search/TermCollectingRewrite.java#L51 <https://github.com/apache/lucene-solr/blob/releases/lucene-solr/5.5.0/lucene/core/src/java/org/apache/lucene/search/TermCollectingRewrite.java#L51>

A lot of these methods are final, so this behavior cannot be easily overriden.
Can rewrite() be not parallelized for each segment, or am I missing something more obvious?

Thanks for your help.
Amit