Solr 7.6 query performace question

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Solr 7.6 query performace question

harjags
We are upgrading to solr 7.6 from 6.1
Our query has below pattern predominantly

1.q is * as we filter based on a department of products always
2. 100+ bq's to boost certain document
3. Collapsing using a non DocValue field
4.Many Facet Fields and Many Facet queries

Which of the above is the most memory consuming operations?

Below errors are very common in 7.6 and we have solr nodes failing with
tanking memory.

The request took too long to iterate over terms. Timeout: timeoutAt:
162874656583645 (System.nanoTime(): 162874701942020),
TermsEnum=org.apache.lucene.codecs.blocktree.SegmentTermsEnum@74507f4a

or

#*BitSetDocTopFilter*]; The request took too long to iterate over terms.
Timeout: timeoutAt: 33288640223586 (System.nanoTime(): 33288700895778),
TermsEnum=org.apache.lucene.codecs.blocktree.SegmentTermsEnum@5e458644


or

#SortedIntDocSetTopFilter]; The request took too long to iterate over terms.
Timeout: timeoutAt: 552497919389297 (System.nanoTime(): 552508251053558),
TermsEnum=org.apache.lucene.codecs.blocktree.SegmentTermsEnum@60b7186e


 
OR





--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Solr 7.6 query performace question

Erick Erickson
Well, It Depends (tm).

Certainly 2 and 3 are _not_ memory intensive

4 depends on the number of terms in the fields.

But I suspect your real problem has nothing to do with memory and is <1>. Try q=*:* rather than q=*. In case your e-mail tries to make things bold, that’s q=asterisk-colon-asterisk, not q=asterisk

Best,
Erick

> On Oct 13, 2019, at 12:59 AM, harjags <[hidden email]> wrote:
>
> We are upgrading to solr 7.6 from 6.1
> Our query has below pattern predominantly
>
> 1.q is * as we filter based on a department of products always
> 2. 100+ bq's to boost certain document
> 3. Collapsing using a non DocValue field
> 4.Many Facet Fields and Many Facet queries
>
> Which of the above is the most memory consuming operations?
>
> Below errors are very common in 7.6 and we have solr nodes failing with
> tanking memory.
>
> The request took too long to iterate over terms. Timeout: timeoutAt:
> 162874656583645 (System.nanoTime(): 162874701942020),
> TermsEnum=org.apache.lucene.codecs.blocktree.SegmentTermsEnum@74507f4a
>
> or
>
> #*BitSetDocTopFilter*]; The request took too long to iterate over terms.
> Timeout: timeoutAt: 33288640223586 (System.nanoTime(): 33288700895778),
> TermsEnum=org.apache.lucene.codecs.blocktree.SegmentTermsEnum@5e458644
>
>
> or
>
> #SortedIntDocSetTopFilter]; The request took too long to iterate over terms.
> Timeout: timeoutAt: 552497919389297 (System.nanoTime(): 552508251053558),
> TermsEnum=org.apache.lucene.codecs.blocktree.SegmentTermsEnum@60b7186e
>
>
>
> OR
>
>
>
>
>
> --
> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html