Range queries with Grouping is slow?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Range queries with Grouping is slow?

Kranti Parisa
Is there any known issue with Range queries + grouping?

Case1:
q=id:123&group=true&sort=price
asc&group.field=entityId&group.limit=2&group.ngroups=true

Case2:
q=id:123 AND price:[* TO *]&group=true&sort=price
asc&group.field=entityId&group.limit=2&group.ngroups=true

Index Size:10M/~5GB
After running both queries at least once, I was expecting to hit the query
caches and response should be quick enough, but
Case1: 15-20ms (looks fine)
Case2: 400+ms (this seems constantly >400ms even after the first query)

any thought? if it's a known issue, please point me to the jira link
otherwise I can open an issue if this needs some analysis?


Thanks,
Kranti K. Parisa
http://www.linkedin.com/in/krantiparisa
Reply | Threaded
Open this post in threaded view
|

RE: Range queries with Grouping is slow?

David Smiley
Kranti,

I can't speak to the specific slow-down while grouping, but if you expect to run [* TO *] queries with any frequency then you should index a boolean flag and query for that instead.  You might also reduce the precisionStep value for the field you are using to 6 or even 4.  But wow that's a big difference you noted; it wouldn't hurt to double-check with the debugger that the [* TO *] is treated as a numeric range query instead of a generic term range.

~ David
________________________________________
From: Kranti Parisa [[hidden email]]
Sent: Tuesday, January 07, 2014 10:26 PM
To: [hidden email]
Subject: Range queries with Grouping is slow?

Is there any known issue with Range queries + grouping?

Case1:
q=id:123&group=true&sort=price
asc&group.field=entityId&group.limit=2&group.ngroups=true

Case2:
q=id:123 AND price:[* TO *]&group=true&sort=price
asc&group.field=entityId&group.limit=2&group.ngroups=true

Index Size:10M/~5GB
After running both queries at least once, I was expecting to hit the query
caches and response should be quick enough, but
Case1: 15-20ms (looks fine)
Case2: 400+ms (this seems constantly >400ms even after the first query)

any thought? if it's a known issue, please point me to the jira link
otherwise I can open an issue if this needs some analysis?


Thanks,
Kranti K. Parisa
http://www.linkedin.com/in/krantiparisa
Reply | Threaded
Open this post in threaded view
|

Re: Range queries with Grouping is slow?

Joel Bernstein
Kranti,

The range query also looks like a good candidate to be moved to a filter
query so it can be cached.

Joel Bernstein
Search Engineer at Heliosearch


On Tue, Jan 7, 2014 at 11:34 PM, Smiley, David W. <[hidden email]> wrote:

> Kranti,
>
> I can't speak to the specific slow-down while grouping, but if you expect
> to run [* TO *] queries with any frequency then you should index a boolean
> flag and query for that instead.  You might also reduce the precisionStep
> value for the field you are using to 6 or even 4.  But wow that's a big
> difference you noted; it wouldn't hurt to double-check with the debugger
> that the [* TO *] is treated as a numeric range query instead of a generic
> term range.
>
> ~ David
> ________________________________________
> From: Kranti Parisa [[hidden email]]
> Sent: Tuesday, January 07, 2014 10:26 PM
> To: [hidden email]
> Subject: Range queries with Grouping is slow?
>
> Is there any known issue with Range queries + grouping?
>
> Case1:
> q=id:123&group=true&sort=price
> asc&group.field=entityId&group.limit=2&group.ngroups=true
>
> Case2:
> q=id:123 AND price:[* TO *]&group=true&sort=price
> asc&group.field=entityId&group.limit=2&group.ngroups=true
>
> Index Size:10M/~5GB
> After running both queries at least once, I was expecting to hit the query
> caches and response should be quick enough, but
> Case1: 15-20ms (looks fine)
> Case2: 400+ms (this seems constantly >400ms even after the first query)
>
> any thought? if it's a known issue, please point me to the jira link
> otherwise I can open an issue if this needs some analysis?
>
>
> Thanks,
> Kranti K. Parisa
> http://www.linkedin.com/in/krantiparisa
>
Reply | Threaded
Open this post in threaded view
|

Re: Range queries with Grouping is slow?

Kranti Parisa
I was trying with the  [* TO *] as an example, the real use case is OR
query between 2/more range queries of timestamp fields (saved in
milliseconds). So I can't use FQs as they are ANDed by definition.

Am I missing something here?




Thanks,
Kranti K. Parisa
http://www.linkedin.com/in/krantiparisa



On Wed, Jan 8, 2014 at 8:15 AM, Joel Bernstein <[hidden email]> wrote:

> Kranti,
>
> The range query also looks like a good candidate to be moved to a filter
> query so it can be cached.
>
> Joel Bernstein
> Search Engineer at Heliosearch
>
>
> On Tue, Jan 7, 2014 at 11:34 PM, Smiley, David W. <[hidden email]>
> wrote:
>
> > Kranti,
> >
> > I can't speak to the specific slow-down while grouping, but if you expect
> > to run [* TO *] queries with any frequency then you should index a
> boolean
> > flag and query for that instead.  You might also reduce the precisionStep
> > value for the field you are using to 6 or even 4.  But wow that's a big
> > difference you noted; it wouldn't hurt to double-check with the debugger
> > that the [* TO *] is treated as a numeric range query instead of a
> generic
> > term range.
> >
> > ~ David
> > ________________________________________
> > From: Kranti Parisa [[hidden email]]
> > Sent: Tuesday, January 07, 2014 10:26 PM
> > To: [hidden email]
> > Subject: Range queries with Grouping is slow?
> >
> > Is there any known issue with Range queries + grouping?
> >
> > Case1:
> > q=id:123&group=true&sort=price
> > asc&group.field=entityId&group.limit=2&group.ngroups=true
> >
> > Case2:
> > q=id:123 AND price:[* TO *]&group=true&sort=price
> > asc&group.field=entityId&group.limit=2&group.ngroups=true
> >
> > Index Size:10M/~5GB
> > After running both queries at least once, I was expecting to hit the
> query
> > caches and response should be quick enough, but
> > Case1: 15-20ms (looks fine)
> > Case2: 400+ms (this seems constantly >400ms even after the first query)
> >
> > any thought? if it's a known issue, please point me to the jira link
> > otherwise I can open an issue if this needs some analysis?
> >
> >
> > Thanks,
> > Kranti K. Parisa
> > http://www.linkedin.com/in/krantiparisa
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Range queries with Grouping is slow?

Erick Erickson
Well, actually you can use fqs, it's just that re-using them becomes a bit
more tricky. Specifically,
fq=field1:blah OR field2:blort
is perfectly reasonable. However, it doesn't break things down into
sub-clauses, so
fq=field1:blah
will create a new entry in the filtercache. And
fq=field2:blort OR field1:blah
will not match the first one.

It kind of depends on the query pattern whether the filtercache will be
re-used, you have to take care to construct the fq clauses with re-use in
mind if you want ORs.

Best,
Erick


On Wed, Jan 8, 2014 at 11:56 AM, Kranti Parisa <[hidden email]>wrote:

> I was trying with the  [* TO *] as an example, the real use case is OR
> query between 2/more range queries of timestamp fields (saved in
> milliseconds). So I can't use FQs as they are ANDed by definition.
>
> Am I missing something here?
>
>
>
>
> Thanks,
> Kranti K. Parisa
> http://www.linkedin.com/in/krantiparisa
>
>
>
> On Wed, Jan 8, 2014 at 8:15 AM, Joel Bernstein <[hidden email]> wrote:
>
> > Kranti,
> >
> > The range query also looks like a good candidate to be moved to a filter
> > query so it can be cached.
> >
> > Joel Bernstein
> > Search Engineer at Heliosearch
> >
> >
> > On Tue, Jan 7, 2014 at 11:34 PM, Smiley, David W. <[hidden email]>
> > wrote:
> >
> > > Kranti,
> > >
> > > I can't speak to the specific slow-down while grouping, but if you
> expect
> > > to run [* TO *] queries with any frequency then you should index a
> > boolean
> > > flag and query for that instead.  You might also reduce the
> precisionStep
> > > value for the field you are using to 6 or even 4.  But wow that's a big
> > > difference you noted; it wouldn't hurt to double-check with the
> debugger
> > > that the [* TO *] is treated as a numeric range query instead of a
> > generic
> > > term range.
> > >
> > > ~ David
> > > ________________________________________
> > > From: Kranti Parisa [[hidden email]]
> > > Sent: Tuesday, January 07, 2014 10:26 PM
> > > To: [hidden email]
> > > Subject: Range queries with Grouping is slow?
> > >
> > > Is there any known issue with Range queries + grouping?
> > >
> > > Case1:
> > > q=id:123&group=true&sort=price
> > > asc&group.field=entityId&group.limit=2&group.ngroups=true
> > >
> > > Case2:
> > > q=id:123 AND price:[* TO *]&group=true&sort=price
> > > asc&group.field=entityId&group.limit=2&group.ngroups=true
> > >
> > > Index Size:10M/~5GB
> > > After running both queries at least once, I was expecting to hit the
> > query
> > > caches and response should be quick enough, but
> > > Case1: 15-20ms (looks fine)
> > > Case2: 400+ms (this seems constantly >400ms even after the first query)
> > >
> > > any thought? if it's a known issue, please point me to the jira link
> > > otherwise I can open an issue if this needs some analysis?
> > >
> > >
> > > Thanks,
> > > Kranti K. Parisa
> > > http://www.linkedin.com/in/krantiparisa
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Range queries with Grouping is slow?

Kranti Parisa
yes thats the key, these time ranges change frequently and hitting
filtercache then is a problem. I will try few more samples and probably
debug thru it. thanks.


Thanks,
Kranti K. Parisa
http://www.linkedin.com/in/krantiparisa



On Wed, Jan 8, 2014 at 12:11 PM, Erick Erickson <[hidden email]>wrote:

> Well, actually you can use fqs, it's just that re-using them becomes a bit
> more tricky. Specifically,
> fq=field1:blah OR field2:blort
> is perfectly reasonable. However, it doesn't break things down into
> sub-clauses, so
> fq=field1:blah
> will create a new entry in the filtercache. And
> fq=field2:blort OR field1:blah
> will not match the first one.
>
> It kind of depends on the query pattern whether the filtercache will be
> re-used, you have to take care to construct the fq clauses with re-use in
> mind if you want ORs.
>
> Best,
> Erick
>
>
> On Wed, Jan 8, 2014 at 11:56 AM, Kranti Parisa <[hidden email]
> >wrote:
>
> > I was trying with the  [* TO *] as an example, the real use case is OR
> > query between 2/more range queries of timestamp fields (saved in
> > milliseconds). So I can't use FQs as they are ANDed by definition.
> >
> > Am I missing something here?
> >
> >
> >
> >
> > Thanks,
> > Kranti K. Parisa
> > http://www.linkedin.com/in/krantiparisa
> >
> >
> >
> > On Wed, Jan 8, 2014 at 8:15 AM, Joel Bernstein <[hidden email]>
> wrote:
> >
> > > Kranti,
> > >
> > > The range query also looks like a good candidate to be moved to a
> filter
> > > query so it can be cached.
> > >
> > > Joel Bernstein
> > > Search Engineer at Heliosearch
> > >
> > >
> > > On Tue, Jan 7, 2014 at 11:34 PM, Smiley, David W. <[hidden email]>
> > > wrote:
> > >
> > > > Kranti,
> > > >
> > > > I can't speak to the specific slow-down while grouping, but if you
> > expect
> > > > to run [* TO *] queries with any frequency then you should index a
> > > boolean
> > > > flag and query for that instead.  You might also reduce the
> > precisionStep
> > > > value for the field you are using to 6 or even 4.  But wow that's a
> big
> > > > difference you noted; it wouldn't hurt to double-check with the
> > debugger
> > > > that the [* TO *] is treated as a numeric range query instead of a
> > > generic
> > > > term range.
> > > >
> > > > ~ David
> > > > ________________________________________
> > > > From: Kranti Parisa [[hidden email]]
> > > > Sent: Tuesday, January 07, 2014 10:26 PM
> > > > To: [hidden email]
> > > > Subject: Range queries with Grouping is slow?
> > > >
> > > > Is there any known issue with Range queries + grouping?
> > > >
> > > > Case1:
> > > > q=id:123&group=true&sort=price
> > > > asc&group.field=entityId&group.limit=2&group.ngroups=true
> > > >
> > > > Case2:
> > > > q=id:123 AND price:[* TO *]&group=true&sort=price
> > > > asc&group.field=entityId&group.limit=2&group.ngroups=true
> > > >
> > > > Index Size:10M/~5GB
> > > > After running both queries at least once, I was expecting to hit the
> > > query
> > > > caches and response should be quick enough, but
> > > > Case1: 15-20ms (looks fine)
> > > > Case2: 400+ms (this seems constantly >400ms even after the first
> query)
> > > >
> > > > any thought? if it's a known issue, please point me to the jira link
> > > > otherwise I can open an issue if this needs some analysis?
> > > >
> > > >
> > > > Thanks,
> > > > Kranti K. Parisa
> > > > http://www.linkedin.com/in/krantiparisa
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Range queries with Grouping is slow?

Erick Erickson
Yeah, time fqs in particular can be tricky to re-use if you specify NOW,
see:
http://searchhub.org/2012/02/23/date-math-now-and-filter-queries/

Best,
Erick



On Wed, Jan 8, 2014 at 12:18 PM, Kranti Parisa <[hidden email]>wrote:

> yes thats the key, these time ranges change frequently and hitting
> filtercache then is a problem. I will try few more samples and probably
> debug thru it. thanks.
>
>
> Thanks,
> Kranti K. Parisa
> http://www.linkedin.com/in/krantiparisa
>
>
>
> On Wed, Jan 8, 2014 at 12:11 PM, Erick Erickson <[hidden email]
> >wrote:
>
> > Well, actually you can use fqs, it's just that re-using them becomes a
> bit
> > more tricky. Specifically,
> > fq=field1:blah OR field2:blort
> > is perfectly reasonable. However, it doesn't break things down into
> > sub-clauses, so
> > fq=field1:blah
> > will create a new entry in the filtercache. And
> > fq=field2:blort OR field1:blah
> > will not match the first one.
> >
> > It kind of depends on the query pattern whether the filtercache will be
> > re-used, you have to take care to construct the fq clauses with re-use in
> > mind if you want ORs.
> >
> > Best,
> > Erick
> >
> >
> > On Wed, Jan 8, 2014 at 11:56 AM, Kranti Parisa <[hidden email]
> > >wrote:
> >
> > > I was trying with the  [* TO *] as an example, the real use case is OR
> > > query between 2/more range queries of timestamp fields (saved in
> > > milliseconds). So I can't use FQs as they are ANDed by definition.
> > >
> > > Am I missing something here?
> > >
> > >
> > >
> > >
> > > Thanks,
> > > Kranti K. Parisa
> > > http://www.linkedin.com/in/krantiparisa
> > >
> > >
> > >
> > > On Wed, Jan 8, 2014 at 8:15 AM, Joel Bernstein <[hidden email]>
> > wrote:
> > >
> > > > Kranti,
> > > >
> > > > The range query also looks like a good candidate to be moved to a
> > filter
> > > > query so it can be cached.
> > > >
> > > > Joel Bernstein
> > > > Search Engineer at Heliosearch
> > > >
> > > >
> > > > On Tue, Jan 7, 2014 at 11:34 PM, Smiley, David W. <[hidden email]
> >
> > > > wrote:
> > > >
> > > > > Kranti,
> > > > >
> > > > > I can't speak to the specific slow-down while grouping, but if you
> > > expect
> > > > > to run [* TO *] queries with any frequency then you should index a
> > > > boolean
> > > > > flag and query for that instead.  You might also reduce the
> > > precisionStep
> > > > > value for the field you are using to 6 or even 4.  But wow that's a
> > big
> > > > > difference you noted; it wouldn't hurt to double-check with the
> > > debugger
> > > > > that the [* TO *] is treated as a numeric range query instead of a
> > > > generic
> > > > > term range.
> > > > >
> > > > > ~ David
> > > > > ________________________________________
> > > > > From: Kranti Parisa [[hidden email]]
> > > > > Sent: Tuesday, January 07, 2014 10:26 PM
> > > > > To: [hidden email]
> > > > > Subject: Range queries with Grouping is slow?
> > > > >
> > > > > Is there any known issue with Range queries + grouping?
> > > > >
> > > > > Case1:
> > > > > q=id:123&group=true&sort=price
> > > > > asc&group.field=entityId&group.limit=2&group.ngroups=true
> > > > >
> > > > > Case2:
> > > > > q=id:123 AND price:[* TO *]&group=true&sort=price
> > > > > asc&group.field=entityId&group.limit=2&group.ngroups=true
> > > > >
> > > > > Index Size:10M/~5GB
> > > > > After running both queries at least once, I was expecting to hit
> the
> > > > query
> > > > > caches and response should be quick enough, but
> > > > > Case1: 15-20ms (looks fine)
> > > > > Case2: 400+ms (this seems constantly >400ms even after the first
> > query)
> > > > >
> > > > > any thought? if it's a known issue, please point me to the jira
> link
> > > > > otherwise I can open an issue if this needs some analysis?
> > > > >
> > > > >
> > > > > Thanks,
> > > > > Kranti K. Parisa
> > > > > http://www.linkedin.com/in/krantiparisa
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Range queries with Grouping is slow?

David Smiley
In reply to this post by Kranti Parisa
It won┬╣t hit the filter cache if you set {! cache=false} local-param.

On 1/8/14, 12:18 PM, "Kranti Parisa" <[hidden email]> wrote:

>yes thats the key, these time ranges change frequently and hitting
>filtercache then is a problem. I will try few more samples and probably
>debug thru it. thanks.
>
>
>Thanks,
>Kranti K. Parisa
>http://www.linkedin.com/in/krantiparisa
>
>
>
>On Wed, Jan 8, 2014 at 12:11 PM, Erick Erickson
><[hidden email]>wrote:
>
>> Well, actually you can use fqs, it's just that re-using them becomes a
>>bit
>> more tricky. Specifically,
>> fq=field1:blah OR field2:blort
>> is perfectly reasonable. However, it doesn't break things down into
>> sub-clauses, so
>> fq=field1:blah
>> will create a new entry in the filtercache. And
>> fq=field2:blort OR field1:blah
>> will not match the first one.
>>
>> It kind of depends on the query pattern whether the filtercache will be
>> re-used, you have to take care to construct the fq clauses with re-use
>>in
>> mind if you want ORs.
>>
>> Best,
>> Erick
>>
>>
>> On Wed, Jan 8, 2014 at 11:56 AM, Kranti Parisa <[hidden email]
>> >wrote:
>>
>> > I was trying with the  [* TO *] as an example, the real use case is OR
>> > query between 2/more range queries of timestamp fields (saved in
>> > milliseconds). So I can't use FQs as they are ANDed by definition.
>> >
>> > Am I missing something here?
>> >
>> >
>> >
>> >
>> > Thanks,
>> > Kranti K. Parisa
>> > http://www.linkedin.com/in/krantiparisa
>> >
>> >
>> >
>> > On Wed, Jan 8, 2014 at 8:15 AM, Joel Bernstein <[hidden email]>
>> wrote:
>> >
>> > > Kranti,
>> > >
>> > > The range query also looks like a good candidate to be moved to a
>> filter
>> > > query so it can be cached.
>> > >
>> > > Joel Bernstein
>> > > Search Engineer at Heliosearch
>> > >
>> > >
>> > > On Tue, Jan 7, 2014 at 11:34 PM, Smiley, David W.
>><[hidden email]>
>> > > wrote:
>> > >
>> > > > Kranti,
>> > > >
>> > > > I can't speak to the specific slow-down while grouping, but if you
>> > expect
>> > > > to run [* TO *] queries with any frequency then you should index a
>> > > boolean
>> > > > flag and query for that instead.  You might also reduce the
>> > precisionStep
>> > > > value for the field you are using to 6 or even 4.  But wow that's
>>a
>> big
>> > > > difference you noted; it wouldn't hurt to double-check with the
>> > debugger
>> > > > that the [* TO *] is treated as a numeric range query instead of a
>> > > generic
>> > > > term range.
>> > > >
>> > > > ~ David
>> > > > ________________________________________
>> > > > From: Kranti Parisa [[hidden email]]
>> > > > Sent: Tuesday, January 07, 2014 10:26 PM
>> > > > To: [hidden email]
>> > > > Subject: Range queries with Grouping is slow?
>> > > >
>> > > > Is there any known issue with Range queries + grouping?
>> > > >
>> > > > Case1:
>> > > > q=id:123&group=true&sort=price
>> > > > asc&group.field=entityId&group.limit=2&group.ngroups=true
>> > > >
>> > > > Case2:
>> > > > q=id:123 AND price:[* TO *]&group=true&sort=price
>> > > > asc&group.field=entityId&group.limit=2&group.ngroups=true
>> > > >
>> > > > Index Size:10M/~5GB
>> > > > After running both queries at least once, I was expecting to hit
>>the
>> > > query
>> > > > caches and response should be quick enough, but
>> > > > Case1: 15-20ms (looks fine)
>> > > > Case2: 400+ms (this seems constantly >400ms even after the first
>> query)
>> > > >
>> > > > any thought? if it's a known issue, please point me to the jira
>>link
>> > > > otherwise I can open an issue if this needs some analysis?
>> > > >
>> > > >
>> > > > Thanks,
>> > > > Kranti K. Parisa
>> > > > http://www.linkedin.com/in/krantiparisa
>> > > >
>> > >
>> >
>>

Reply | Threaded
Open this post in threaded view
|

Re: Range queries with Grouping is slow?

Mikhail Khludnev
In reply to this post by Erick Erickson
Hello,

Here is workaround for caching separate clauses in OR filters.
http://blog.griddynamics.com/2014/01/segmented-filter-cache-in-solr.html
No coding is required, just try to experiment with request parameters.


On Wed, Jan 8, 2014 at 9:11 PM, Erick Erickson <[hidden email]>wrote:

> Well, actually you can use fqs, it's just that re-using them becomes a bit
> more tricky. Specifically,
> fq=field1:blah OR field2:blort
> is perfectly reasonable. However, it doesn't break things down into
> sub-clauses, so
> fq=field1:blah
> will create a new entry in the filtercache. And
> fq=field2:blort OR field1:blah
> will not match the first one.
>
> It kind of depends on the query pattern whether the filtercache will be
> re-used, you have to take care to construct the fq clauses with re-use in
> mind if you want ORs.
>
> Best,
> Erick
>
>
> On Wed, Jan 8, 2014 at 11:56 AM, Kranti Parisa <[hidden email]
> >wrote:
>
> > I was trying with the  [* TO *] as an example, the real use case is OR
> > query between 2/more range queries of timestamp fields (saved in
> > milliseconds). So I can't use FQs as they are ANDed by definition.
> >
> > Am I missing something here?
> >
> >
> >
> >
> > Thanks,
> > Kranti K. Parisa
> > http://www.linkedin.com/in/krantiparisa
> >
> >
> >
> > On Wed, Jan 8, 2014 at 8:15 AM, Joel Bernstein <[hidden email]>
> wrote:
> >
> > > Kranti,
> > >
> > > The range query also looks like a good candidate to be moved to a
> filter
> > > query so it can be cached.
> > >
> > > Joel Bernstein
> > > Search Engineer at Heliosearch
> > >
> > >
> > > On Tue, Jan 7, 2014 at 11:34 PM, Smiley, David W. <[hidden email]>
> > > wrote:
> > >
> > > > Kranti,
> > > >
> > > > I can't speak to the specific slow-down while grouping, but if you
> > expect
> > > > to run [* TO *] queries with any frequency then you should index a
> > > boolean
> > > > flag and query for that instead.  You might also reduce the
> > precisionStep
> > > > value for the field you are using to 6 or even 4.  But wow that's a
> big
> > > > difference you noted; it wouldn't hurt to double-check with the
> > debugger
> > > > that the [* TO *] is treated as a numeric range query instead of a
> > > generic
> > > > term range.
> > > >
> > > > ~ David
> > > > ________________________________________
> > > > From: Kranti Parisa [[hidden email]]
> > > > Sent: Tuesday, January 07, 2014 10:26 PM
> > > > To: [hidden email]
> > > > Subject: Range queries with Grouping is slow?
> > > >
> > > > Is there any known issue with Range queries + grouping?
> > > >
> > > > Case1:
> > > > q=id:123&group=true&sort=price
> > > > asc&group.field=entityId&group.limit=2&group.ngroups=true
> > > >
> > > > Case2:
> > > > q=id:123 AND price:[* TO *]&group=true&sort=price
> > > > asc&group.field=entityId&group.limit=2&group.ngroups=true
> > > >
> > > > Index Size:10M/~5GB
> > > > After running both queries at least once, I was expecting to hit the
> > > query
> > > > caches and response should be quick enough, but
> > > > Case1: 15-20ms (looks fine)
> > > > Case2: 400+ms (this seems constantly >400ms even after the first
> query)
> > > >
> > > > any thought? if it's a known issue, please point me to the jira link
> > > > otherwise I can open an issue if this needs some analysis?
> > > >
> > > >
> > > > Thanks,
> > > > Kranti K. Parisa
> > > > http://www.linkedin.com/in/krantiparisa
> > > >
> > >
> >
>



--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics

<http://www.griddynamics.com>
 <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Range queries with Grouping is slow?

Kranti Parisa
Thank you, will take a look at it.

Thanks,
Kranti K. Parisa
http://www.linkedin.com/in/krantiparisa



On Thu, Jan 9, 2014 at 10:25 AM, Mikhail Khludnev <
[hidden email]> wrote:

> Hello,
>
> Here is workaround for caching separate clauses in OR filters.
> http://blog.griddynamics.com/2014/01/segmented-filter-cache-in-solr.html
> No coding is required, just try to experiment with request parameters.
>
>
> On Wed, Jan 8, 2014 at 9:11 PM, Erick Erickson <[hidden email]
> >wrote:
>
> > Well, actually you can use fqs, it's just that re-using them becomes a
> bit
> > more tricky. Specifically,
> > fq=field1:blah OR field2:blort
> > is perfectly reasonable. However, it doesn't break things down into
> > sub-clauses, so
> > fq=field1:blah
> > will create a new entry in the filtercache. And
> > fq=field2:blort OR field1:blah
> > will not match the first one.
> >
> > It kind of depends on the query pattern whether the filtercache will be
> > re-used, you have to take care to construct the fq clauses with re-use in
> > mind if you want ORs.
> >
> > Best,
> > Erick
> >
> >
> > On Wed, Jan 8, 2014 at 11:56 AM, Kranti Parisa <[hidden email]
> > >wrote:
> >
> > > I was trying with the  [* TO *] as an example, the real use case is OR
> > > query between 2/more range queries of timestamp fields (saved in
> > > milliseconds). So I can't use FQs as they are ANDed by definition.
> > >
> > > Am I missing something here?
> > >
> > >
> > >
> > >
> > > Thanks,
> > > Kranti K. Parisa
> > > http://www.linkedin.com/in/krantiparisa
> > >
> > >
> > >
> > > On Wed, Jan 8, 2014 at 8:15 AM, Joel Bernstein <[hidden email]>
> > wrote:
> > >
> > > > Kranti,
> > > >
> > > > The range query also looks like a good candidate to be moved to a
> > filter
> > > > query so it can be cached.
> > > >
> > > > Joel Bernstein
> > > > Search Engineer at Heliosearch
> > > >
> > > >
> > > > On Tue, Jan 7, 2014 at 11:34 PM, Smiley, David W. <[hidden email]
> >
> > > > wrote:
> > > >
> > > > > Kranti,
> > > > >
> > > > > I can't speak to the specific slow-down while grouping, but if you
> > > expect
> > > > > to run [* TO *] queries with any frequency then you should index a
> > > > boolean
> > > > > flag and query for that instead.  You might also reduce the
> > > precisionStep
> > > > > value for the field you are using to 6 or even 4.  But wow that's a
> > big
> > > > > difference you noted; it wouldn't hurt to double-check with the
> > > debugger
> > > > > that the [* TO *] is treated as a numeric range query instead of a
> > > > generic
> > > > > term range.
> > > > >
> > > > > ~ David
> > > > > ________________________________________
> > > > > From: Kranti Parisa [[hidden email]]
> > > > > Sent: Tuesday, January 07, 2014 10:26 PM
> > > > > To: [hidden email]
> > > > > Subject: Range queries with Grouping is slow?
> > > > >
> > > > > Is there any known issue with Range queries + grouping?
> > > > >
> > > > > Case1:
> > > > > q=id:123&group=true&sort=price
> > > > > asc&group.field=entityId&group.limit=2&group.ngroups=true
> > > > >
> > > > > Case2:
> > > > > q=id:123 AND price:[* TO *]&group=true&sort=price
> > > > > asc&group.field=entityId&group.limit=2&group.ngroups=true
> > > > >
> > > > > Index Size:10M/~5GB
> > > > > After running both queries at least once, I was expecting to hit
> the
> > > > query
> > > > > caches and response should be quick enough, but
> > > > > Case1: 15-20ms (looks fine)
> > > > > Case2: 400+ms (this seems constantly >400ms even after the first
> > query)
> > > > >
> > > > > any thought? if it's a known issue, please point me to the jira
> link
> > > > > otherwise I can open an issue if this needs some analysis?
> > > > >
> > > > >
> > > > > Thanks,
> > > > > Kranti K. Parisa
> > > > > http://www.linkedin.com/in/krantiparisa
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
> Principal Engineer,
> Grid Dynamics
>
> <http://www.griddynamics.com>
>  <[hidden email]>
>