Help with a slow filter query

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Help with a slow filter query

heaven
This post was updated on .
Hi, please help me to figure out what's wrong with this query: http://pastie.org/pastes/9544433/text?key=v9favzj2ulcaq0qvorda

Without cache=false and cost it takes more than 15 seconds. What's weird type:"Award::Nomination" takes a few milliseconds when created_at_d:[* TO 2014-09-08T23:59:59Z] takes 15 seconds. What do I do wrong?

fields:
{code}
<field name="type" stored="false" type="string" multiValued="true" indexed="true"/>
<dynamicField name="*_d" stored="false" type="date" multiValued="false" indexed="true"/>
{code}

About the same number of docs where these fields are set, type has even higher number documents.

Best,
Alex
Reply | Threaded
Open this post in threaded view
|

Re: Help with a slow filter query

heaven
Reply | Threaded
Open this post in threaded view
|

Re: Help with a slow filter query

Mikhail Khludnev
>
> when created_at_d:[* TO
> 2014-09-08T23:59:59Z] takes 15 seconds. What do I do wrong?
>

Nothing serious, you just send darn expensive range.
Filter evaluation costs a sum of enumerating terms and enumerating docs
under that range.
To speed it up: you can reduce the number of terms by dropping some
precision in index time:
2014-09-08T23:59:59Z*/MINUTES*
see https://cwiki.apache.org/confluence/display/solr/Working+with+Dates
if you can drop it, try to it in range query, filter cache will be much
more efficient. It will be till expensive, but cached well.
another way to deal with range apply it purely as a post filter, use frange
and cost > 100. see expensive filter
http://heliosearch.org/advanced-filter-caching-in-solr/



On Thu, Sep 11, 2014 at 4:29 PM, heaven <[hidden email]> wrote:



--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics

<http://www.griddynamics.com>
<[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Help with a slow filter query

heaven
The syntax for frange query parser is weird:
{!frange cache=false cost=200 l=2001-01-01T00:00:00Z u=2013-12-01T23:59:59Z}nominated_at_d
or simply
{!frange cache=false cost=200 u=2013-12-01T23:59:59Z}nominated_at_d

And I don't see any docs in Solr wiki explaining this syntax, these examples above I've found at third parties' blogs.

I would expect something like
{!frange cache=false cost=200}created_at_d:[* TO 2014-09-08T23:59:59Z]
but this fails with a syntax error. What is "I" and what is "U"?

I hardly imagine how this could be used in a real world application (e.g. how to implement this syntax in Sunspot gem for Rails,  etc). Is there a wiki that explains frange parser in more details?

Thank you,
Alexander
Reply | Threaded
Open this post in threaded view
|

Re: Help with a slow filter query

Mikhail Khludnev
https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-FunctionRangeQueryParser
http://lucene.apache.org/solr/4_10_0/solr-core/org/apache/solr/search/FunctionRangeQParserPlugin.html
re "this syntax"
https://cwiki.apache.org/confluence/display/solr/Local+Parameters+in+Queries

On Tue, Oct 7, 2014 at 6:56 PM, heaven <[hidden email]> wrote:

> The syntax for frange query parser is weird:
> {!frange cache=false cost=200 l=2001-01-01T00:00:00Z
> u=2013-12-01T23:59:59Z}nominated_at_d
> or simply
> {!frange cache=false cost=200 u=2013-12-01T23:59:59Z}nominated_at_d
>
> And I don't see any docs in Solr wiki explaining this syntax, these
> examples
> above I've found at third parties' blogs.
>
> I would expect something like
> {!frange cache=false cost=200}created_at_d:[* TO 2014-09-08T23:59:59Z]
> but this fails with a syntax error. What is "I" and what is "U"?
>
> I hardly imagine how this could be used in a real world application (e.g.
> how to implement this syntax in Sunspot gem for Rails,  etc). Is there a
> wiki that explains frange parser in more details?
>
> Thank you,
> Alexander
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Help-with-a-slow-filter-query-tp4158159p4163151.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics

<http://www.griddynamics.com>
<[hidden email]>