fq * vs [* TO *]

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

fq * vs [* TO *]

Vincenzo D'Amore
Hi all,

talking about how to write solr queries I was investigating if there is a
difference of performance in these two filter queries: field:[* TO *]  or
field:*

In other words:

q=*:*&fq=field:[* TO *]&rows=0

q=*:*&fq=field:*&rows=0

Could someone enlighten me?

--
Vincenzo D'Amore
Reply | Threaded
Open this post in threaded view
|

Re: fq * vs [* TO *]

Mikhail Khludnev-2
Hello, Vincenzo.
I traced both code pathes, they are different. It's hard to predict the
difference between them. Probably some thorough microbenchmark can show two
times fold or so, but don't think it's significant for practical usage.

On Thu, Sep 19, 2019 at 10:23 AM Vincenzo D'Amore <[hidden email]>
wrote:

> Hi all,
>
> talking about how to write solr queries I was investigating if there is a
> difference of performance in these two filter queries: field:[* TO *]  or
> field:*
>
> In other words:
>
> q=*:*&fq=field:[* TO *]&rows=0
>
> q=*:*&fq=field:*&rows=0
>
> Could someone enlighten me?
>
> --
> Vincenzo D'Amore
>


--
Sincerely yours
Mikhail Khludnev
Reply | Threaded
Open this post in threaded view
|

Re: fq * vs [* TO *]

Vincenzo D'Amore
Thanks Mikhail, could you please share the code paths you found?

On Thu, Sep 19, 2019 at 10:57 AM Mikhail Khludnev <[hidden email]> wrote:

> Hello, Vincenzo.
> I traced both code pathes, they are different. It's hard to predict the
> difference between them. Probably some thorough microbenchmark can show two
> times fold or so, but don't think it's significant for practical usage.
>
> On Thu, Sep 19, 2019 at 10:23 AM Vincenzo D'Amore <[hidden email]>
> wrote:
>
> > Hi all,
> >
> > talking about how to write solr queries I was investigating if there is a
> > difference of performance in these two filter queries: field:[* TO *]  or
> > field:*
> >
> > In other words:
> >
> > q=*:*&fq=field:[* TO *]&rows=0
> >
> > q=*:*&fq=field:*&rows=0
> >
> > Could someone enlighten me?
> >
> > --
> > Vincenzo D'Amore
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>


--
Vincenzo D'Amore
Reply | Threaded
Open this post in threaded view
|

Re: fq * vs [* TO *]

Mikhail Khludnev-2
https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/parser/SolrQueryParserBase.java#L1234

https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/parser/SolrQueryParserBase.java#L1184


On Thu, Sep 19, 2019 at 1:10 PM Vincenzo D'Amore <[hidden email]> wrote:

> Thanks Mikhail, could you please share the code paths you found?
>
> On Thu, Sep 19, 2019 at 10:57 AM Mikhail Khludnev <[hidden email]> wrote:
>
> > Hello, Vincenzo.
> > I traced both code pathes, they are different. It's hard to predict the
> > difference between them. Probably some thorough microbenchmark can show
> two
> > times fold or so, but don't think it's significant for practical usage.
> >
> > On Thu, Sep 19, 2019 at 10:23 AM Vincenzo D'Amore <[hidden email]>
> > wrote:
> >
> > > Hi all,
> > >
> > > talking about how to write solr queries I was investigating if there
> is a
> > > difference of performance in these two filter queries: field:[* TO *]
> or
> > > field:*
> > >
> > > In other words:
> > >
> > > q=*:*&fq=field:[* TO *]&rows=0
> > >
> > > q=*:*&fq=field:*&rows=0
> > >
> > > Could someone enlighten me?
> > >
> > > --
> > > Vincenzo D'Amore
> > >
> >
> >
> > --
> > Sincerely yours
> > Mikhail Khludnev
> >
>
>
> --
> Vincenzo D'Amore
>


--
Sincerely yours
Mikhail Khludnev
Reply | Threaded
Open this post in threaded view
|

Re: fq * vs [* TO *]

Shawn Heisey-2
In reply to this post by Vincenzo D'Amore
On 9/19/2019 1:23 AM, Vincenzo D'Amore wrote:
> talking about how to write solr queries I was investigating if there is a
> difference of performance in these two filter queries: field:[* TO *]  or
> field:*
>
> In other words:
>
> q=*:*&fq=field:[* TO *]&rows=0
>
> q=*:*&fq=field:*&rows=0

The first one is a range query, the second is a wildcard query.

Ordinarily we strongly recommend against wildcard queries for selecting
all documents where a field exists (has a value).  For the general case,
a range query will be faster.

If the field's cardinality is VERY low, a wildcard query can be fast,
and might even be faster than the range query ... but if the field has
ten million possible values (terms) in the index, the query that Solr
constructs from a wildcard will quite literally contain all ten million
of those values, and it will be VERY slow.

Thanks,
Shawn
Reply | Threaded
Open this post in threaded view
|

Re: fq * vs [* TO *]

Vincenzo D'Amore
Hi Shawn, Mikhail,

thanks for the feedback. Really appreciate it.

Best regards,
Vincenzo

On Thu, Sep 19, 2019 at 3:55 PM Shawn Heisey <[hidden email]> wrote:

> On 9/19/2019 1:23 AM, Vincenzo D'Amore wrote:
> > talking about how to write solr queries I was investigating if there is a
> > difference of performance in these two filter queries: field:[* TO *]  or
> > field:*
> >
> > In other words:
> >
> > q=*:*&fq=field:[* TO *]&rows=0
> >
> > q=*:*&fq=field:*&rows=0
>
> The first one is a range query, the second is a wildcard query.
>
> Ordinarily we strongly recommend against wildcard queries for selecting
> all documents where a field exists (has a value).  For the general case,
> a range query will be faster.
>
> If the field's cardinality is VERY low, a wildcard query can be fast,
> and might even be faster than the range query ... but if the field has
> ten million possible values (terms) in the index, the query that Solr
> constructs from a wildcard will quite literally contain all ten million
> of those values, and it will be VERY slow.
>
> Thanks,
> Shawn
>


--
Vincenzo D'Amore