Default Values and Missing Field Queries

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Default Values and Missing Field Queries

Chris Dempsey
I'm new to Solr and made an honest stab to finding this in info the docs.

I'm working on an update to an existing large collection in Solr 7.7 to add
a BoolField to mark it as "soft deleted" or not. My understanding is that
updating the schema will mean the new field will only exist and have a
value (or the default value) for documents indexed after the change,
correct? If that's the case, is it possible to query for all documents that
have that field set to `true` or if that field is completely missing? If is
a Bad Idea(tm) from a performance or resource usage standpoint to use a
"where field X doesn't exist" query (i.e. am I going to end up running a
"table scan" if I do)?

Thanks in advance!
Reply | Threaded
Open this post in threaded view
|

Re: Default Values and Missing Field Queries

Erick Erickson
Try q=*:* -boolfield=false

And it's not as costly as you might think, there's special handling for *:*
queries. And if you put that in an fq clause instead, the result set will
be put into the filter cache and be reused assuming you want to do this
repeatedly.

BTW, Solr doesn't use strict Boolean logic, which may be a bit confusing.
Google for Chris Hostetter's (Hossman) blog at Lucidwirks for a great
explanation.

And yes, your understanding of adding a new field is correct

Best,
Erick
On Mon, May 25, 2020, 11:39 Chris Dempsey <[hidden email]> wrote:

> I'm new to Solr and made an honest stab to finding this in info the docs.
>
> I'm working on an update to an existing large collection in Solr 7.7 to add
> a BoolField to mark it as "soft deleted" or not. My understanding is that
> updating the schema will mean the new field will only exist and have a
> value (or the default value) for documents indexed after the change,
> correct? If that's the case, is it possible to query for all documents that
> have that field set to `true` or if that field is completely missing? If is
> a Bad Idea(tm) from a performance or resource usage standpoint to use a
> "where field X doesn't exist" query (i.e. am I going to end up running a
> "table scan" if I do)?
>
> Thanks in advance!
>
Reply | Threaded
Open this post in threaded view
|

Re: Default Values and Missing Field Queries

Chris Dempsey
Thanks for the clarification and pointers Erick! Much appreciated!

On Mon, May 25, 2020 at 11:18 AM Erick Erickson <[hidden email]>
wrote:

> Try q=*:* -boolfield=false
>
> And it's not as costly as you might think, there's special handling for *:*
> queries. And if you put that in an fq clause instead, the result set will
> be put into the filter cache and be reused assuming you want to do this
> repeatedly.
>
> BTW, Solr doesn't use strict Boolean logic, which may be a bit confusing.
> Google for Chris Hostetter's (Hossman) blog at Lucidwirks for a great
> explanation.
>
> And yes, your understanding of adding a new field is correct
>
> Best,
> Erick
> On Mon, May 25, 2020, 11:39 Chris Dempsey <[hidden email]> wrote:
>
> > I'm new to Solr and made an honest stab to finding this in info the docs.
> >
> > I'm working on an update to an existing large collection in Solr 7.7 to
> add
> > a BoolField to mark it as "soft deleted" or not. My understanding is that
> > updating the schema will mean the new field will only exist and have a
> > value (or the default value) for documents indexed after the change,
> > correct? If that's the case, is it possible to query for all documents
> that
> > have that field set to `true` or if that field is completely missing? If
> is
> > a Bad Idea(tm) from a performance or resource usage standpoint to use a
> > "where field X doesn't exist" query (i.e. am I going to end up running a
> > "table scan" if I do)?
> >
> > Thanks in advance!
> >
>