using the df parameter to set a default to search all fields

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

using the df parameter to set a default to search all fields

rhys J
 How do I make Solr search on all fields in a document?

I read the documentation about the df field, and added the following to my
solrconfig.xml:

 <lst name="defaults">
      <str name="echoParams">explicit</str>
      <int name="rows">10</int>
         <str name="df">_text_</str>
    </lst>

in my managed-schema file i have the following:

 <field name="_text_" type="text_general" multiValued="true" indexed="true"
stored="true"/>

I have deleted the documents, and re-indexed the csv file.

When I do a search in the api for: _text_:amy - which should return 2
documents, I get nothing.

If I do a search for 'amy' in the q field, I still get nothing.

If I do an explicit search for name:amy, I get 2 documents returned.
Reply | Threaded
Open this post in threaded view
|

Re: using the df parameter to set a default to search all fields

Shawn Heisey-2
On 10/22/2019 10:26 AM, rhys J wrote:
> How do I make Solr search on all fields in a document?

Solr does not have a way to ask for all fields on a search.  If you use
the edismax query parser, you can specify multiple fields with the qf
parameter, but there is nothing you can put in that parameter as a
shortcut for "all fields."  Using qf with multiple fields is the
cleanest way to do this.

> I read the documentation about the df field, and added the following to my
> solrconfig.xml:
>
>   <lst name="defaults">
>        <str name="echoParams">explicit</str>
>        <int name="rows">10</int>
>           <str name="df">_text_</str>
>      </lst>

The df parameter just means "default field".  It can only search one field.

> in my managed-schema file i have the following:
>
>   <field name="_text_" type="text_general" multiValued="true" indexed="true"
> stored="true"/>
>
> I have deleted the documents, and re-indexed the csv file.
>
> When I do a search in the api for: _text_:amy - which should return 2
> documents, I get nothing.

Just having a field named _text_ doesn't make anything happen, unless
your indexing specifically adds documents with that field defined.
There is nothing special about _text_.  Other field names that start and
end with an underscore, like _version_ or _root_, are special ... but
_text_ is not.

Probably what you are looking for here is to set up one or more
copyField definitions in your schema, which are configured to copy one
or more of your other fields to _text_ so it can be searched as a
catchall field.  I find it useful to name that field "catchall" rather
than something like _text_ which seems like a special field name, but isn't.

Thanks,
Shawn
Reply | Threaded
Open this post in threaded view
|

Re: using the df parameter to set a default to search all fields

rhys J
> Solr does not have a way to ask for all fields on a search.  If you use
> the edismax query parser, you can specify multiple fields with the qf
> parameter, but there is nothing you can put in that parameter as a
> shortcut for "all fields."  Using qf with multiple fields is the
> cleanest way to do this.
>
>
How would I enter qf parameters in the solrconfig.xml?


> Probably what you are looking for here is to set up one or more
> copyField definitions in your schema, which are configured to copy one
> or more of your other fields to _text_ so it can be searched as a
> catchall field.  I find it useful to name that field "catchall" rather
> than something like _text_ which seems like a special field name, but
> isn't.
>

I did as you suggested, and created a field called 'all_fields' and added
copyFields too. I re-indexed, and this works when i do the search.

Thanks

Rhys
Reply | Threaded
Open this post in threaded view
|

Re: Re: using the df parameter to set a default to search all fields

Audrey Lorberfeld - Audrey.Lorberfeld@ibm.com
In reply to this post by Shawn Heisey-2
I think you actually can search over all fields, but not in the df parameter. We have a big list of fields we want to search over. So, we just put a dummy one in the df param field, and then we use the fl parameter. With the edismax parser, this works. It looks something like this:

<requestHandler name="/w3search_en" class="solr.SearchHandler">
        <lst name="defaults">
            <str name="defType">edismax</str>
            <float name="tie">1.0</float>
            <str name="echoParams">explicit</str>
            <int name="rows">30</int>
            <str name="df">content_en</str>
        <str name="fl">
                update_date, display_url, url, id, uid, scopes, source_id, json_payload, language, snippet, [elevated],
                title:title_en, description:description_en, score
            </str>
        </lst>
</requestHandler>

--
Audrey Lorberfeld
Data Scientist, w3 Search
IBM
[hidden email]
 

On 10/22/19, 1:01 PM, "Shawn Heisey" <[hidden email]> wrote:

    On 10/22/2019 10:26 AM, rhys J wrote:
    > How do I make Solr search on all fields in a document?
   
    Solr does not have a way to ask for all fields on a search.  If you use
    the edismax query parser, you can specify multiple fields with the qf
    parameter, but there is nothing you can put in that parameter as a
    shortcut for "all fields."  Using qf with multiple fields is the
    cleanest way to do this.
   
    > I read the documentation about the df field, and added the following to my
    > solrconfig.xml:
    >
    >   <lst name="defaults">
    >        <str name="echoParams">explicit</str>
    >        <int name="rows">10</int>
    >           <str name="df">_text_</str>
    >      </lst>
   
    The df parameter just means "default field".  It can only search one field.
   
    > in my managed-schema file i have the following:
    >
    >   <field name="_text_" type="text_general" multiValued="true" indexed="true"
    > stored="true"/>
    >
    > I have deleted the documents, and re-indexed the csv file.
    >
    > When I do a search in the api for: _text_:amy - which should return 2
    > documents, I get nothing.
   
    Just having a field named _text_ doesn't make anything happen, unless
    your indexing specifically adds documents with that field defined.
    There is nothing special about _text_.  Other field names that start and
    end with an underscore, like _version_ or _root_, are special ... but
    _text_ is not.
   
    Probably what you are looking for here is to set up one or more
    copyField definitions in your schema, which are configured to copy one
    or more of your other fields to _text_ so it can be searched as a
    catchall field.  I find it useful to name that field "catchall" rather
    than something like _text_ which seems like a special field name, but isn't.
   
    Thanks,
    Shawn
   

Reply | Threaded
Open this post in threaded view
|

Re: using the df parameter to set a default to search all fields

Shawn Heisey-2
In reply to this post by rhys J
On 10/22/2019 11:17 AM, rhys J wrote:
> How would I enter qf parameters in the solrconfig.xml?

In the handler definition you would simply add the parameter like you
did when you added df.

   <requestHandler name="/select" class="solr.SearchHandler">
     <lst name="defaults">
       <str name="defType">edismax</str>
       <str name="qf">field1 field2 field3</str>
   </requestHandler>

I included the parameter in the above example that switches the query
parser to edismax, which is also required when using qf.  The qf
parameter does not work with the default lucene query parser.

You should definitely read up on edismax.  It works very differently
than the default parser.  For most uses I would say it's better.

Thanks,
Shawn
Reply | Threaded
Open this post in threaded view
|

Re: using the df parameter to set a default to search all fields

Shawn Heisey-2
In reply to this post by Audrey Lorberfeld - Audrey.Lorberfeld@ibm.com
On 10/22/2019 11:42 AM, Audrey Lorberfeld - [hidden email] wrote:
> I think you actually can search over all fields, but not in the df parameter. We have a big list of fields we want to search over. So, we just put a dummy one in the df param field, and then we use the fl parameter. With the edismax parser, this works. It looks something like this:

The fl parameter means "field list" and controls which fields are
included in the search results.  It does not control which fields are
searched.

Thanks,
Shawn
Reply | Threaded
Open this post in threaded view
|

Re: Re: using the df parameter to set a default to search all fields

Audrey Lorberfeld - Audrey.Lorberfeld@ibm.com
Eek, Shawn, you're right -- I'm sorry, all! I meant to say the the QF (!) parameter. And pasted the wrong thing too ☹ This is what ours looks like with the qf parameter (and the edismax parser)

  <str name="qf">
              title_en^1.5 description_en^0.5 content_en^0.5 headings_en^1.3 keywords_en^1.5 url^0.5
  </str>

--
Audrey Lorberfeld
Data Scientist, w3 Search
IBM
[hidden email]
 

On 10/22/19, 1:50 PM, "Shawn Heisey" <[hidden email]> wrote:

    On 10/22/2019 11:42 AM, Audrey Lorberfeld - [hidden email] wrote:
    > I think you actually can search over all fields, but not in the df parameter. We have a big list of fields we want to search over. So, we just put a dummy one in the df param field, and then we use the fl parameter. With the edismax parser, this works. It looks something like this:
   
    The fl parameter means "field list" and controls which fields are
    included in the search results.  It does not control which fields are
    searched.
   
    Thanks,
    Shawn