Solr - case-insensitive search do not work

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Solr - case-insensitive search do not work

meghana
This post was updated on .
I want to apply case-insensitive search for field myfield in solr.

I googled a bit for that , and i found that , i need to apply LowerCaseFilterFactory to Field Type and field should be of solr.TextFeild.

I applied that in my schema.xml and re-index the data, then also my search seems to be case-sensitive.

Below is search that i perform.

http://localhost:8080/solr/select?q=myfield:"cloud university"&hl=on&hl.snippets=99&hl.fl=myfield

Below is definition for field type

 <fieldType name="text_en_splitting" class="solr.TextField" positionIncrementGap="100" autoGeneratePhraseQueries="true">
      <analyzer type="index">
        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
       
       
        <filter class="solr.StopFilterFactory"
                ignoreCase="true"
                words="stopwords_en.txt"
                enablePositionIncrements="true"
                />
        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
        <filter class="solr.LowerCaseFilterFactory"/>
        <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
        <filter class="solr.PorterStemFilterFactory"/>
      </analyzer>
      <analyzer type="query">
        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
        <filter class="solr.StopFilterFactory"
                ignoreCase="true"
                words="stopwords_en.txt"
                enablePositionIncrements="true"
                />
        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
        <filter class="solr.LowerCaseFilterFactory"/>
        <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
        <filter class="solr.PorterStemFilterFactory"/>
      </analyzer>
    </fieldType>

and below is my field definition

 <field name="myfield" type="text_en_splitting" indexed="true" stored="true" />

Not sure , what is wrong with this. Please help me to resolve this.

Thanks
Reply | Threaded
Open this post in threaded view
|

Re: Solr - case-insensitive search do not work

Ravish Bhagdev
 <filter class="solr.LowerCaseFilterFactory"/> is already present in your
field type definition (its twice now)

Are you adding quotes around your query by any chance?

Ravish

On Wed, Aug 22, 2012 at 11:31 AM, meghana <[hidden email]>wrote:

> I want to apply case-insensitive search for field *myfield* in solr.
>
> I googled a bit for that , and i found that , i need to apply
> *LowerCaseFilterFactory *to Field Type and field should be of
> solr.TextFeild.
>
> I applied that in my *schema.xml* and re-index the data, then also my
> search
> seems to be case-sensitive.
>
> Below is search that i perform.
> *
> http://localhost:8080/solr/select?q=myfield:"cloud
> university"&hl=on&hl.snippets=99&hl.fl=myfield*
>
> Below is definition for field type
>
>  <fieldType name="text_en_splitting" class="solr.TextField"
> positionIncrementGap="100" autoGeneratePhraseQueries="true">
>       <analyzer type="index">
>         <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>
>
>         <filter class="solr.StopFilterFactory"
>                 ignoreCase="true"
>                 words="stopwords_en.txt"
>                 enablePositionIncrements="true"
>                 />
>         <filter class="solr.WordDelimiterFilterFactory"
> generateWordParts="1" generateNumberParts="1" catenateWords="1"
> catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
>         <filter class="solr.LowerCaseFilterFactory"/>
>         <filter class="solr.KeywordMarkerFilterFactory"
> protected="protwords.txt"/>
>         <filter class="solr.PorterStemFilterFactory"/>
>       </analyzer>
>       <analyzer type="query">
>         <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>         <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt"
> ignoreCase="true" expand="true"/>
>         <filter class="solr.StopFilterFactory"
>                 ignoreCase="true"
>                 words="stopwords_en.txt"
>                 enablePositionIncrements="true"
>                 />
>         <filter class="solr.WordDelimiterFilterFactory"
> generateWordParts="1" generateNumberParts="1" catenateWords="0"
> catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
>         <filter class="solr.LowerCaseFilterFactory"/>
>         <filter class="solr.KeywordMarkerFilterFactory"
> protected="protwords.txt"/>
>         <filter class="solr.PorterStemFilterFactory"/>
>       </analyzer>
>     </fieldType>
>
> and below is my field definition
>
>  <field name="myfield" type="text_en_splitting" indexed="true"
> stored="true"
> />
>
> Not sure , what is wrong with this. Please help me to resolve this.
>
> Thanks
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-case-insensitive-search-do-not-work-tp4002605.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: Solr - case-insensitive search do not work

meghana
@Ravish Bhagdev , Yes I am adding double quotes around my search , as shown in my post. Like,

myfield:"cloud university"

Reply | Threaded
Open this post in threaded view
|

Re: Solr - case-insensitive search do not work

Ravish Bhagdev
OK.  Try without quotes like myfield:cloud+university and see if it has any
effect.

Also, try both queries with debugging turned on and post the output of the
same ( http://wiki.apache.org/solr/CommonQueryParameters#Debugging )

It must be some field configuration issue or that double quotes are causing
some analyzers to not work on your query.

Hope this helps.

Ravish

On Wed, Aug 22, 2012 at 12:11 PM, meghana <[hidden email]>wrote:

> @Ravish Bhagdev , Yes I am adding double quotes around my search , as shown
> in my post. Like,
>
> myfield:"cloud university"
>
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-case-insensitive-search-do-not-work-tp4002605p4002610.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: Solr - case-insensitive search do not work

Ravish Bhagdev
Also, try comparing your field configuration to Solrs default text field
and see if you can spot any differences.

Ravish

On Wed, Aug 22, 2012 at 1:09 PM, Ravish Bhagdev <[hidden email]>wrote:

> OK.  Try without quotes like myfield:cloud+university and see if it has
> any effect.
>
> Also, try both queries with debugging turned on and post the output of the
> same ( http://wiki.apache.org/solr/CommonQueryParameters#Debugging )
>
> It must be some field configuration issue or that double quotes are
> causing some analyzers to not work on your query.
>
> Hope this helps.
>
> Ravish
>
> On Wed, Aug 22, 2012 at 12:11 PM, meghana <[hidden email]>wrote:
>
>> @Ravish Bhagdev , Yes I am adding double quotes around my search , as
>> shown
>> in my post. Like,
>>
>> myfield:"cloud university"
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/Solr-case-insensitive-search-do-not-work-tp4002605p4002610.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Solr - case-insensitive search do not work

meghana
This post was updated on .
if i search as myfield:clould+university , then it will return all results having either 'cloud' or 'university' or both , but i need terms which have both 'cloud' and 'university' like a phrase , and yes i'll compare defualt solr schema with my schema
Reply | Threaded
Open this post in threaded view
|

Re: Solr - case-insensitive search do not work

meghana
Hi Ravish , the defination for text_en_splitting in solr default schema and of mine are same.. still its not working... any idea?
Reply | Threaded
Open this post in threaded view
|

Re: Solr - case-insensitive search do not work

Ravish Bhagdev
Did you see my message about debugging parameters?  Try that and see what's
happening behind the scenes.

I can confirm that by default the queries are NOT case sensitive.

Ravish

On Wed, Aug 22, 2012 at 2:45 PM, meghana <[hidden email]> wrote:

> Hi Ravish , the defination for text_en_splitting in solr default schema and
> of mine are same.. still its not working... any idea?
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-case-insensitive-search-do-not-work-tp4002605p4002645.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>