Case Insensitive Search for StringField

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Case Insensitive Search for StringField

Chellasamy G


Hi,



I can't find any way to perform case insensitive search on StringField. Please help me out.





i.e If the field value is "Flying Robots", then the phrases "flying robots", "fLying RObots" etc should match the value.





I also need the original case of the field value to be preserved in the search results.





Thanks,

Satyan



Reply | Threaded
Open this post in threaded view
|

Re: Case Insensitive Search for StringField

Adrien Grand
Hi Satyan,

You need to add a LowercaseTokenFilter to your analysis chain. The way to
do it depends on how you are building your analyzer today (pre-built
analyzer, extending Analyzer or using CustomAnalyzer). This will preserve
the original case in field values because lowercasing will only be applied
to the content of the inverted index, not stored fields where hits are
fetched from.

Le mer. 23 mai 2018 à 08:36, Chellasamy G <[hidden email]> a
écrit :

>
>
> Hi,
>
>
>
> I can't find any way to perform case insensitive search on StringField.
> Please help me out.
>
>
>
>
>
> i.e If the field value is "Flying Robots", then the phrases "flying
> robots", "fLying RObots" etc should match the value.
>
>
>
>
>
> I also need the original case of the field value to be preserved in the
> search results.
>
>
>
>
>
> Thanks,
>
> Satyan
>
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Case Insensitive Search for StringField

Chellasamy G
Hi,



Thanks for the reply.



Actually I need to implement it for StringField which is non-analyzed. So, if I am not wrong I can't add the analyzer for StringField.





My scenario is something similar to the one discussed in the below thread,



https://discuss.elastic.co/t/es-5-0-case-insensitive-search-for-keyword-fields/64111/10



Could you please let me know how to do the same the thing in lucene.





Thanks,

Satyan






---- On Wed, 23 May 2018 12:09:31 +0530 Adrien Grand &lt;[hidden email]&gt; wrote ----




Hi Satyan,

 

You need to add a LowercaseTokenFilter to your analysis chain. The way to

do it depends on how you are building your analyzer today (pre-built

analyzer, extending Analyzer or using CustomAnalyzer). This will preserve

the original case in field values because lowercasing will only be applied

to the content of the inverted index, not stored fields where hits are

fetched from.

 

Le mer. 23 mai 2018 à 08:36, Chellasamy G &lt;[hidden email]&gt; a

écrit :

 

&gt;

&gt;

&gt; Hi,

&gt;

&gt;

&gt;

&gt; I can't find any way to perform case insensitive search on StringField.

&gt; Please help me out.

&gt;

&gt;

&gt;

&gt;

&gt;

&gt; i.e If the field value is "Flying Robots", then the phrases "flying

&gt; robots", "fLying RObots" etc should match the value.

&gt;

&gt;

&gt;

&gt;

&gt;

&gt; I also need the original case of the field value to be preserved in the

&gt; search results.

&gt;

&gt;

&gt;

&gt;

&gt;

&gt; Thanks,

&gt;

&gt; Satyan

&gt;

&gt;

&gt;

&gt;






Reply | Threaded
Open this post in threaded view
|

Re: Case Insensitive Search for StringField

Chellasamy G


Hi Team,





Kindly help me out with this problem.





Thanks,

Satyan





---- On Wed, 23 May 2018 15:01:39 +0530 Chellasamy G &lt;[hidden email]&gt; wrote ----




Hi,

 

 

 

Thanks for the reply.

 

 

 

Actually I need to implement it for StringField which is non-analyzed. So, if I am not wrong I can't add the analyzer for StringField.

 

 

 

 

 

My scenario is something similar to the one discussed in the below thread,

 

 

 

https://discuss.elastic.co/t/es-5-0-case-insensitive-search-for-keyword-fields/64111/10 

 

 

 

Could you please let me know how to do the same the thing in lucene.

 

 

 

 

 

Thanks,

 

Satyan

 

 

 

 

 

 

---- On Wed, 23 May 2018 12:09:31 +0530 Adrien Grand &amp;lt;[hidden email]&amp;gt; wrote ----

 

 

 

 

Hi Satyan,

 

 

 

You need to add a LowercaseTokenFilter to your analysis chain. The way to

 

do it depends on how you are building your analyzer today (pre-built

 

analyzer, extending Analyzer or using CustomAnalyzer). This will preserve

 

the original case in field values because lowercasing will only be applied

 

to the content of the inverted index, not stored fields where hits are

 

fetched from.

 

 

 

Le mer. 23 mai 2018 à 08:36, Chellasamy G &amp;lt;[hidden email]&amp;gt; a

 

écrit :

 

 

 

&amp;gt;

 

&amp;gt;

 

&amp;gt; Hi,

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt; I can't find any way to perform case insensitive search on StringField.

 

&amp;gt; Please help me out.

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt; i.e If the field value is "Flying Robots", then the phrases "flying

 

&amp;gt; robots", "fLying RObots" etc should match the value.

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt; I also need the original case of the field value to be preserved in the

 

&amp;gt; search results.

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt; Thanks,

 

&amp;gt;

 

&amp;gt; Satyan

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

 

 

 

 

 






Reply | Threaded
Open this post in threaded view
|

Re: Case Insensitive Search for StringField

Ahmet Arslan
 Hi,
string_ci type could be constructed from: keyword tokenizer + lowercase filter + may be trim filter.
Ahmet
    On Friday, May 25, 2018, 1:50:19 PM GMT+3, Chellasamy G <[hidden email]> wrote:  
 
 

Hi Team,





Kindly help me out with this problem.





Thanks,

Satyan





---- On Wed, 23 May 2018 15:01:39 +0530 Chellasamy G &lt;[hidden email]&gt; wrote ----




Hi,

 

 

 

Thanks for the reply.

 

 

 

Actually I need to implement it for StringField which is non-analyzed. So, if I am not wrong I can't add the analyzer for StringField.

 

 

 

 

 

My scenario is something similar to the one discussed in the below thread,

 

 

 

https://discuss.elastic.co/t/es-5-0-case-insensitive-search-for-keyword-fields/64111/10 

 

 

 

Could you please let me know how to do the same the thing in lucene.

 

 

 

 

 

Thanks,

 

Satyan

 

 

 

 

 

 

---- On Wed, 23 May 2018 12:09:31 +0530 Adrien Grand &amp;lt;[hidden email]&amp;gt; wrote ----

 

 

 

 

Hi Satyan,

 

 

 

You need to add a LowercaseTokenFilter to your analysis chain. The way to

 

do it depends on how you are building your analyzer today (pre-built

 

analyzer, extending Analyzer or using CustomAnalyzer). This will preserve

 

the original case in field values because lowercasing will only be applied

 

to the content of the inverted index, not stored fields where hits are

 

fetched from.

 

 

 

Le mer. 23 mai 2018 à 08:36, Chellasamy G &amp;lt;[hidden email]&amp;gt; a

 

écrit :

 

 

 

&amp;gt;

 

&amp;gt;

 

&amp;gt; Hi,

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt; I can't find any way to perform case insensitive search on StringField.

 

&amp;gt; Please help me out.

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt; i.e If the field value is "Flying Robots", then the phrases "flying

 

&amp;gt; robots", "fLying RObots" etc should match the value.

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt; I also need the original case of the field value to be preserved in the

 

&amp;gt; search results.

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt; Thanks,

 

&amp;gt;

 

&amp;gt; Satyan

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

&amp;gt;

 

 

 

 

 

 





 
Reply | Threaded
Open this post in threaded view
|

Re: Case Insensitive Search for StringField

Michael Sokolov-4
In reply to this post by Chellasamy G
It's hard to help you because you haven't really explained what the problem
is. You say you must use StringField, which is a non analysed field, but
you also want to lower case the text values in the field, which requires
analysis.

I would suggest that you probably want to use TextField with a lowercase
analyzer, but that is contrary to your stated requirements. It maybe you
can just do the lower casing outside of lucene.

So I guess if you want help you'll need to explain your problem better.

On Fri, May 25, 2018, 6:50 AM Chellasamy G <[hidden email]>
wrote:

>
>
> Hi Team,
>
>
>
>
>
> Kindly help me out with this problem.
>
>
>
>
>
> Thanks,
>
> Satyan
>
>
>
>
>
> ---- On Wed, 23 May 2018 15:01:39 +0530 Chellasamy G &
> lt;[hidden email]&gt; wrote ----
>
>
>
>
> Hi,
>
>
>
>
>
>
>
> Thanks for the reply.
>
>
>
>
>
>
>
> Actually I need to implement it for StringField which is non-analyzed. So,
> if I am not wrong I can't add the analyzer for StringField.
>
>
>
>
>
>
>
>
>
>
>
> My scenario is something similar to the one discussed in the below thread,
>
>
>
>
>
>
>
>
> https://discuss.elastic.co/t/es-5-0-case-insensitive-search-for-keyword-fields/64111/10
>
>
>
>
>
>
>
> Could you please let me know how to do the same the thing in lucene.
>
>
>
>
>
>
>
>
>
>
>
> Thanks,
>
>
>
> Satyan
>
>
>
>
>
>
>
>
>
>
>
>
>
> ---- On Wed, 23 May 2018 12:09:31 +0530 Adrien Grand &
> amp;lt;[hidden email]&amp;gt; wrote ----
>
>
>
>
>
>
>
>
>
> Hi Satyan,
>
>
>
>
>
>
>
> You need to add a LowercaseTokenFilter to your analysis chain. The way to
>
>
>
> do it depends on how you are building your analyzer today (pre-built
>
>
>
> analyzer, extending Analyzer or using CustomAnalyzer). This will preserve
>
>
>
> the original case in field values because lowercasing will only be applied
>
>
>
> to the content of the inverted index, not stored fields where hits are
>
>
>
> fetched from.
>
>
>
>
>
>
>
> Le mer. 23 mai 2018 à 08:36, Chellasamy G &
> amp;lt;[hidden email]&amp;gt; a
>
>
>
> écrit :
>
>
>
>
>
>
>
> &amp;gt;
>
>
>
> &amp;gt;
>
>
>
> &amp;gt; Hi,
>
>
>
> &amp;gt;
>
>
>
> &amp;gt;
>
>
>
> &amp;gt;
>
>
>
> &amp;gt; I can't find any way to perform case insensitive search on
> StringField.
>
>
>
> &amp;gt; Please help me out.
>
>
>
> &amp;gt;
>
>
>
> &amp;gt;
>
>
>
> &amp;gt;
>
>
>
> &amp;gt;
>
>
>
> &amp;gt;
>
>
>
> &amp;gt; i.e If the field value is "Flying Robots", then the phrases
> "flying
>
>
>
> &amp;gt; robots", "fLying RObots" etc should match the value.
>
>
>
> &amp;gt;
>
>
>
> &amp;gt;
>
>
>
> &amp;gt;
>
>
>
> &amp;gt;
>
>
>
> &amp;gt;
>
>
>
> &amp;gt; I also need the original case of the field value to be preserved
> in the
>
>
>
> &amp;gt; search results.
>
>
>
> &amp;gt;
>
>
>
> &amp;gt;
>
>
>
> &amp;gt;
>
>
>
> &amp;gt;
>
>
>
> &amp;gt;
>
>
>
> &amp;gt; Thanks,
>
>
>
> &amp;gt;
>
>
>
> &amp;gt; Satyan
>
>
>
> &amp;gt;
>
>
>
> &amp;gt;
>
>
>
> &amp;gt;
>
>
>
> &amp;gt;
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Case Insensitive Search for StringField

Chellasamy G
In reply to this post by Ahmet Arslan
keyword tokenizer + lowercase filter resolved my problem.



Thank you all,

Satyan






---- On Sat, 26 May 2018 04:03:32 +0530 Ahmet Arslan &lt;[hidden email]&gt; wrote ----




Hi,

string_ci type could be constructed from: keyword tokenizer + lowercase filter + may be trim filter.

Ahmet

 On Friday, May 25, 2018, 1:50:19 PM GMT+3, Chellasamy G &lt;[hidden email]&gt; wrote:

 

 

 

Hi Team,

 

 

 

 

 

Kindly help me out with this problem.

 

 

 

 

 

Thanks,

 

Satyan

 

 

 

 

 

---- On Wed, 23 May 2018 15:01:39 +0530 Chellasamy G &amp;lt;[hidden email]&amp;gt; wrote ----

 

 

 

 

Hi,

 

 

 

 

 

 

 

Thanks for the reply.

 

 

 

 

 

 

 

Actually I need to implement it for StringField which is non-analyzed. So, if I am not wrong I can't add the analyzer for StringField.

 

 

 

 

 

 

 

 

 

 

 

My scenario is something similar to the one discussed in the below thread,

 

 

 

 

 

 

 

https://discuss.elastic.co/t/es-5-0-case-insensitive-search-for-keyword-fields/64111/10 

 

 

 

 

 

 

 

Could you please let me know how to do the same the thing in lucene.

 

 

 

 

 

 

 

 

 

 

 

Thanks,

 

 

 

Satyan

 

 

 

 

 

 

 

 

 

 

 

 

 

---- On Wed, 23 May 2018 12:09:31 +0530 Adrien Grand &amp;amp;lt;[hidden email]&amp;amp;gt; wrote ----

 

 

 

 

 

 

 

 

 

Hi Satyan,

 

 

 

 

 

 

 

You need to add a LowercaseTokenFilter to your analysis chain. The way to

 

 

 

do it depends on how you are building your analyzer today (pre-built

 

 

 

analyzer, extending Analyzer or using CustomAnalyzer). This will preserve

 

 

 

the original case in field values because lowercasing will only be applied

 

 

 

to the content of the inverted index, not stored fields where hits are

 

 

 

fetched from.

 

 

 

 

 

 

 

Le mer. 23 mai 2018 à 08:36, Chellasamy G &amp;amp;lt;[hidden email]&amp;amp;gt; a

 

 

 

écrit :

 

 

 

 

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt; Hi,

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt; I can't find any way to perform case insensitive search on StringField.

 

 

 

&amp;amp;gt; Please help me out.

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt; i.e If the field value is "Flying Robots", then the phrases "flying

 

 

 

&amp;amp;gt; robots", "fLying RObots" etc should match the value.

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt; I also need the original case of the field value to be preserved in the

 

 

 

&amp;amp;gt; search results.

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt; Thanks,

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt; Satyan

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt;

 

 

 

&amp;amp;gt;