Solr list operator

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Solr list operator

Nick Way
Hi, I have a custom field "listOfIDs" = "1,2,4,33"

I want the equivalent of:

select * where '1' IN (listOfIDs)  --> should get a match

select * where '33' IN (listOfIDs)  --> should get a match

select * where '3' IN (listOfIDs)  --> should NOT get a match


Can anyone help me out please as I can't seem to find any documentation on
this. Thanks very much in advance.

Kind regards,


Nick Way
Reply | Threaded
Open this post in threaded view
|

Re: Solr list operator

alex goretoy
https://www.youtube.com/watch?v=pNe1wWeaHOU&list=PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1
https://www.youtube.com/watch?v=pNe1wWeaHOU&list=PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1

http://audiobible.life CHECK IT OUT!


On Wed, Sep 6, 2017 at 5:57 PM, Nick Way <[hidden email]> wrote:

> Hi, I have a custom field "listOfIDs" = "1,2,4,33"
>
> I want the equivalent of:
>
> select * where '1' IN (listOfIDs)  --> should get a match
>
> select * where '33' IN (listOfIDs)  --> should get a match
>
> select * where '3' IN (listOfIDs)  --> should NOT get a match
>
>
> Can anyone help me out please as I can't seem to find any documentation on
> this. Thanks very much in advance.
>
> Kind regards,
>
>
> Nick Way
Reply | Threaded
Open this post in threaded view
|

Re: Solr list operator

Susheel Kumar-3
Nick, checkout terms query parser
http://lucene.apache.org/solr/guide/6_6/other-parsers.html or streaming
expressions.

Thnx

On Wed, Sep 6, 2017 at 8:33 AM, alex goretoy <[hidden email]> wrote:

> https://www.youtube.com/watch?v=pNe1wWeaHOU&list=
> PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1
> https://www.youtube.com/watch?v=pNe1wWeaHOU&list=
> PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1
>
> http://audiobible.life CHECK IT OUT!
>
>
> On Wed, Sep 6, 2017 at 5:57 PM, Nick Way <[hidden email]>
> wrote:
> > Hi, I have a custom field "listOfIDs" = "1,2,4,33"
> >
> > I want the equivalent of:
> >
> > select * where '1' IN (listOfIDs)  --> should get a match
> >
> > select * where '33' IN (listOfIDs)  --> should get a match
> >
> > select * where '3' IN (listOfIDs)  --> should NOT get a match
> >
> >
> > Can anyone help me out please as I can't seem to find any documentation
> on
> > this. Thanks very much in advance.
> >
> > Kind regards,
> >
> >
> > Nick Way
>
Reply | Threaded
Open this post in threaded view
|

Re: Solr list operator

Walter Underwood
Use a multivalued field. Search for listOfIds:1. Or search for listOfIds:33. This is one of the simplest things that Solr can do.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)


> On Sep 6, 2017, at 6:07 AM, Susheel Kumar <[hidden email]> wrote:
>
> Nick, checkout terms query parser
> http://lucene.apache.org/solr/guide/6_6/other-parsers.html or streaming
> expressions.
>
> Thnx
>
> On Wed, Sep 6, 2017 at 8:33 AM, alex goretoy <[hidden email]> wrote:
>
>> https://www.youtube.com/watch?v=pNe1wWeaHOU&list=
>> PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1
>> https://www.youtube.com/watch?v=pNe1wWeaHOU&list=
>> PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1
>>
>> http://audiobible.life CHECK IT OUT!
>>
>>
>> On Wed, Sep 6, 2017 at 5:57 PM, Nick Way <[hidden email]>
>> wrote:
>>> Hi, I have a custom field "listOfIDs" = "1,2,4,33"
>>>
>>> I want the equivalent of:
>>>
>>> select * where '1' IN (listOfIDs)  --> should get a match
>>>
>>> select * where '33' IN (listOfIDs)  --> should get a match
>>>
>>> select * where '3' IN (listOfIDs)  --> should NOT get a match
>>>
>>>
>>> Can anyone help me out please as I can't seem to find any documentation
>> on
>>> this. Thanks very much in advance.
>>>
>>> Kind regards,
>>>
>>>
>>> Nick Way
>>

Reply | Threaded
Open this post in threaded view
|

Re: Solr list operator

Erick Erickson
You'll have to split up the input on commas if you don't just do it
the multiValued way Walter suggests, perhaps one of the pattern
tokenizers mentioned here:

https://cwiki.apache.org/confluence/display/solr/Tokenizers

Best,
Erick

On Wed, Sep 6, 2017 at 6:29 AM, Walter Underwood <[hidden email]> wrote:

> Use a multivalued field. Search for listOfIds:1. Or search for listOfIds:33. This is one of the simplest things that Solr can do.
>
> wunder
> Walter Underwood
> [hidden email]
> http://observer.wunderwood.org/  (my blog)
>
>
>> On Sep 6, 2017, at 6:07 AM, Susheel Kumar <[hidden email]> wrote:
>>
>> Nick, checkout terms query parser
>> http://lucene.apache.org/solr/guide/6_6/other-parsers.html or streaming
>> expressions.
>>
>> Thnx
>>
>> On Wed, Sep 6, 2017 at 8:33 AM, alex goretoy <[hidden email]> wrote:
>>
>>> https://www.youtube.com/watch?v=pNe1wWeaHOU&list=
>>> PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1
>>> https://www.youtube.com/watch?v=pNe1wWeaHOU&list=
>>> PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1
>>>
>>> http://audiobible.life CHECK IT OUT!
>>>
>>>
>>> On Wed, Sep 6, 2017 at 5:57 PM, Nick Way <[hidden email]>
>>> wrote:
>>>> Hi, I have a custom field "listOfIDs" = "1,2,4,33"
>>>>
>>>> I want the equivalent of:
>>>>
>>>> select * where '1' IN (listOfIDs)  --> should get a match
>>>>
>>>> select * where '33' IN (listOfIDs)  --> should get a match
>>>>
>>>> select * where '3' IN (listOfIDs)  --> should NOT get a match
>>>>
>>>>
>>>> Can anyone help me out please as I can't seem to find any documentation
>>> on
>>>> this. Thanks very much in advance.
>>>>
>>>> Kind regards,
>>>>
>>>>
>>>> Nick Way
>>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Solr list operator

Nick Way
In reply to this post by Nick Way
Thank you very much Erik, Walter and Susheel.

To be honest I didn't really understand the suggested routes (due to my
limited knowledge) but managed to get things working by inserting my data
with a double comma at the beginning eg:

custom field "listOfIDs" = ",,1,2,4,33"

and then searching for "*,myVal,*" which seems to work.

Out of interest does anyone have experience accessing Solr via Adobe
Coldfusion (as this is what we do) - and it would be helpful to have a
contact for some Solr consulting from time to time, if anyone might be
interested in that?

​Thank you very much for your help which was much appreciated.

Best,

Nick

On 6 September 2017 at 16:46, Erick Erickson <[hidden email]>
wrote:

> You'll have to split up the input on commas if you don't just do it
> the multiValued way Walter suggests, perhaps one of the pattern
> tokenizers mentioned here:
>
> https://cwiki.apache.org/confluence/display/solr/Tokenizers
>
> Best,
> Erick
>
> On Wed, Sep 6, 2017 at 6:29 AM, Walter Underwood <[hidden email]>
> wrote:
> > Use a multivalued field. Search for listOfIds:1. Or search for
> listOfIds:33. This is one of the simplest things that Solr can do.
> >
> > wunder
> > Walter Underwood
> > [hidden email]
> > http://observer.wunderwood.org/  (my blog)
> >
> >
> >> On Sep 6, 2017, at 6:07 AM, Susheel Kumar <[hidden email]>
> wrote:
> >>
> >> Nick, checkout terms query parser
> >> http://lucene.apache.org/solr/guide/6_6/other-parsers.html or streaming
> >> expressions.
> >>
> >> Thnx
> >>
> >> On Wed, Sep 6, 2017 at 8:33 AM, alex goretoy <[hidden email]> wrote:
> >>
> >>> https://www.youtube.com/watch?v=pNe1wWeaHOU&list=
> >>> PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1
> >>> https://www.youtube.com/watch?v=pNe1wWeaHOU&list=
> >>> PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1
> >>>
> >>> http://audiobible.life CHECK IT OUT!
> >>>
> >>>
> >>> On Wed, Sep 6, 2017 at 5:57 PM, Nick Way <[hidden email]
> >
> >>> wrote:
> >>>> Hi, I have a custom field "listOfIDs" = "1,2,4,33"
> >>>>
> >>>> I want the equivalent of:
> >>>>
> >>>> select * where '1' IN (listOfIDs)  --> should get a match
> >>>>
> >>>> select * where '33' IN (listOfIDs)  --> should get a match
> >>>>
> >>>> select * where '3' IN (listOfIDs)  --> should NOT get a match
> >>>>
> >>>>
> >>>> Can anyone help me out please as I can't seem to find any
> documentation
> >>> on
> >>>> this. Thanks very much in advance.
> >>>>
> >>>> Kind regards,
> >>>>
> >>>>
> >>>> Nick Way
> >>>
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Solr list operator

Walter Underwood
That is not the way to do it. It is slow and will not scale. Wildcards are usually not a good idea in Solr.

The simplest way is to search for the numbers as words. Index the string “1 2 4 33”. Search for one of those words, like listOfIDs:4.

It is better to index the numbers as an array, but I don’t know anything about the ColdFusion API. I last looked at ColdFusion in 1997.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)


> On Sep 12, 2017, at 6:21 AM, Nick Way <[hidden email]> wrote:
>
> Thank you very much Erik, Walter and Susheel.
>
> To be honest I didn't really understand the suggested routes (due to my
> limited knowledge) but managed to get things working by inserting my data
> with a double comma at the beginning eg:
>
> custom field "listOfIDs" = ",,1,2,4,33"
>
> and then searching for "*,myVal,*" which seems to work.
>
> Out of interest does anyone have experience accessing Solr via Adobe
> Coldfusion (as this is what we do) - and it would be helpful to have a
> contact for some Solr consulting from time to time, if anyone might be
> interested in that?
>
> ​Thank you very much for your help which was much appreciated.
>
> Best,
>
> Nick
>
> On 6 September 2017 at 16:46, Erick Erickson <[hidden email]>
> wrote:
>
>> You'll have to split up the input on commas if you don't just do it
>> the multiValued way Walter suggests, perhaps one of the pattern
>> tokenizers mentioned here:
>>
>> https://cwiki.apache.org/confluence/display/solr/Tokenizers
>>
>> Best,
>> Erick
>>
>> On Wed, Sep 6, 2017 at 6:29 AM, Walter Underwood <[hidden email]>
>> wrote:
>>> Use a multivalued field. Search for listOfIds:1. Or search for
>> listOfIds:33. This is one of the simplest things that Solr can do.
>>>
>>> wunder
>>> Walter Underwood
>>> [hidden email]
>>> http://observer.wunderwood.org/  (my blog)
>>>
>>>
>>>> On Sep 6, 2017, at 6:07 AM, Susheel Kumar <[hidden email]>
>> wrote:
>>>>
>>>> Nick, checkout terms query parser
>>>> http://lucene.apache.org/solr/guide/6_6/other-parsers.html or streaming
>>>> expressions.
>>>>
>>>> Thnx
>>>>
>>>> On Wed, Sep 6, 2017 at 8:33 AM, alex goretoy <[hidden email]> wrote:
>>>>
>>>>> https://www.youtube.com/watch?v=pNe1wWeaHOU&list=
>>>>> PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1
>>>>> https://www.youtube.com/watch?v=pNe1wWeaHOU&list=
>>>>> PLYI8318YYdkCsZ7dsYV01n6TZhXA6Wf9i&index=1
>>>>>
>>>>> http://audiobible.life CHECK IT OUT!
>>>>>
>>>>>
>>>>> On Wed, Sep 6, 2017 at 5:57 PM, Nick Way <[hidden email]
>>>
>>>>> wrote:
>>>>>> Hi, I have a custom field "listOfIDs" = "1,2,4,33"
>>>>>>
>>>>>> I want the equivalent of:
>>>>>>
>>>>>> select * where '1' IN (listOfIDs)  --> should get a match
>>>>>>
>>>>>> select * where '33' IN (listOfIDs)  --> should get a match
>>>>>>
>>>>>> select * where '3' IN (listOfIDs)  --> should NOT get a match
>>>>>>
>>>>>>
>>>>>> Can anyone help me out please as I can't seem to find any
>> documentation
>>>>> on
>>>>>> this. Thanks very much in advance.
>>>>>>
>>>>>> Kind regards,
>>>>>>
>>>>>>
>>>>>> Nick Way
>>>>>
>>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Solr list operator

Shawn Heisey-2
In reply to this post by Nick Way
On 9/12/2017 7:21 AM, Nick Way wrote:
> Thank you very much Erik, Walter and Susheel.
>
> To be honest I didn't really understand the suggested routes (due to my
> limited knowledge) but managed to get things working by inserting my data
> with a double comma at the beginning eg:
>
> custom field "listOfIDs" = ",,1,2,4,33"
>
> and then searching for "*,myVal,*" which seems to work.

If you're going to index it as a single string and leave it that way,
that will be your only real option.  A wildcard search is generally
slower than other options.

Instead, you should break that list apart in your indexing software, and
index multiple values for that field instead of a single value
containing commas.  Then you can search for single values easily and
quickly.  To do this, the field must be marked as multiValued in your
schema.

Alternatively, you could use a TextField type in your schema and include
a tokenizer or filter that will split on the commas.  If the field is
stored, the search results would contain the original comma separated
string, not the separate values.  Also, if the list will always be
numbers, you would not be able to do a numeric range query, because the
values would be strings, not numbers.

> Out of interest does anyone have experience accessing Solr via Adobe
> Coldfusion (as this is what we do) - and it would be helpful to have a
> contact for some Solr consulting from time to time, if anyone might be
> interested in that?

I have never done any work in ColdFusion.

There are some CF Solr libraries.  This page also says that version 9
includes it natively:

https://wiki.apache.org/solr/IntegratingSolr

Thanks,
Shawn