suggestors on shingles

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

suggestors on shingles

govind nitk
Hi,

I have a fieldtype "suggestion" with definition as :


*<fieldType name="suggestion" class="solr.TextField"
positionIncrementGap="100">*
*      <analyzer type="index">*
*        <tokenizer class="solr.StandardTokenizerFactory"/>*
*        <filter class="solr.LowerCaseFilterFactory"/>*
*        <filter class="solr.ShingleFilterFactory" minShingleSize="3"
maxShingleSize="6" outputUnigramsIfNoShingles="true" /> *
*      </analyzer>*
*      <analyzer type="query">*
*        <tokenizer class="solr.WhitespaceTokenizerFactory"/>*
*        <filter class="solr.LowerCaseFilterFactory"/>*

*      </analyzer>*
*    </fieldType>*

I have a field named "mysuggestion" with definition as :

*<field name="mysuggestion" type="suggestion" multiValued="true"
indexed="true" stored="false"/>*

*I copy other field - names, countries, short description to
"mysuggestion".*


I am building a suggestor on top of this field as:

*    <lst name="suggester">*
*      <str name="name">fuzzySuggester</str>*
*      <str name="lookupImpl">FuzzyLookupFactory</str>*
*      <str name="dictionaryImpl">DocumentDictionaryFactory</str>*
*      <str name="buildOnStartup">true</str>*
*      <str name="buildOnCommit">true</str>*
*      <str name="field">mysuggestion</str>*
*      <str name="exactMatchFirst">true</str>*
*      <str name="suggestAnalyzerFieldType">suggestion</str> *
*    </lst>*



Expectation: returned Syggestions should be shingles not the entire line of
description or name.

1. Is it possible to pass suggestors a tokenized/analyzed field ?
2. Is it possible to retrieve tokenized values from solr ?


Regards,
Govind
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: suggestors on shingles

alessandro.benedetti
I would recommend this blog of mine to get a better understanding of how tokenization and the suggester work together [1] .

If you take a look to the FuzzyLookupFactory, you will see that it is one of the suggesters that return the entire content of the field.

You may be interested to the FreeTextLookupFactory.

Cheers


[1] http://alexbenedetti.blogspot.co.uk/2015/07/solr-you-complete-me.html
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: suggestors on shingles

govind nitk
Hi Alessandro,

Thanks a lot. I followed your blog and able to get the suggestions.

But I am curious about solr returning the tokenized data.
So, lot of filtering happended, shingles are generated, now Is it possible
to get those finally genrated tokens any way ?


Regards,
Govind


On Thu, Jul 13, 2017 at 4:00 PM, alessandro.benedetti <[hidden email]>
wrote:

> I would recommend this blog of mine to get a better understanding of how
> tokenization and the suggester work together [1] .
>
> If you take a look to the FuzzyLookupFactory, you will see that it is one
> of
> the suggesters that return the entire content of the field.
>
> You may be interested to the FreeTextLookupFactory.
>
> Cheers
>
>
> [1] http://alexbenedetti.blogspot.co.uk/2015/07/solr-you-complete-me.html
>
>
>
> -----
> ---------------
> Alessandro Benedetti
> Search Consultant, R&D Software Engineer, Director
> Sease Ltd. - www.sease.io
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/suggestors-on-shingles-tp4345763p4345793.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: suggestors on shingles

alessandro.benedetti
To do what ?
If it is a use case, please explain us.

If it is just to check that the analysis chain worked correctly, you can check the schema browser or use Luke.

If you just want to test your analysis chain, you can use the analysis tool in the Solr admin.

Cheers
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: suggestors on shingles

govind nitk
Hi Alessandro,

Currently fuzzy or lookup suggester is not supporting - returning
suggestions generated from the shingles at index time.
As you guided, I am using FreeText suggester. But freetext suggester won't
support the fuzzyness.

Let me know your inputs on this edge case.


Regards,
Govind




On Thu, Jul 13, 2017 at 8:03 PM, alessandro.benedetti <[hidden email]>
wrote:

> To do what ?
> If it is a use case, please explain us.
>
> If it is just to check that the analysis chain worked correctly, you can
> check the schema browser or use Luke.
>
> If you just want to test your analysis chain, you can use the analysis tool
> in the Solr admin.
>
> Cheers
>
>
>
> -----
> ---------------
> Alessandro Benedetti
> Search Consultant, R&D Software Engineer, Director
> Sease Ltd. - www.sease.io
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/suggestors-on-shingles-tp4345763p4345836.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Loading...