Autocomplete returning shingles

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Autocomplete returning shingles

O. Klein
I need to use autocomplete with edismax (ngrams,edgegrams) to return shingled
suggestions. Field value "new york city" needs to return on query "ne" ->
"new","new york","new york city". With suggester this is easy. But im forced
to use edismax because I need to apply mutliple filter queries.

What is best approach to deal with this?

Any suggestions are appreciated.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Autocomplete returning shingles

Federico Méndez
Can you just add the SingleFilter to your field?
https://lucene.apache.org/solr/guide/6_6/filter-descriptions.html#FilterDescriptions-ShingleFilter

<analyzer>
  <tokenizer class="solr.StandardTokenizerFactory"/>
  <filter class="solr.ShingleFilterFactory"/></analyzer>



On Wed, May 2, 2018 at 2:04 PM, O. Klein <[hidden email]> wrote:

> I need to use autocomplete with edismax (ngrams,edgegrams) to return
> shingled
> suggestions. Field value "new york city" needs to return on query "ne" ->
> "new","new york","new york city". With suggester this is easy. But im
> forced
> to use edismax because I need to apply mutliple filter queries.
>
> What is best approach to deal with this?
>
> Any suggestions are appreciated.
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>
Reply | Threaded
Open this post in threaded view
|

Re: Autocomplete returning shingles

Alessandro Benedetti
So, your problem is you want to return shingle suggestions from a field in
input but apply multiple filter queries to the documents you want to fetch
suggestions from.

Are you building an auxiliary index for that ?
You need to design it accordingly.
If you want to to map each suggestion to a single document in the auxiliary
index, when you build this auxiliary index you need to calculate the
shingles client side and push the multiple documents ( suggestion) per
original field content.

To do that automatically in Solr I was thinking you could write an
UpdateRequestProcessor that given in input the document, split it in
multiple docs, but unfortunately the current architecture of
UpdateRequestProcessors takes in input 1 Doc and and returns in output just
1 doc.
So it is not a viable approach.

Unfortunately the shingle filter here doesn't help, as the user want shingle
in output ( analysers doesn't affect stored content)

Cheers




-----
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
Reply | Threaded
Open this post in threaded view
|

Re: Autocomplete returning shingles

O. Klein
Yes, splitting in more documents would probably work.

Don't think I can do this easliy with Solr.

Looking into using facets now.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Autocomplete returning shingles

Alessandro Benedetti
Yes, faceting will work, you can use an old approach used for
autocompletion[1] .
Be sure you add the shingle filter to the appropriate index time analysis
for the field you want.
Facet values are extracted from the indexed terms, so calculating faceting
and filtering by prefix should do the trick.

[1]
https://solr.pl/en/2013/03/25/autocomplete-on-multivalued-fields-using-faceting/



-----
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io