[jira] [Commented] (LUCENE-8344) TokenStreamToAutomaton doesn't ignore trailing posInc when preservePositionIncrements=false

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-8344) TokenStreamToAutomaton doesn't ignore trailing posInc when preservePositionIncrements=false

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-8344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511879#comment-16511879 ]

Jim Ferenczi commented on LUCENE-8344:
--------------------------------------

+1 to the patch and +1 to backport to 7.4 especially if ConcatenateGraphFilter is released in this version.

> TokenStreamToAutomaton doesn't ignore trailing posInc when preservePositionIncrements=false
> -------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-8344
>                 URL: https://issues.apache.org/jira/browse/LUCENE-8344
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: modules/suggest
>            Reporter: David Smiley
>            Priority: Major
>         Attachments: LUCENE-8344.patch, LUCENE-8344.patch, LUCENE-8344.patch
>
>
> TokenStreamToAutomaton in Lucene core is used by the AnalyzingSuggester (incl. FuzzySuggester subclass ) and NRT Document Suggester and soon the SolrTextTagger. ¬†It has a setting¬†{{preservePositionIncrements}} defaulting to true.  If it's set to false (e.g. to ignore stopwords) and if there is a _trailing_ position increment greater than 1, TS2A will _still_ add position increments (holes) into the automata even though it was configured not to.
> I'm filing this issue separate from LUCENE-8332 where I first found it.  The fix is very simple but I'm concerned about back-compat ramifications so I'm filing it separately.  I'll attach a patch to show the problem.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]