[jira] [Commented] (LUCENE-4857) StemmerOverrideFilter should not copy the stem override dictionary in it's ctor.

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (LUCENE-4857) StemmerOverrideFilter should not copy the stem override dictionary in it's ctor.

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-4857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13607641#comment-13607641 ]

Commit Tag Bot commented on LUCENE-4857:
----------------------------------------

[trunk commit] Simon Willnauer
http://svn.apache.org/viewvc?view=revision&revision=1458848

LUCENE-4857: Don't unnecessarily copy stem override map in StemmerOverrideFilter

               

> StemmerOverrideFilter should not copy the stem override dictionary in it's ctor.
> --------------------------------------------------------------------------------
>
>                 Key: LUCENE-4857
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4857
>             Project: Lucene - Core
>          Issue Type: Improvement
>          Components: modules/analysis
>    Affects Versions: 4.0, 4.1, 4.2
>            Reporter: Simon Willnauer
>            Priority: Minor
>             Fix For: 5.0, 4.2.1
>
>         Attachments: LUCENE-4857.patch
>
>
> Currently the dictionary is cloned each time the token filter is created which is a serious bottleneck if you use this filter with large dictionaries and can also lead to OOMs if lots of those filters sit in ThreadLocals and new threads are added etc. I think cloning the map should be done in the analyzer (which all of our analyzers do btw. but this is the only TF that does that) no need to really copy that map.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]