[jira] Created: (LUCENE-2093) Use query-private scope instead of shared Term->TermInfo cache

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (LUCENE-2093) Use query-private scope instead of shared Term->TermInfo cache

Tim Allison (Jira)
Use query-private scope instead of shared Term->TermInfo cache
--------------------------------------------------------------

                 Key: LUCENE-2093
                 URL: https://issues.apache.org/jira/browse/LUCENE-2093
             Project: Lucene - Java
          Issue Type: Improvement
            Reporter: Michael McCandless
            Priority: Minor
             Fix For: 3.1


Spinoff of LUCENE-2075.

We currently use a shared terms cache so multiple resolves of the same term within execution of a single query save CPU.  But this ties up a good amount of long term RAM...

So, it might be better to instead create a "query private scope", where places in Lucene like the terms dict could store & retrieve results.  The scope would be private to each running query, and would be GCable as soon as the query completes.  Then we've have perfect within query hit rate...

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-2093) Use query-private scope instead of shared Term->TermInfo cache

Tim Allison (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-2093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12781910#action_12781910 ]

Michael McCandless commented on LUCENE-2093:
--------------------------------------------

If we don't do this in 3.1, we should at least drop the size of the terms dict cache -- by rough math, that cache will consume 4 MB on a 20 segment index, even for a smallish index.

When flex lands, the cache is no longer beneficial for automaton query so it need not be so large.

> Use query-private scope instead of shared Term->TermInfo cache
> --------------------------------------------------------------
>
>                 Key: LUCENE-2093
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2093
>             Project: Lucene - Java
>          Issue Type: Improvement
>            Reporter: Michael McCandless
>            Priority: Minor
>             Fix For: 3.1
>
>
> Spinoff of LUCENE-2075.
> We currently use a shared terms cache so multiple resolves of the same term within execution of a single query save CPU.  But this ties up a good amount of long term RAM...
> So, it might be better to instead create a "query private scope", where places in Lucene like the terms dict could store & retrieve results.  The scope would be private to each running query, and would be GCable as soon as the query completes.  Then we've have perfect within query hit rate...

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]