[jira] Created: (LUCENE-1177) IW.optimize() can do too many merges at the very end

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (LUCENE-1177) IW.optimize() can do too many merges at the very end

JIRA jira@apache.org
IW.optimize() can do too many merges at the very end
----------------------------------------------------

                 Key: LUCENE-1177
                 URL: https://issues.apache.org/jira/browse/LUCENE-1177
             Project: Lucene - Java
          Issue Type: Bug
          Components: Index
    Affects Versions: 2.3
            Reporter: Michael McCandless
            Assignee: Michael McCandless
            Priority: Minor
             Fix For: 2.4


This was fixed on trunk in LUCENE-1044 but I'd like to separately
backport it to 2.3.

With ConcurrentMergeScheduler there is a bug, only when CFS is on,
whereby after the final merge of an optimize has finished and while
it's building its CFS, the merge policy may incorrectly ask for
another merge to collapse that segment into a compound file.  The net
effect is optimize can spend many extra iterations unecessarily
merging a single segment to collapse it to compound file.

I believe the case is rare (hard to hit), and maybe only if you have
multiple threads calling optimize at once (the TestThreadedOptimize
test can hit it), but it's a low-risk fix so I plan to commit to 2.3
shortly.



--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-1177) IW.optimize() can do too many merges at the very end

JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-1177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael McCandless updated LUCENE-1177:
---------------------------------------

    Attachment: LUCENE-1177.patch

Attached patch.  Will commit shortly to 2.3.

> IW.optimize() can do too many merges at the very end
> ----------------------------------------------------
>
>                 Key: LUCENE-1177
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1177
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 2.3
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 2.4
>
>         Attachments: LUCENE-1177.patch
>
>
> This was fixed on trunk in LUCENE-1044 but I'd like to separately
> backport it to 2.3.
> With ConcurrentMergeScheduler there is a bug, only when CFS is on,
> whereby after the final merge of an optimize has finished and while
> it's building its CFS, the merge policy may incorrectly ask for
> another merge to collapse that segment into a compound file.  The net
> effect is optimize can spend many extra iterations unecessarily
> merging a single segment to collapse it to compound file.
> I believe the case is rare (hard to hit), and maybe only if you have
> multiple threads calling optimize at once (the TestThreadedOptimize
> test can hit it), but it's a low-risk fix so I plan to commit to 2.3
> shortly.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Resolved: (LUCENE-1177) IW.optimize() can do too many merges at the very end

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-1177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael McCandless resolved LUCENE-1177.
----------------------------------------

    Resolution: Fixed

Committed to 2.3 (and was already fixed on trunk).

> IW.optimize() can do too many merges at the very end
> ----------------------------------------------------
>
>                 Key: LUCENE-1177
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1177
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 2.3
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 2.4
>
>         Attachments: LUCENE-1177.patch
>
>
> This was fixed on trunk in LUCENE-1044 but I'd like to separately
> backport it to 2.3.
> With ConcurrentMergeScheduler there is a bug, only when CFS is on,
> whereby after the final merge of an optimize has finished and while
> it's building its CFS, the merge policy may incorrectly ask for
> another merge to collapse that segment into a compound file.  The net
> effect is optimize can spend many extra iterations unecessarily
> merging a single segment to collapse it to compound file.
> I believe the case is rare (hard to hit), and maybe only if you have
> multiple threads calling optimize at once (the TestThreadedOptimize
> test can hit it), but it's a low-risk fix so I plan to commit to 2.3
> shortly.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-1177) IW.optimize() can do too many merges at the very end

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-1177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hoss Man updated LUCENE-1177:
-----------------------------

    Fix Version/s: 2.3.1

> IW.optimize() can do too many merges at the very end
> ----------------------------------------------------
>
>                 Key: LUCENE-1177
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1177
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>    Affects Versions: 2.3
>            Reporter: Michael McCandless
>            Assignee: Michael McCandless
>            Priority: Minor
>             Fix For: 2.3.1, 2.4
>
>         Attachments: LUCENE-1177.patch
>
>
> This was fixed on trunk in LUCENE-1044 but I'd like to separately
> backport it to 2.3.
> With ConcurrentMergeScheduler there is a bug, only when CFS is on,
> whereby after the final merge of an optimize has finished and while
> it's building its CFS, the merge policy may incorrectly ask for
> another merge to collapse that segment into a compound file.  The net
> effect is optimize can spend many extra iterations unecessarily
> merging a single segment to collapse it to compound file.
> I believe the case is rare (hard to hit), and maybe only if you have
> multiple threads calling optimize at once (the TestThreadedOptimize
> test can hit it), but it's a low-risk fix so I plan to commit to 2.3
> shortly.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]