[jira] Created: (LUCENE-2561) Fix exception handling and thread safety in realtime branch

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (LUCENE-2561) Fix exception handling and thread safety in realtime branch

Markus Jelsma (Jira)
Fix exception handling and thread safety in realtime branch
-----------------------------------------------------------

                 Key: LUCENE-2561
                 URL: https://issues.apache.org/jira/browse/LUCENE-2561
             Project: Lucene - Java
          Issue Type: Bug
          Components: Index
            Reporter: Michael Busch
            Assignee: Michael Busch
            Priority: Minor
             Fix For: Realtime Branch


Several tests are currently failing in the realtime branch - most of them due to thread safety problems (often exceptions in ConcurrentMergeScheduler) and in tests that test for aborting and non-aborting exceptions.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-2561) Fix exception handling and thread safety in realtime branch

Markus Jelsma (Jira)

     [ https://issues.apache.org/jira/browse/LUCENE-2561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Busch updated LUCENE-2561:
----------------------------------

    Attachment: lucene-2561.patch

The patch fixes most of the threading and exception issues.

Now >99% of the core tests pass!  Some failures are expected, because some features aren't implemented yet (e.g. flush by RAM or maxBufferedDeletes).  A test though that I still want to fix with this patch is TestStressIndexing2 - not sure yet what's going on.

Other changes:
- Factored ReaderPool out of IndexWriter into its own class
- Added a FilterDirectory that forwards all method calls to a delegate
- Use an extended FilterDirectory in DW to track all files the consumers and codecs open, so that they can be closed on abort
- Fixed some more nocommits

Using the FilterDirectory might not be the cleanest approach?  Maybe a IndexOutputFactory or something would be cleaner?  Or maybe on abort we should just delete all files that have the prefix of the segment(s) the DWPT(s) were working on?  This should be possible now that the shared doc stores are gone and no files are shared anymore across segments.

> Fix exception handling and thread safety in realtime branch
> -----------------------------------------------------------
>
>                 Key: LUCENE-2561
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2561
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>            Reporter: Michael Busch
>            Assignee: Michael Busch
>            Priority: Minor
>             Fix For: Realtime Branch
>
>         Attachments: lucene-2561.patch
>
>
> Several tests are currently failing in the realtime branch - most of them due to thread safety problems (often exceptions in ConcurrentMergeScheduler) and in tests that test for aborting and non-aborting exceptions.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-2561) Fix exception handling and thread safety in realtime branch

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-2561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12892918#action_12892918 ]

Michael Busch commented on LUCENE-2561:
---------------------------------------

Committed the current patch to the realtime branch. (revision 979856)

Leaving this issue open to fix TestStressIndexing2 and for more cleanup.

> Fix exception handling and thread safety in realtime branch
> -----------------------------------------------------------
>
>                 Key: LUCENE-2561
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2561
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>            Reporter: Michael Busch
>            Assignee: Michael Busch
>            Priority: Minor
>             Fix For: Realtime Branch
>
>         Attachments: lucene-2561.patch
>
>
> Several tests are currently failing in the realtime branch - most of them due to thread safety problems (often exceptions in ConcurrentMergeScheduler) and in tests that test for aborting and non-aborting exceptions.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Resolved: (LUCENE-2561) Fix exception handling and thread safety in realtime branch

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

     [ https://issues.apache.org/jira/browse/LUCENE-2561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Busch resolved LUCENE-2561.
-----------------------------------

    Resolution: Fixed

TestStressIndexing2 is not failing because of concurrency problems, so I'm closing this issue.
All contrib tests pass now too!  

The reason why TestStressIndexing2 is failing is that deletes and sequenceIDs aren't fully implemented yet.  The remapDeletes step is still commented out, which results in a wrong behavior as soon as segment merges happen while deletes are buffered.  (I'll use LUCENE-2558 to fix that)

> Fix exception handling and thread safety in realtime branch
> -----------------------------------------------------------
>
>                 Key: LUCENE-2561
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2561
>             Project: Lucene - Java
>          Issue Type: Bug
>          Components: Index
>            Reporter: Michael Busch
>            Assignee: Michael Busch
>            Priority: Minor
>             Fix For: Realtime Branch
>
>         Attachments: lucene-2561.patch
>
>
> Several tests are currently failing in the realtime branch - most of them due to thread safety problems (often exceptions in ConcurrentMergeScheduler) and in tests that test for aborting and non-aborting exceptions.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]