[jira] Created: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

classic Classic list List threaded Threaded
47 messages Options
123
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
-------------------------------------------------------------------------

                 Key: SOLR-1155
                 URL: https://issues.apache.org/jira/browse/SOLR-1155
             Project: Solr
          Issue Type: Improvement
          Components: search
    Affects Versions: 1.3
            Reporter: Jayson Minard


Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  

See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707572#action_12707572 ]

Jayson Minard commented on SOLR-1155:
-------------------------------------

This will involve a change to locking as seen in the thread mentioned above between add/delete/commit methods, but there are also synchronized methods on the CommitTracker scehduleCommitWithin method, run method, and getCommitCount method that will cause similar blocking.  

> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707575#action_12707575 ]

Jayson Minard commented on SOLR-1155:
-------------------------------------

Yonik mentioned: "We'd need to think about what type of synchronization may be needed
for postCommit and postOptimize hooks too."

Is the expectation for those callbacks that nothing will change until the callback completes?  (i.e. a script might run a process that scans the index after optimize expecting it to remain optimized during the script execution)   Or is that not the contract of those callbacks?


> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Issue Comment Edited: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707572#action_12707572 ]

Jayson Minard edited comment on SOLR-1155 at 5/8/09 7:52 PM:
-------------------------------------------------------------

This will involve a change to locking as seen in the thread mentioned above between add/delete/commit methods, but there are also synchronized methods on the CommitTracker scehduleCommitWithin method, run method, and getCommitCount method that will cause similar blocking at least for deletes.

      was (Author: jminard):
    This will involve a change to locking as seen in the thread mentioned above between add/delete/commit methods, but there are also synchronized methods on the CommitTracker scehduleCommitWithin method, run method, and getCommitCount method that will cause similar blocking.  
 

> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707600#action_12707600 ]

Jayson Minard commented on SOLR-1155:
-------------------------------------

Thinking about pending counts as well.  If adds are ongoing during a commit, there has to be a point in time where the counters clear the number of documents committed.  Currently they are set to 0 which is safe since nothing was ongoing while the commit happened.  Something to remember...

> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707602#action_12707602 ]

Jayson Minard commented on SOLR-1155:
-------------------------------------

Preparing a first patch, calling it DirectUpdateHandler3 as there is a lot of reshuffling and a new commit tracker implementation that is lighter-weight.

> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jayson Minard updated SOLR-1155:
--------------------------------

    Attachment: DirectUpdateHandler3.java

I created DirectUpdateHandler3 as a copy of DirectUpdateHandler2 and then did a pass across it rebuilding the locking, stats, and tracker.  I am basically at the "IDE show no errors" stage and going to read it a few more times, and start testing.  Feedback is welcome as I'm in the dark on a few areas of this code and I am working from a lot of assumptions.  Some event placeholder methods remain while I think through the issues, will trim it back when done.

Events to the auto commit tracker are within the locks to help avoid cases where we set the counts to 0 after other work has concurrently been done.  The stats information has similar concerns and is not yet dealt with, so some stats might reset to 0 on commit when there was further work being done (so they should not go to 0).

> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>         Attachments: DirectUpdateHandler3.java
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jayson Minard updated SOLR-1155:
--------------------------------

    Attachment: DirectUpdateHandler3.java

Updated with different locking around the post commit/optimize handlers.  There is a gap in time between commit and these calls, is that safe?  left TODO's in the code at places of concern.

> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>         Attachments: DirectUpdateHandler3.java, DirectUpdateHandler3.java
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jayson Minard updated SOLR-1155:
--------------------------------

    Attachment: SOLR-1155.patch
                DirectUpdateHandlerWithAutoCommit.java
                DirectUpdateHandler3.java

Updated file, and associated interface used to help DirectUpdateHandler2 and 3 co-exist with tests, SnapPuller and ReplicationHandler classes.

A larger patch with all changes to tests, config files, and other items makeing DirectUpdateHandler3 the default (purely for running all of the unit tests using it) is attached as well.  

Patch includes fix for SOLR-1157 which got in the way of running unit tests.


> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>         Attachments: DirectUpdateHandler3.java, DirectUpdateHandler3.java, DirectUpdateHandler3.java, DirectUpdateHandlerWithAutoCommit.java, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707636#action_12707636 ]

Jayson Minard commented on SOLR-1155:
-------------------------------------

All tests pass with the attached patch file and DirectUpdateHandler3 as the handler for each solrconfig variant in the test-files directory.  Needs review for locking holes and some of the TODO comments answered still.  And needs a lot of concurrent update testing.

> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>         Attachments: DirectUpdateHandler3.java, DirectUpdateHandler3.java, DirectUpdateHandler3.java, DirectUpdateHandlerWithAutoCommit.java, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707672#action_12707672 ]

Jayson Minard commented on SOLR-1155:
-------------------------------------

Things likely needing further work:

* serialize commits?  (two shouldn't happen at same time)
* optimize should us iwManageWriter lock from optimize through call-back call, so break it away from commit which only needs iwUseWriter?)  This also takes care of the contract that optimize callback is still on an optimized index.

> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>         Attachments: DirectUpdateHandler3.java, DirectUpdateHandler3.java, DirectUpdateHandler3.java, DirectUpdateHandlerWithAutoCommit.java, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jayson Minard updated SOLR-1155:
--------------------------------

    Attachment: SOLR-1155.patch

Fixed some of the issue mentioned in the comment above.  Split commit and optimize work out since optimize has a stronger need to lock and be consistent.

> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>         Attachments: SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jayson Minard updated SOLR-1155:
--------------------------------

    Attachment:     (was: DirectUpdateHandler3.java)

> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>         Attachments: SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jayson Minard updated SOLR-1155:
--------------------------------

    Attachment:     (was: DirectUpdateHandlerWithAutoCommit.java)

> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>         Attachments: SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jayson Minard updated SOLR-1155:
--------------------------------

    Attachment:     (was: DirectUpdateHandler3.java)

> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>         Attachments: SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jayson Minard updated SOLR-1155:
--------------------------------

    Attachment:     (was: DirectUpdateHandler3.java)

> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>         Attachments: SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707692#action_12707692 ]

Jayson Minard commented on SOLR-1155:
-------------------------------------

new TODO...

Right now auto commit will eventually fire off a commit that might run into a manual commit or optimize and sit at the syncrhonization block, then fire off on the tail of the other action.  I would rather it wait until those running commands clear and THEN evaluate if it needs to do a commit or not since its information it used to make the decision could be moot due to the other action that beat it in.  So if a commit or optimize is running it will not evaluate for auto commit until they complete.  


> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>         Attachments: SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jayson Minard updated SOLR-1155:
--------------------------------

    Attachment: SOLR-1155.patch

New patch resolves the serialization of commit/optimize/close/rollback actions that should not be concurrent with each other.  Also protect methods after close to keep other threads from accidentally re-opening writer.

> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>         Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707701#action_12707701 ]

Jayson Minard commented on SOLR-1155:
-------------------------------------

Does anyone know if the UpdateHandler.close() method is called when everything else is shut down and no transactions can come through?  If so, I can remove the blocking added to prevent a writer from re-opening after close is called.  

> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>         Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow concurrent adds during an autocommit

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jayson Minard updated SOLR-1155:
--------------------------------

    Attachment: SOLR-1155.patch

move wait for searcher out of locks in optimize and commit

> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
>                 Key: SOLR-1155
>                 URL: https://issues.apache.org/jira/browse/SOLR-1155
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.3
>            Reporter: Jayson Minard
>         Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.  
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

123