[jira] [Commented] (SOLR-12338) Replay buffering tlog in parallel

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-12338) Replay buffering tlog in parallel

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/SOLR-12338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16479083#comment-16479083 ]

David Smiley commented on SOLR-12338:
-------------------------------------

{quote}Upload a patch that makes a change from using an array of lock into a {{SetBlockingQueue}}.
{quote}
BTW I've twice gotten confused in this issue conversation when you referred to things I didn't know existed before because it was unclear if I simply didn't know about it or if you were adding/introducing some new mechanism.  It would be helpful to me if you try to clarify that new things are new things, e.g. "(added in this patch)" or "added a new ..." or some-such.

It's super tempting to simply use Striped as it's difficult to write & review concurrent control structures such as this.  I have a bunch of pending commentary/review for your SetBlockingQueue but are you choosing to not use it because the numThreads * 1000 is too much internal memory/waste?

> Replay buffering tlog in parallel
> ---------------------------------
>
>                 Key: SOLR-12338
>                 URL: https://issues.apache.org/jira/browse/SOLR-12338
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public)
>            Reporter: Cao Manh Dat
>            Assignee: Cao Manh Dat
>            Priority: Major
>         Attachments: SOLR-12338.patch, SOLR-12338.patch, SOLR-12338.patch
>
>
> Since updates with different id are independent, therefore it is safe to replay them in parallel. This will significantly reduce recovering time of replicas in high load indexing environment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]