[jira] [Commented] (SOLR-12338) Replay buffering tlog in parallel

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[jira] [Commented] (SOLR-12338) Replay buffering tlog in parallel

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/SOLR-12338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16479083#comment-16479083 ]

David Smiley commented on SOLR-12338:

{quote}Upload a patch that makes a change from using an array of lock into a {{SetBlockingQueue}}.
BTW I've twice gotten confused in this issue conversation when you referred to things I didn't know existed before because it was unclear if I simply didn't know about it or if you were adding/introducing some new mechanism.  It would be helpful to me if you try to clarify that new things are new things, e.g. "(added in this patch)" or "added a new ..." or some-such.

It's super tempting to simply use Striped as it's difficult to write & review concurrent control structures such as this.  I have a bunch of pending commentary/review for your SetBlockingQueue but are you choosing to not use it because the numThreads * 1000 is too much internal memory/waste?

> Replay buffering tlog in parallel
> ---------------------------------
>                 Key: SOLR-12338
>                 URL: https://issues.apache.org/jira/browse/SOLR-12338
>             Project: Solr
>          Issue Type: Improvement
>      Security Level: Public(Default Security Level. Issues are Public)
>            Reporter: Cao Manh Dat
>            Assignee: Cao Manh Dat
>            Priority: Major
>         Attachments: SOLR-12338.patch, SOLR-12338.patch, SOLR-12338.patch
> Since updates with different id are independent, therefore it is safe to replay them in parallel. This will significantly reduce recovering time of replicas in high load indexing environment.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]