[jira] [Commented] (SOLR-11481) Ref guide page explaining nuances of the recovery process

Previous Topic Next Topic
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[jira] [Commented] (SOLR-11481) Ref guide page explaining nuances of the recovery process

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/SOLR-11481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16202731#comment-16202731 ]

Varun Thacker commented on SOLR-11481:

  <str name="dir">${solr.ulog.dir:}</str>
   <int name="numRecordsToKeep">10000</int>

One might assume that if he sets numRecordsToKeep that PeerSync will actually try to sync upto these many records. But looking at the code we only keep 10 transaction log files by default even when when this number is set.

Yonik explains on SOLR-6359 that this is because we don't want to run out of file handles in the corner case of add 1 doc + commit , add 1 doc + commit case.

So we also need to add specify maxNumLogsToKeep and keep it to a high number as well .

Honestly my first reaction is that to accommodate users who are explicitly specifying a high numRecordsToKeep and then doing commits after every add we are choosing trappy behaviour for the remaining 99% users is a bad tradeoff. I'll file a separate Jira to see where that goes.

So for PeerSync we need to document both maxNumLogsToKeep and numRecordsToKeep and how they help improve recovery times.

> Ref guide page explaining nuances of the recovery process
> ---------------------------------------------------------
>                 Key: SOLR-11481
>                 URL: https://issues.apache.org/jira/browse/SOLR-11481
>             Project: Solr
>          Issue Type: Bug
>      Security Level: Public(Default Security Level. Issues are Public)
>            Reporter: Varun Thacker
>            Priority: Minor
> The Solr recovery process involves PeerSync , which has configuration parameters to allow the number of records it should keep.
> If this fails we do a index replication where possibly we can throttle replication
> I think it's worth explaining to users what these configuration parameters are and how does a node actually recover.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]