Possible regression for Solr 4.6.0 - commitWithin does not work with replicas

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Possible regression for Solr 4.6.0 - commitWithin does not work with replicas

Elodie Sannier
Hello,

I am using SolrCloud 4.6.0 with two shards, two replicas by shard and with two collections.

collection fr_blue:
- shard1 -> server-01 (replica1), server-01 (replica2)
- shard2 -> server-02 (replica1), server-02 (replica2)

collection fr_green:
- shard1 -> server-01 (replica1), server-01 (replica2)
- shard2 -> server-02 (replica1), server-02 (replica2)

I add documents using solrj CloudSolrServer and using commitWithin feature :
int commitWithinMs = 30000;
SolrServer server = new CloudSolrServer(zkHost);
server.add(doc, commitWithinMs);

When I query an instance,  for 5 indexed documents, the numFound value changes for each call, randomly 0,1,4 or 5.
When I query the instances with distrib=false, I have:
- leader shard1: numFound=1
- leader shard2: numFound=4
- replica shard1: numFound=0
- replica shard1: numFound=0

The documents are not commited in the replicas, even after waiting more than 30 seconds.

If I force a commit usinghttp://server-01:8080/solr/update/?commit=true, the documents are commited in the replicas and numFound=5.
I suppose that the leader forwards the documents to the replica, but they are not commited.

Is it a new bug with commitWithin feature for distributed mode ?

This problem does not occur with the version 4.5.1.

Elodie Sannier


Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 8, rue du Sentier 75002 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce message, merci de le détruire et d'en avertir l'expéditeur.
Reply | Threaded
Open this post in threaded view
|

Re: Possible regression for Solr 4.6.0 - commitWithin does not work with replicas

Varun Thacker
Hi Elodie,

Thanks for pointing it out. I have created a Jira for this (
https://issues.apache.org/jira/browse/SOLR-5658 )

You could track the progress of it there.



On Wed, Dec 11, 2013 at 3:11 PM, Elodie Sannier <[hidden email]>wrote:

> Hello,
>
> I am using SolrCloud 4.6.0 with two shards, two replicas by shard and with
> two collections.
>
> collection fr_blue:
> - shard1 -> server-01 (replica1), server-01 (replica2)
> - shard2 -> server-02 (replica1), server-02 (replica2)
>
> collection fr_green:
> - shard1 -> server-01 (replica1), server-01 (replica2)
> - shard2 -> server-02 (replica1), server-02 (replica2)
>
> I add documents using solrj CloudSolrServer and using commitWithin feature
> :
> int commitWithinMs = 30000;
> SolrServer server = new CloudSolrServer(zkHost);
> server.add(doc, commitWithinMs);
>
> When I query an instance,  for 5 indexed documents, the numFound value
> changes for each call, randomly 0,1,4 or 5.
> When I query the instances with distrib=false, I have:
> - leader shard1: numFound=1
> - leader shard2: numFound=4
> - replica shard1: numFound=0
> - replica shard1: numFound=0
>
> The documents are not commited in the replicas, even after waiting more
> than 30 seconds.
>
> If I force a commit usinghttp://server-01:8080/solr/update/?commit=true,
> the documents are commited in the replicas and numFound=5.
> I suppose that the leader forwards the documents to the replica, but they
> are not commited.
>
> Is it a new bug with commitWithin feature for distributed mode ?
>
> This problem does not occur with the version 4.5.1.
>
> Elodie Sannier
>
>
> Kelkoo SAS
> Société par Actions Simplifiée
> Au capital de € 4.168.964,30
> Siège social : 8, rue du Sentier 75002 Paris
> 425 093 069 RCS Paris
>
> Ce message et les pièces jointes sont confidentiels et établis à
> l'attention exclusive de leurs destinataires. Si vous n'êtes pas le
> destinataire de ce message, merci de le détruire et d'en avertir
> l'expéditeur.
>



--


Regards,
Varun Thacker
http://www.vthacker.in/
Reply | Threaded
Open this post in threaded view
|

Re: Possible regression for Solr 4.6.0 - commitWithin does not work with replicas

Shawn Heisey-4
In reply to this post by Elodie Sannier
On 12/11/2013 2:41 AM, Elodie Sannier wrote:
> collection fr_blue:
> - shard1 -> server-01 (replica1), server-01 (replica2)
> - shard2 -> server-02 (replica1), server-02 (replica2)
>
> collection fr_green:
> - shard1 -> server-01 (replica1), server-01 (replica2)
> - shard2 -> server-02 (replica1), server-02 (replica2)

I'm pretty sure this won't affect the issue you've mentioned, but it's
worth pointing out.

If this is really how you've arranged your shard replicas, your system
cannot survive a failure, because you've got both replicas for each
shard on the same server.  If that server dies, half of each collection
will be gone.

Thanks,
Shawn

Reply | Threaded
Open this post in threaded view
|

Re: Possible regression for Solr 4.6.0 - commitWithin does not work with replicas

Elodie Sannier
I have this configuration for test servers (the order of the instance
start leads to this conf.) not for production.

Elodie

On 01/23/2014 04:35 PM, Shawn Heisey wrote:

> On 12/11/2013 2:41 AM, Elodie Sannier wrote:
>> collection fr_blue:
>> - shard1 -> server-01 (replica1), server-01 (replica2)
>> - shard2 -> server-02 (replica1), server-02 (replica2)
>>
>> collection fr_green:
>> - shard1 -> server-01 (replica1), server-01 (replica2)
>> - shard2 -> server-02 (replica1), server-02 (replica2)
> I'm pretty sure this won't affect the issue you've mentioned, but it's
> worth pointing out.
>
> If this is really how you've arranged your shard replicas, your system
> cannot survive a failure, because you've got both replicas for each
> shard on the same server.  If that server dies, half of each collection
> will be gone.
>
> Thanks,
> Shawn
>


Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 8, rue du Sentier 75002 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce message, merci de le détruire et d'en avertir l'expéditeur.