Solrcloud replication not working

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Solrcloud replication not working

solr2020
Hi,

We are using Solr 6.4.2 & SolrCloud setup. We have two solr instances in the
solr cluster.This solrcloud running in ubuntu OS. The problem is replication
is not happening between these two solr instances. sometimes it replicate
10% of the content and sometimes not.

In Zookeeper ensemble we have three zookeeper instances running in a
different box.

thanks.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Solrcloud replication not working

Erick Erickson
We need a lot more data to say anything useful, please read:

https://wiki.apache.org/solr/UsingMailingLists

What do you see in your Solr logs? What have you tried to do to
diagnose this? Do you have enough disk space?

Best,
Erick

On Thu, Oct 5, 2017 at 6:56 AM, solr2020 <[hidden email]> wrote:

> Hi,
>
> We are using Solr 6.4.2 & SolrCloud setup. We have two solr instances in the
> solr cluster.This solrcloud running in ubuntu OS. The problem is replication
> is not happening between these two solr instances. sometimes it replicate
> 10% of the content and sometimes not.
>
> In Zookeeper ensemble we have three zookeeper instances running in a
> different box.
>
> thanks.
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Solrcloud replication not working

solr2020
thanks.

We dont see any error message/any message in logs. And we have enough disk
space.

We are running solr as root user in ubuntu box but zookeeper process running
as zookeeper user.Will that cause the problem?



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Solrcloud replication not working

solr2020
In reply to this post by Erick Erickson
i could see different version of the below entries in Leader and replica.
While doing index , in replica instance logs we could see it is keep
receiving update request from leader but it says no changes, skipping
commit.

Master (Searching)
Master (Replicable)

There is no other error messages related to replication.Any idea why this is
happening?
Is there any API to run replication manually.
-



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Solrcloud replication not working

solr2020
The problem was replicationFactor was set to 1.Now replication works fine
while setting replicationFactor as 2.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Solrcloud replication not working

Shawn Heisey
In reply to this post by solr2020
On 10/10/2017 2:51 AM, solr2020 wrote:

> i could see different version of the below entries in Leader and replica.
> While doing index , in replica instance logs we could see it is keep
> receiving update request from leader but it says no changes, skipping
> commit.
>
> Master (Searching)
> Master (Replicable)
>
> There is no other error messages related to replication.Any idea why this is
> happening?
> Is there any API to run replication manually.

The replication feature (which is what exposes the version numbers you
have referenced) is *not* a part of normal SolrCloud operation. 
Replication is only used for recovery operations -- when SolrCloud
determines that a replica has been out of touch with the rest of the
cloud for enough updates that it must completely overwrite the index
with a verbatim copy from the leader.  When that kind of recovery is
required, Solr will temporarily designate one index as a master, another
index as a slave, and utilize the replication feature to copy the index
from one to the other.

For SolrCloud, you cannot make any kind of judgement based on the
replication index version numbers.  It is normal for those numbers to
vary between replicas.

During normal operation, SolrCloud keeps indexes in sync by performing
the same indexing operations on all replicas and keeping track of those
updates in the transaction log.

Regarding your most recent update on the thread, the replicationFactor
value normally has absolutely no bearing on normal SolrCloud operation. 
Unless your indexes are stored in HDFS with the HDFSDIrectoryFactory,
the only time Solr ever does anything with replicationFactor is when the
collection is initially created.

Running Solr as root is not recommended for security reasons, but isn't
going to cause this problem.

If there are no error messages in your logs, then I would suspect
problems with the network or with the operating system that are keeping
your Solr servers from communicating with each other properly.  Is the
Solr log on the server that is the shard leader also error-free?

Thanks,
Shawn