Any solr api to force leader on a specified node

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Any solr api to force leader on a specified node

yaswanthcse
Can someone pls help me to know if there is any solr api /config where we can make sure to always opt leader on a particular solr node in solr cloud??

Using solr 8.2 and zoo 3.4

I have four nodes and my requirement is to always make a particular node as leader

Sent from my iPhone
Reply | Threaded
Open this post in threaded view
|

Re: Any solr api to force leader on a specified node

Walter Underwood
That requirement is not necessary. Let Solr choose a leader.

Why is someone making this bad requirement?

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)

> On Oct 11, 2020, at 8:01 PM, yaswanth kumar <[hidden email]> wrote:
>
> Can someone pls help me to know if there is any solr api /config where we can make sure to always opt leader on a particular solr node in solr cloud??
>
> Using solr 8.2 and zoo 3.4
>
> I have four nodes and my requirement is to always make a particular node as leader
>
> Sent from my iPhone

Reply | Threaded
Open this post in threaded view
|

Re: Any solr api to force leader on a specified node

yaswanthcse
Hi wunder

Thanks for replying on this..

I did setup solr cloud with 4 nodes being one node having DIH configured that pulls data from ms sql every minute.. if I install DIH on rest of the nodes it’s causing connection issues on the source dB which I don’t want and manage with only one sever polling dB while rest are used as replicas for search.

So now everything works fine but when the severs are rebooted for maintenance and once they come back and if the leader is not the one that doesn’t have DIH it stops pulling data from sql , so that’s the reason why I want always to force a node as leader

Sent from my iPhone

> On Oct 11, 2020, at 11:05 PM, Walter Underwood <[hidden email]> wrote:
>
> That requirement is not necessary. Let Solr choose a leader.
>
> Why is someone making this bad requirement?
>
> wunder
> Walter Underwood
> [hidden email]
> http://observer.wunderwood.org/  (my blog)
>
>> On Oct 11, 2020, at 8:01 PM, yaswanth kumar <[hidden email]> wrote:
>>
>> Can someone pls help me to know if there is any solr api /config where we can make sure to always opt leader on a particular solr node in solr cloud??
>>
>> Using solr 8.2 and zoo 3.4
>>
>> I have four nodes and my requirement is to always make a particular node as leader
>>
>> Sent from my iPhone
>
Reply | Threaded
Open this post in threaded view
|

Re: Any solr api to force leader on a specified node

Walter Underwood
Don’t use DIH. DIH has a lot of limitations and problems, as you are discovering.

Write a simple program that fetches from the database and sends documents
in batches to Solr. I did this before DIH was invented (Solr 1.3) and I’m doing it
now.

You can send the updates to the load balancer for the Solr Cloud cluster. The
updates will be automatically routed to the right leader. It is very fast.

My loader is written in Python and I don’t even bother with a special Solr library.
It just sends JSON to the update handler with the right options.

We do this for all of our clusters. Our biggest one is 48 hosts with 55 million
documents.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)

> On Oct 11, 2020, at 8:40 PM, yaswanth kumar <[hidden email]> wrote:
>
> Hi wunder
>
> Thanks for replying on this..
>
> I did setup solr cloud with 4 nodes being one node having DIH configured that pulls data from ms sql every minute.. if I install DIH on rest of the nodes it’s causing connection issues on the source dB which I don’t want and manage with only one sever polling dB while rest are used as replicas for search.
>
> So now everything works fine but when the severs are rebooted for maintenance and once they come back and if the leader is not the one that doesn’t have DIH it stops pulling data from sql , so that’s the reason why I want always to force a node as leader
>
> Sent from my iPhone
>
>> On Oct 11, 2020, at 11:05 PM, Walter Underwood <[hidden email]> wrote:
>>
>> That requirement is not necessary. Let Solr choose a leader.
>>
>> Why is someone making this bad requirement?
>>
>> wunder
>> Walter Underwood
>> [hidden email]
>> http://observer.wunderwood.org/  (my blog)
>>
>>> On Oct 11, 2020, at 8:01 PM, yaswanth kumar <[hidden email]> wrote:
>>>
>>> Can someone pls help me to know if there is any solr api /config where we can make sure to always opt leader on a particular solr node in solr cloud??
>>>
>>> Using solr 8.2 and zoo 3.4
>>>
>>> I have four nodes and my requirement is to always make a particular node as leader
>>>
>>> Sent from my iPhone
>>

Reply | Threaded
Open this post in threaded view
|

Re: Any solr api to force leader on a specified node

Erick Erickson
First, I totally agree with Walter. See:
https://lucidworks.com/post/indexing-with-solrj/

Second, DIH is being deprecated. It is being moved to
a package that will be supported if, and only if, there is
enough community support for it. “Community support”
means people who use it need to step up and maintain
it.

Third, there’s nothing in Solr that requires DIH to
be run on a leader so your premise is wrong. You need
to look at your logs to see what’s happening there. It
should be perfectly fine to run it on a replica.

Best,
Erick

> On Oct 11, 2020, at 11:53 PM, Walter Underwood <[hidden email]> wrote:
>
> Don’t use DIH. DIH has a lot of limitations and problems, as you are discovering.
>
> Write a simple program that fetches from the database and sends documents
> in batches to Solr. I did this before DIH was invented (Solr 1.3) and I’m doing it
> now.
>
> You can send the updates to the load balancer for the Solr Cloud cluster. The
> updates will be automatically routed to the right leader. It is very fast.
>
> My loader is written in Python and I don’t even bother with a special Solr library.
> It just sends JSON to the update handler with the right options.
>
> We do this for all of our clusters. Our biggest one is 48 hosts with 55 million
> documents.
>
> wunder
> Walter Underwood
> [hidden email]
> http://observer.wunderwood.org/  (my blog)
>
>> On Oct 11, 2020, at 8:40 PM, yaswanth kumar <[hidden email]> wrote:
>>
>> Hi wunder
>>
>> Thanks for replying on this..
>>
>> I did setup solr cloud with 4 nodes being one node having DIH configured that pulls data from ms sql every minute.. if I install DIH on rest of the nodes it’s causing connection issues on the source dB which I don’t want and manage with only one sever polling dB while rest are used as replicas for search.
>>
>> So now everything works fine but when the severs are rebooted for maintenance and once they come back and if the leader is not the one that doesn’t have DIH it stops pulling data from sql , so that’s the reason why I want always to force a node as leader
>>
>> Sent from my iPhone
>>
>>> On Oct 11, 2020, at 11:05 PM, Walter Underwood <[hidden email]> wrote:
>>>
>>> That requirement is not necessary. Let Solr choose a leader.
>>>
>>> Why is someone making this bad requirement?
>>>
>>> wunder
>>> Walter Underwood
>>> [hidden email]
>>> http://observer.wunderwood.org/  (my blog)
>>>
>>>> On Oct 11, 2020, at 8:01 PM, yaswanth kumar <[hidden email]> wrote:
>>>>
>>>> Can someone pls help me to know if there is any solr api /config where we can make sure to always opt leader on a particular solr node in solr cloud??
>>>>
>>>> Using solr 8.2 and zoo 3.4
>>>>
>>>> I have four nodes and my requirement is to always make a particular node as leader
>>>>
>>>> Sent from my iPhone
>>>
>