CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

Zimmermann, Thomas
Question about CloudSolrClient and CLUSTERSTATUS. We just deployed a 3 server ZK cluster and a 5 node solr cluster using the CloudSolrClient in Solr 7.4.

We're seeing a TON of traffic going to one server with just cluster status commands. Every single query seems to be hitting this box for status, but the rest of the query load is divided evenly amongst the servers. Is this an expected interaction in this client?

For example - 75k request per minute going to this one box, and 3.5k RPM to all other nodes in the cloud.

All of those extra requests on the one box are "/solr/admin/collections?collection=collectionName&action=CLUSTERSTATUS&wt=javabin&version=2"

Our plan right now is to roll back to the basic HTTP client and pass all traffic through our load balancer, but would like to understand if this is an expected interaction for the Cloud Client, a misconfiguration on our end, or a bug
Reply | Threaded
Open this post in threaded view
|

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

Erick Erickson
Is the box you're seeing this on the Overseer? Or is it in any other
way "special", like has all the leaders? And I'm assuming all these
are NRT replicas, not TLOG or PULL.

What are you doing when these occur? Queries? Updates? If you're doing
updates, are these coincident with each request? Each commit (which
you shouldn't be doing from the client anyway)? If they're coincident
with updating, are you updating in batches or a single doc at a time?

I can imagine each update or commit gets the status, although even
that seems questionable.

If you can pin down a bit what actions trigger the request that'd help a lot.

Best,
Erick



On Tue, Nov 6, 2018 at 8:06 AM Zimmermann, Thomas
<[hidden email]> wrote:

>
> Question about CloudSolrClient and CLUSTERSTATUS. We just deployed a 3 server ZK cluster and a 5 node solr cluster using the CloudSolrClient in Solr 7.4.
>
> We're seeing a TON of traffic going to one server with just cluster status commands. Every single query seems to be hitting this box for status, but the rest of the query load is divided evenly amongst the servers. Is this an expected interaction in this client?
>
> For example - 75k request per minute going to this one box, and 3.5k RPM to all other nodes in the cloud.
>
> All of those extra requests on the one box are "/solr/admin/collections?collection=collectionName&action=CLUSTERSTATUS&wt=javabin&version=2"
>
> Our plan right now is to roll back to the basic HTTP client and pass all traffic through our load balancer, but would like to understand if this is an expected interaction for the Cloud Client, a misconfiguration on our end, or a bug
Reply | Threaded
Open this post in threaded view
|

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

Jason Gerlowski
My understanding was that we always tried to use the cached version of
this information until either (a) Solr responds in a way that
indicates our cache is out of date, or (b) the TTL on the cache entry
expires.  Though there might very well be a code path that behaves
differently as Erick suggests above.

A few more questions that might shed light on this for you (or for us):
1. How are you creating your CloudSolrClient?  Can you share the
2. Did you modify the TTL on your cache via CloudSolrClient's
"setCollectionCacheTTl" method?
3. Are all of the CLUSTERSTATUS requests you're seeing for the same
collection, or different collections?  How many collections do you
have on your cluster?

Best,

Jason

On Tue, Nov 6, 2018 at 11:25 AM Erick Erickson <[hidden email]> wrote:

>
> Is the box you're seeing this on the Overseer? Or is it in any other
> way "special", like has all the leaders? And I'm assuming all these
> are NRT replicas, not TLOG or PULL.
>
> What are you doing when these occur? Queries? Updates? If you're doing
> updates, are these coincident with each request? Each commit (which
> you shouldn't be doing from the client anyway)? If they're coincident
> with updating, are you updating in batches or a single doc at a time?
>
> I can imagine each update or commit gets the status, although even
> that seems questionable.
>
> If you can pin down a bit what actions trigger the request that'd help a lot.
>
> Best,
> Erick
>
>
>
> On Tue, Nov 6, 2018 at 8:06 AM Zimmermann, Thomas
> <[hidden email]> wrote:
> >
> > Question about CloudSolrClient and CLUSTERSTATUS. We just deployed a 3 server ZK cluster and a 5 node solr cluster using the CloudSolrClient in Solr 7.4.
> >
> > We're seeing a TON of traffic going to one server with just cluster status commands. Every single query seems to be hitting this box for status, but the rest of the query load is divided evenly amongst the servers. Is this an expected interaction in this client?
> >
> > For example - 75k request per minute going to this one box, and 3.5k RPM to all other nodes in the cloud.
> >
> > All of those extra requests on the one box are "/solr/admin/collections?collection=collectionName&action=CLUSTERSTATUS&wt=javabin&version=2"
> >
> > Our plan right now is to roll back to the basic HTTP client and pass all traffic through our load balancer, but would like to understand if this is an expected interaction for the Cloud Client, a misconfiguration on our end, or a bug
Reply | Threaded
Open this post in threaded view
|

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

Shawn Heisey-2
In reply to this post by Zimmermann, Thomas
On 11/6/2018 9:06 AM, Zimmermann, Thomas wrote:
> For example - 75k request per minute going to this one box, and 3.5k RPM to all other nodes in the cloud.
>
> All of those extra requests on the one box are "/solr/admin/collections?collection=collectionName&action=CLUSTERSTATUS&wt=javabin&version=2"

That sounds like either a bug or some kind of problem in your setup. 
Over a thousand requests per second will overwhelm a single Solr node,
even if the info can be satisfied entirely from memory and doesn't
require complex calculations or large-scale data retrieval like a
regular query does.

If you manually execute that request, do you get a response, and does it
return quickly or take a significant amount of time?  If the request
itself has problems, maybe CloudSolrClient is repeating it frequently
because it's not getting the info it's after.  Can you share the full
log entry from solr.log for one of those requests?

I try to keep an eye on things with CloudSolrClient, but I have very
limited experience with it.  I cannot imagine that the behavior you're
seeing is normal.  It sounds very wrong to me.

Since I do not know all that much about how CloudSolrClient's background
threads work, I cannot say for sure whether it's a bug or a problem with
your setup.  Can you try upgrading the Solr jars in your client app to
7.5.0 and see if that makes any difference?  What version of Solr are
you running on the server side?

> Our plan right now is to roll back to the basic HTTP client and pass all traffic through our load balancer, but would like to understand if this is an expected interaction for the Cloud Client, a misconfiguration on our end, or a bug

At least you have that as an option!  Some people might not be able to
do that.

Thanks,
Shawn

Reply | Threaded
Open this post in threaded view
|

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

Zimmermann, Thomas
Erik -

This box did have all the leaders for the dozen or so collections we have
when the cloud spun up. We were able to force the leaders for other cores
onto other nodes using the apis, but did not see this traffic load migrate
to the new hosts when leadership changed. All nodes are NRT. The requests
are 99% queries to load content on the web front ends, a few intermittent
updates with comments, new content creation, etc.

Jason -

1. We are instantiating the cloud client with our VIP Load Balancer url.
We ran into a memory leak issue when passing in ZK server addresses that
forced this path.
2. No we did not tweak any cache TTLs
3. This codebase interacts with three collections in our cloud, and we are
seeing CLUSTERSTATUS checks for all 3.

Shawn -

Server performance is fine and request time are great. We are tolerating
the level of traffic, but the server that is taking all the hits is
obviously performing a bit slower than the others. Response times are
under 5MS avg for queries on all servers, which is within our perf
thresholds.

We are running 7.4 on the client and server side, moving to 7.5 was
troublesome for us so we are holding off for the time being.

Thanks,
TZ



On 11/6/18, 11:39 AM, "Shawn Heisey" <[hidden email]> wrote:

>On 11/6/2018 9:06 AM, Zimmermann, Thomas wrote:
>> For example - 75k request per minute going to this one box, and 3.5k
>>RPM to all other nodes in the cloud.
>>
>> All of those extra requests on the one box are
>>"/solr/admin/collections?collection=collectionName&action=CLUSTERSTATUS&w
>>t=javabin&version=2"
>
>That sounds like either a bug or some kind of problem in your setup.
>Over a thousand requests per second will overwhelm a single Solr node,
>even if the info can be satisfied entirely from memory and doesn't
>require complex calculations or large-scale data retrieval like a
>regular query does.
>
>If you manually execute that request, do you get a response, and does it
>return quickly or take a significant amount of time?  If the request
>itself has problems, maybe CloudSolrClient is repeating it frequently
>because it's not getting the info it's after.  Can you share the full
>log entry from solr.log for one of those requests?
>
>I try to keep an eye on things with CloudSolrClient, but I have very
>limited experience with it.  I cannot imagine that the behavior you're
>seeing is normal.  It sounds very wrong to me.
>
>Since I do not know all that much about how CloudSolrClient's background
>threads work, I cannot say for sure whether it's a bug or a problem with
>your setup.  Can you try upgrading the Solr jars in your client app to
>7.5.0 and see if that makes any difference?  What version of Solr are
>you running on the server side?
>
>> Our plan right now is to roll back to the basic HTTP client and pass
>>all traffic through our load balancer, but would like to understand if
>>this is an expected interaction for the Cloud Client, a misconfiguration
>>on our end, or a bug
>
>At least you have that as an option!  Some people might not be able to
>do that.
>
>Thanks,
>Shawn
>

Reply | Threaded
Open this post in threaded view
|

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

Zimmermann, Thomas
In reply to this post by Shawn Heisey-2
I should mention I¹m also hanging out in the Solr IRC Channel today under
the nick ³apatheticnow² if anyone wants to follow up in real time during
business hours EST.

On 11/6/18, 11:39 AM, "Shawn Heisey" <[hidden email]> wrote:

>On 11/6/2018 9:06 AM, Zimmermann, Thomas wrote:
>> For example - 75k request per minute going to this one box, and 3.5k
>>RPM to all other nodes in the cloud.
>>
>> All of those extra requests on the one box are
>>"/solr/admin/collections?collection=collectionName&action=CLUSTERSTATUS&w
>>t=javabin&version=2"
>
>That sounds like either a bug or some kind of problem in your setup.
>Over a thousand requests per second will overwhelm a single Solr node,
>even if the info can be satisfied entirely from memory and doesn't
>require complex calculations or large-scale data retrieval like a
>regular query does.
>
>If you manually execute that request, do you get a response, and does it
>return quickly or take a significant amount of time?  If the request
>itself has problems, maybe CloudSolrClient is repeating it frequently
>because it's not getting the info it's after.  Can you share the full
>log entry from solr.log for one of those requests?
>
>I try to keep an eye on things with CloudSolrClient, but I have very
>limited experience with it.  I cannot imagine that the behavior you're
>seeing is normal.  It sounds very wrong to me.
>
>Since I do not know all that much about how CloudSolrClient's background
>threads work, I cannot say for sure whether it's a bug or a problem with
>your setup.  Can you try upgrading the Solr jars in your client app to
>7.5.0 and see if that makes any difference?  What version of Solr are
>you running on the server side?
>
>> Our plan right now is to roll back to the basic HTTP client and pass
>>all traffic through our load balancer, but would like to understand if
>>this is an expected interaction for the Cloud Client, a misconfiguration
>>on our end, or a bug
>
>At least you have that as an option!  Some people might not be able to
>do that.
>
>Thanks,
>Shawn
>

Reply | Threaded
Open this post in threaded view
|

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

Shawn Heisey-2
In reply to this post by Zimmermann, Thomas
On 11/6/2018 10:12 AM, Zimmermann, Thomas wrote:
> Shawn -
>
> Server performance is fine and request time are great. We are tolerating
> the level of traffic, but the server that is taking all the hits is
> obviously performing a bit slower than the others. Response times are
> under 5MS avg for queries on all servers, which is within our perf
> thresholds.

I was asking specifically about the clusterstatus requests -- whether
the response looks complete if you manually execute the same request and
whether it returns quickly.  And I'd like to see the solr.log where
these are happening.

Knowing that requests in general are performing well is good info,
although I have no idea how that is possible on the node that is getting
over a thousand clusterstatus requests per second.  I would expect that
node to be essentially dead under that much load.  Since it's apparently
handling it fine ... that's really impressive.

> We are running 7.4 on the client and server side, moving to 7.5 was
> troublesome for us so we are holding off for the time being.

I was hoping you could just upgrade the SolrJ client, which would
involve either replacing the solrj jar or bumping the version number in
the config for a dependency manager (things like ivy, maven, gradle,
etc).  A 7.5 client should be pretty safe against 7.4 servers.  The
client would be newer than the server and very close to the same
version, which is the general recommendation for CloudSolrClient when
the two versions cannot be identical for some reason.

Are you absolutely sure that those requests are coming from the program
with CloudSolrClient?  To find out, you'll need to enable the request
log in jetty.xml (it just needs to be un-commented) and restart the
server.  The source address is not logged in solr.log.  It's very
important to be absolutely sure where the requests are coming from.  If
you're running the client code on the same machine as one of your Solr
servers, it will be difficult to be sure about the source, so I would
definitely suggest running the client code on a completely different
machine, so the source addresses in the request log are useful.

Thanks,
Shawn

Reply | Threaded
Open this post in threaded view
|

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

Tomáš Hampl
This error comes every request,
in solr client or if i call url in chrome browser or curl from console.
I have no replicas actually for this test but it is NRT type.
There is no writes or another reads on this server (solr cloud) completely
isolated.  (version 7.5 single docker container)
I have 6 collection on server.

--
Tomáš Hampl
Mob.: +420774850702
E-Mail: [hidden email]


út 6. 11. 2018 v 18:35 odesílatel Shawn Heisey <[hidden email]> napsal:

> On 11/6/2018 10:12 AM, Zimmermann, Thomas wrote:
> > Shawn -
> >
> > Server performance is fine and request time are great. We are tolerating
> > the level of traffic, but the server that is taking all the hits is
> > obviously performing a bit slower than the others. Response times are
> > under 5MS avg for queries on all servers, which is within our perf
> > thresholds.
>
> I was asking specifically about the clusterstatus requests -- whether
> the response looks complete if you manually execute the same request and
> whether it returns quickly.  And I'd like to see the solr.log where
> these are happening.
>
> Knowing that requests in general are performing well is good info,
> although I have no idea how that is possible on the node that is getting
> over a thousand clusterstatus requests per second.  I would expect that
> node to be essentially dead under that much load.  Since it's apparently
> handling it fine ... that's really impressive.
>
> > We are running 7.4 on the client and server side, moving to 7.5 was
> > troublesome for us so we are holding off for the time being.
>
> I was hoping you could just upgrade the SolrJ client, which would
> involve either replacing the solrj jar or bumping the version number in
> the config for a dependency manager (things like ivy, maven, gradle,
> etc).  A 7.5 client should be pretty safe against 7.4 servers.  The
> client would be newer than the server and very close to the same
> version, which is the general recommendation for CloudSolrClient when
> the two versions cannot be identical for some reason.
>
> Are you absolutely sure that those requests are coming from the program
> with CloudSolrClient?  To find out, you'll need to enable the request
> log in jetty.xml (it just needs to be un-commented) and restart the
> server.  The source address is not logged in solr.log.  It's very
> important to be absolutely sure where the requests are coming from.  If
> you're running the client code on the same machine as one of your Solr
> servers, it will be difficult to be sure about the source, so I would
> definitely suggest running the client code on a completely different
> machine, so the source addresses in the request log are useful.
>
> Thanks,
> Shawn
>
>
Reply | Threaded
Open this post in threaded view
|

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

Zimmermann, Thomas
In reply to this post by Shawn Heisey-2
Hi Shawn,

We¹re equally impressed by how well the server is handling it. We¹re using
Sematext for monitoring and the load on the box has been steady under 1
and not entering a swap state memory wise.

We are 100% certain the traffic is coming from the 3 web hosts running
this code. We have put some custom logging in place that logs all requests
to an access style log and stores that data in kibana/logstash. In
logstash we are able to confirm that all these requests (~40million in the
last 12 hours) are coming from our web front ends directly to a single box
in the cluster.

Our client codes is on separate servers from our solr servers and zk has
it¹s own boxes as well.

Here¹s a scrubbed pastbin of our cluster status response from that machine
that is getting all the traffic, I pulled this via browser on my local
machine.
https://pastebin.com/42haKVME

We can attempt to update the SolrJ dependency on our lower env and see if
that fixes the problem if you think that a good course of action, but we
are also in the midst of switching over to HTTP Client to resolve the
production issues we are seeing ASAP, so I can¹t promise a timeline. If
you think there¹s a chance that will fix this, we could of course give it
a quick go.


-TZ



On 11/6/18, 12:35 PM, "Shawn Heisey" <[hidden email]> wrote:

>On 11/6/2018 10:12 AM, Zimmermann, Thomas wrote:
>> Shawn -
>>
>> Server performance is fine and request time are great. We are tolerating
>> the level of traffic, but the server that is taking all the hits is
>> obviously performing a bit slower than the others. Response times are
>> under 5MS avg for queries on all servers, which is within our perf
>> thresholds.
>
>I was asking specifically about the clusterstatus requests -- whether
>the response looks complete if you manually execute the same request and
>whether it returns quickly.  And I'd like to see the solr.log where
>these are happening.
>
>Knowing that requests in general are performing well is good info,
>although I have no idea how that is possible on the node that is getting
>over a thousand clusterstatus requests per second.  I would expect that
>node to be essentially dead under that much load.  Since it's apparently
>handling it fine ... that's really impressive.
>
>> We are running 7.4 on the client and server side, moving to 7.5 was
>> troublesome for us so we are holding off for the time being.
>
>I was hoping you could just upgrade the SolrJ client, which would
>involve either replacing the solrj jar or bumping the version number in
>the config for a dependency manager (things like ivy, maven, gradle,
>etc).  A 7.5 client should be pretty safe against 7.4 servers.  The
>client would be newer than the server and very close to the same
>version, which is the general recommendation for CloudSolrClient when
>the two versions cannot be identical for some reason.
>
>Are you absolutely sure that those requests are coming from the program
>with CloudSolrClient?  To find out, you'll need to enable the request
>log in jetty.xml (it just needs to be un-commented) and restart the
>server.  The source address is not logged in solr.log.  It's very
>important to be absolutely sure where the requests are coming from.  If
>you're running the client code on the same machine as one of your Solr
>servers, it will be difficult to be sure about the source, so I would
>definitely suggest running the client code on a completely different
>machine, so the source addresses in the request log are useful.
>
>Thanks,
>Shawn
>

Reply | Threaded
Open this post in threaded view
|

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

Gus Heck
Tomáš,

One thing that causes a clusterstatus call is alias resolution if the
HttpClusterStateProvider is in use instead of the ZkClusterStateProvider.
I've just been fixing spurious error messages generated by this
in SOLR-12938.

-Gus

On Tue, Nov 6, 2018 at 1:08 PM Zimmermann, Thomas <
[hidden email]> wrote:

> Hi Shawn,
>
> We¹re equally impressed by how well the server is handling it. We¹re using
> Sematext for monitoring and the load on the box has been steady under 1
> and not entering a swap state memory wise.
>
> We are 100% certain the traffic is coming from the 3 web hosts running
> this code. We have put some custom logging in place that logs all requests
> to an access style log and stores that data in kibana/logstash. In
> logstash we are able to confirm that all these requests (~40million in the
> last 12 hours) are coming from our web front ends directly to a single box
> in the cluster.
>
> Our client codes is on separate servers from our solr servers and zk has
> it¹s own boxes as well.
>
> Here¹s a scrubbed pastbin of our cluster status response from that machine
> that is getting all the traffic, I pulled this via browser on my local
> machine.
> https://pastebin.com/42haKVME
>
> We can attempt to update the SolrJ dependency on our lower env and see if
> that fixes the problem if you think that a good course of action, but we
> are also in the midst of switching over to HTTP Client to resolve the
> production issues we are seeing ASAP, so I can¹t promise a timeline. If
> you think there¹s a chance that will fix this, we could of course give it
> a quick go.
>
>
> -TZ
>
>
>
> On 11/6/18, 12:35 PM, "Shawn Heisey" <[hidden email]> wrote:
>
> >On 11/6/2018 10:12 AM, Zimmermann, Thomas wrote:
> >> Shawn -
> >>
> >> Server performance is fine and request time are great. We are tolerating
> >> the level of traffic, but the server that is taking all the hits is
> >> obviously performing a bit slower than the others. Response times are
> >> under 5MS avg for queries on all servers, which is within our perf
> >> thresholds.
> >
> >I was asking specifically about the clusterstatus requests -- whether
> >the response looks complete if you manually execute the same request and
> >whether it returns quickly.  And I'd like to see the solr.log where
> >these are happening.
> >
> >Knowing that requests in general are performing well is good info,
> >although I have no idea how that is possible on the node that is getting
> >over a thousand clusterstatus requests per second.  I would expect that
> >node to be essentially dead under that much load.  Since it's apparently
> >handling it fine ... that's really impressive.
> >
> >> We are running 7.4 on the client and server side, moving to 7.5 was
> >> troublesome for us so we are holding off for the time being.
> >
> >I was hoping you could just upgrade the SolrJ client, which would
> >involve either replacing the solrj jar or bumping the version number in
> >the config for a dependency manager (things like ivy, maven, gradle,
> >etc).  A 7.5 client should be pretty safe against 7.4 servers.  The
> >client would be newer than the server and very close to the same
> >version, which is the general recommendation for CloudSolrClient when
> >the two versions cannot be identical for some reason.
> >
> >Are you absolutely sure that those requests are coming from the program
> >with CloudSolrClient?  To find out, you'll need to enable the request
> >log in jetty.xml (it just needs to be un-commented) and restart the
> >server.  The source address is not logged in solr.log.  It's very
> >important to be absolutely sure where the requests are coming from.  If
> >you're running the client code on the same machine as one of your Solr
> >servers, it will be difficult to be sure about the source, so I would
> >definitely suggest running the client code on a completely different
> >machine, so the source addresses in the request log are useful.
> >
> >Thanks,
> >Shawn
> >
>
>

--
http://www.the111shift.com
Reply | Threaded
Open this post in threaded view
|

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

Shawn Heisey-2
On 11/6/2018 10:06 PM, Gus Heck wrote:
> One thing that causes a clusterstatus call is alias resolution if the
> HttpClusterStateProvider is in use instead of the ZkClusterStateProvider.
> I've just been fixing spurious error messages generated by this
> in SOLR-12938.

Gus,

If CloudSolrClient is created using URLs instead of a zkHost, does it do
EVERYTHING with http instead of ZK?  Because if it does, it might
actually make sense for it to initiate lot of http requests.  The ZK
client is capable of nearly instantaneous notification of cluster
changes ... duplicating that with HTTP would require constant checking.

What I would have hoped for when constructing CloudSolrClient with URLs
was that it would ask the server(s) for the zkHost setting, and then
proceed to use ZK.  But I suppose if you're using URLs because ZK isn't
reachable, that would be problematic.

Thomas,

Are you initializing CloudSolrClient with your ZK server info, or using
one or more Solr URLs?

Thanks,
Shawn

Reply | Threaded
Open this post in threaded view
|

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

Jason Gerlowski
Hey Shawn,

A few answers, where I can give them.

1. It's easy to miss in the thread, but the user mentioned that
they're creating their CloudSolrClient via solr URLs.
2. When you create a CloudSolrClient with a Solr URL, it's not just
used to fetch the ZK connString so that it can use ZK from then on.
It continues to use the Solr URL for fetching cluster state.  If
you're curious, the codepaths are "ZkClientClusterStateProvider", and
"HttpClusterStateProvider", respectively.

I don't know much about the ClusterStateProvider implementations to
say anything more helpful about the original problem, but figured I'd
chime in where I could.

Best,

Jason
On Wed, Nov 7, 2018 at 12:42 PM Shawn Heisey <[hidden email]> wrote:

>
> On 11/6/2018 10:06 PM, Gus Heck wrote:
> > One thing that causes a clusterstatus call is alias resolution if the
> > HttpClusterStateProvider is in use instead of the ZkClusterStateProvider.
> > I've just been fixing spurious error messages generated by this
> > in SOLR-12938.
>
> Gus,
>
> If CloudSolrClient is created using URLs instead of a zkHost, does it do
> EVERYTHING with http instead of ZK?  Because if it does, it might
> actually make sense for it to initiate lot of http requests.  The ZK
> client is capable of nearly instantaneous notification of cluster
> changes ... duplicating that with HTTP would require constant checking.
>
> What I would have hoped for when constructing CloudSolrClient with URLs
> was that it would ask the server(s) for the zkHost setting, and then
> proceed to use ZK.  But I suppose if you're using URLs because ZK isn't
> reachable, that would be problematic.
>
> Thomas,
>
> Are you initializing CloudSolrClient with your ZK server info, or using
> one or more Solr URLs?
>
> Thanks,
> Shawn
>