JMX metrics for solr cloud cluster state

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

JMX metrics for solr cloud cluster state

GaneshSe
Hi

We use New Relic to monitor Sold Cloud Cluster 7.2.1. we would like to get
alerted on any cluster state change. Like for example degraded shard.
Replica down. New relic can monitor any JMX metrices.

Can you suggest JMX metrics that will help monitor degraded cluster,
replica recovering, shard replica down, etc?

I couldn't find any metric on Solr documents.

Regards
Ganesh
Reply | Threaded
Open this post in threaded view
|

Re: JMX metrics for solr cloud cluster state

matthew sporleder
complain to new relic on their lagging solr support!!!  I have and
could use some support!

To address your actual question I have found JMX in solr to be crazy
unreliable but the admin/metrics web endpoint is pretty good.

I have some (crappy) python for parsing it for datadog:
https://github.com/msporleder/dd-solrcloud  you might be able to ship
something similar to insights if you were so inclined

On Sun, May 31, 2020 at 7:15 PM Ganesh Sethuraman
<[hidden email]> wrote:

>
> Hi
>
> We use New Relic to monitor Sold Cloud Cluster 7.2.1. we would like to get
> alerted on any cluster state change. Like for example degraded shard.
> Replica down. New relic can monitor any JMX metrices.
>
> Can you suggest JMX metrics that will help monitor degraded cluster,
> replica recovering, shard replica down, etc?
>
> I couldn't find any metric on Solr documents.
>
> Regards
> Ganesh
Reply | Threaded
Open this post in threaded view
|

Re: JMX metrics for solr cloud cluster state

Walter Underwood
I wrote a Python demon that gets clusterstatus from the API, parses it, and sends the counts of replicas in each state to InfluxDB. From there, we chart and alert in Grafana. New Relic is good, but we need other kinds of metrics, like the load balancer status from CloudWatch.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)

> On May 31, 2020, at 4:24 PM, matthew sporleder <[hidden email]> wrote:
>
> complain to new relic on their lagging solr support!!!  I have and
> could use some support!
>
> To address your actual question I have found JMX in solr to be crazy
> unreliable but the admin/metrics web endpoint is pretty good.
>
> I have some (crappy) python for parsing it for datadog:
> https://github.com/msporleder/dd-solrcloud  you might be able to ship
> something similar to insights if you were so inclined
>
> On Sun, May 31, 2020 at 7:15 PM Ganesh Sethuraman
> <[hidden email]> wrote:
>>
>> Hi
>>
>> We use New Relic to monitor Sold Cloud Cluster 7.2.1. we would like to get
>> alerted on any cluster state change. Like for example degraded shard.
>> Replica down. New relic can monitor any JMX metrices.
>>
>> Can you suggest JMX metrics that will help monitor degraded cluster,
>> replica recovering, shard replica down, etc?
>>
>> I couldn't find any metric on Solr documents.
>>
>> Regards
>> Ganesh

Reply | Threaded
Open this post in threaded view
|

Re: JMX metrics for solr cloud cluster state

GaneshSe
Can you suggest Solr Cloud JMX metrics for collection and replica status?
Trying to centralize the alert generation in NewRelic. New Relic only seems
to support JMX for the same.

On Sun, May 31, 2020, 7:29 PM Walter Underwood <[hidden email]>
wrote:

> I wrote a Python demon that gets clusterstatus from the API, parses it,
> and sends the counts of replicas in each state to InfluxDB. From there, we
> chart and alert in Grafana. New Relic is good, but we need other kinds of
> metrics, like the load balancer status from CloudWatch.
>
> wunder
> Walter Underwood
> [hidden email]
> http://observer.wunderwood.org/  (my blog)
>
> > On May 31, 2020, at 4:24 PM, matthew sporleder <[hidden email]>
> wrote:
> >
> > complain to new relic on their lagging solr support!!!  I have and
> > could use some support!
> >
> > To address your actual question I have found JMX in solr to be crazy
> > unreliable but the admin/metrics web endpoint is pretty good.
> >
> > I have some (crappy) python for parsing it for datadog:
> > https://github.com/msporleder/dd-solrcloud  you might be able to ship
> > something similar to insights if you were so inclined
> >
> > On Sun, May 31, 2020 at 7:15 PM Ganesh Sethuraman
> > <[hidden email]> wrote:
> >>
> >> Hi
> >>
> >> We use New Relic to monitor Sold Cloud Cluster 7.2.1. we would like to
> get
> >> alerted on any cluster state change. Like for example degraded shard.
> >> Replica down. New relic can monitor any JMX metrices.
> >>
> >> Can you suggest JMX metrics that will help monitor degraded cluster,
> >> replica recovering, shard replica down, etc?
> >>
> >> I couldn't find any metric on Solr documents.
> >>
> >> Regards
> >> Ganesh
>
>
Reply | Threaded
Open this post in threaded view
|

Re: JMX metrics for solr cloud cluster state

Walter Underwood
I gave up on JMX ages ago, so I can’t help there.

I’d open a bug with New Relic.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)

> On May 31, 2020, at 7:59 PM, Ganesh Sethuraman <[hidden email]> wrote:
>
> Can you suggest Solr Cloud JMX metrics for collection and replica status?
> Trying to centralize the alert generation in NewRelic. New Relic only seems
> to support JMX for the same.
>
> On Sun, May 31, 2020, 7:29 PM Walter Underwood <[hidden email]>
> wrote:
>
>> I wrote a Python demon that gets clusterstatus from the API, parses it,
>> and sends the counts of replicas in each state to InfluxDB. From there, we
>> chart and alert in Grafana. New Relic is good, but we need other kinds of
>> metrics, like the load balancer status from CloudWatch.
>>
>> wunder
>> Walter Underwood
>> [hidden email]
>> http://observer.wunderwood.org/  (my blog)
>>
>>> On May 31, 2020, at 4:24 PM, matthew sporleder <[hidden email]>
>> wrote:
>>>
>>> complain to new relic on their lagging solr support!!!  I have and
>>> could use some support!
>>>
>>> To address your actual question I have found JMX in solr to be crazy
>>> unreliable but the admin/metrics web endpoint is pretty good.
>>>
>>> I have some (crappy) python for parsing it for datadog:
>>> https://github.com/msporleder/dd-solrcloud  you might be able to ship
>>> something similar to insights if you were so inclined
>>>
>>> On Sun, May 31, 2020 at 7:15 PM Ganesh Sethuraman
>>> <[hidden email]> wrote:
>>>>
>>>> Hi
>>>>
>>>> We use New Relic to monitor Sold Cloud Cluster 7.2.1. we would like to
>> get
>>>> alerted on any cluster state change. Like for example degraded shard.
>>>> Replica down. New relic can monitor any JMX metrices.
>>>>
>>>> Can you suggest JMX metrics that will help monitor degraded cluster,
>>>> replica recovering, shard replica down, etc?
>>>>
>>>> I couldn't find any metric on Solr documents.
>>>>
>>>> Regards
>>>> Ganesh
>>
>>