Anyone have any comments on current solr monitoring favorites?

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Anyone have any comments on current solr monitoring favorites?

Petersen, Robert (Contr)
OK I'm probably going to open a can of worms here...  lol


In the old old days I used PSI probe to monitor solr running on tomcat which worked ok on a machine by machine basis.


Later I had a grafana dashboard on top of graphite monitoring which was really nice looking but kind of complicated to set up.


Even later I successfully just dropped in a newrelic java agent which had solr monitors and a dashboard right out of the box, but it costs money for the full tamale.


For basic JVM health and Solr QPS and time percentiles, does anyone have any favorites or other alternative suggestions?


Thanks in advance!

Robi

________________________________

This communication is confidential. Frontier only sends and receives email on the basis of the terms set out at http://www.frontier.com/email_disclaimer.
Reply | Threaded
Open this post in threaded view
|

Re: Anyone have any comments on current solr monitoring favorites?

Walter Underwood
We use New Relic for JVM, CPU, and disk monitoring.

I tried the built-in metrics support in 6.4, but it just didn’t do what we want. We want rates and percentiles for each request handler. That gives us 95th percentile for textbooks suggest or for homework search results page, etc. The Solr metrics didn’t do that. The Jetty metrics didn’t do that.

We built a dedicated servlet filter that goes in front of the Solr webapp and reports metrics. It has some special hacks to handle some weird behavior in SolrJ. A request to the “/srp” handler is sent as “/select?qt=/srp”, so we normalize that.

The metrics start with the cluster name, the hostname, and the collection. The rest is generated like this:

URL: GET /solr/textbooks/select?q=foo&qt=/auto
Metric: textbooks.GET./auto

URL: GET /solr/textbooks/select?q=foo
Metric: textbooks.GET./select

URL: GET /solr/questions/auto
Metric: questions.GET./auto

So a full metric for the cluster “solr-cloud” and the host “search01" would look like “solr-cloud.search01.solr.textbooks.GET./auto.m1_rate”.

We send all that to InfluxDB. We’ve configured a template so that each part of the metric name is mapped to a field, so we can write efficient queries in InfluxQL.

Metrics are graphed in Grafana. We have dashboards that mix Cloudwatch (for the load balancer) and InfluxDB.

I’m still working out the kinks in some of the more complicated queries, but the data is all there. I also want to expand the servlet filter to report HTTP response codes.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)


> On Nov 2, 2017, at 9:30 AM, Petersen, Robert (Contr) <[hidden email]> wrote:
>
> OK I'm probably going to open a can of worms here...  lol
>
>
> In the old old days I used PSI probe to monitor solr running on tomcat which worked ok on a machine by machine basis.
>
>
> Later I had a grafana dashboard on top of graphite monitoring which was really nice looking but kind of complicated to set up.
>
>
> Even later I successfully just dropped in a newrelic java agent which had solr monitors and a dashboard right out of the box, but it costs money for the full tamale.
>
>
> For basic JVM health and Solr QPS and time percentiles, does anyone have any favorites or other alternative suggestions?
>
>
> Thanks in advance!
>
> Robi
>
> ________________________________
>
> This communication is confidential. Frontier only sends and receives email on the basis of the terms set out at http://www.frontier.com/email_disclaimer.

Reply | Threaded
Open this post in threaded view
|

Re: Anyone have any comments on current solr monitoring favorites?

Emir Arnautović
Hi Robi,
Did you try Sematext’s SPM? It provides host, JVM and Solr metrics and more. We use it for monitoring our Solr instances and for consulting.

Disclaimer - see signature :)

Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/



> On 2 Nov 2017, at 19:35, Walter Underwood <[hidden email]> wrote:
>
> We use New Relic for JVM, CPU, and disk monitoring.
>
> I tried the built-in metrics support in 6.4, but it just didn’t do what we want. We want rates and percentiles for each request handler. That gives us 95th percentile for textbooks suggest or for homework search results page, etc. The Solr metrics didn’t do that. The Jetty metrics didn’t do that.
>
> We built a dedicated servlet filter that goes in front of the Solr webapp and reports metrics. It has some special hacks to handle some weird behavior in SolrJ. A request to the “/srp” handler is sent as “/select?qt=/srp”, so we normalize that.
>
> The metrics start with the cluster name, the hostname, and the collection. The rest is generated like this:
>
> URL: GET /solr/textbooks/select?q=foo&qt=/auto
> Metric: textbooks.GET./auto
>
> URL: GET /solr/textbooks/select?q=foo
> Metric: textbooks.GET./select
>
> URL: GET /solr/questions/auto
> Metric: questions.GET./auto
>
> So a full metric for the cluster “solr-cloud” and the host “search01" would look like “solr-cloud.search01.solr.textbooks.GET./auto.m1_rate”.
>
> We send all that to InfluxDB. We’ve configured a template so that each part of the metric name is mapped to a field, so we can write efficient queries in InfluxQL.
>
> Metrics are graphed in Grafana. We have dashboards that mix Cloudwatch (for the load balancer) and InfluxDB.
>
> I’m still working out the kinks in some of the more complicated queries, but the data is all there. I also want to expand the servlet filter to report HTTP response codes.
>
> wunder
> Walter Underwood
> [hidden email]
> http://observer.wunderwood.org/  (my blog)
>
>
>> On Nov 2, 2017, at 9:30 AM, Petersen, Robert (Contr) <[hidden email]> wrote:
>>
>> OK I'm probably going to open a can of worms here...  lol
>>
>>
>> In the old old days I used PSI probe to monitor solr running on tomcat which worked ok on a machine by machine basis.
>>
>>
>> Later I had a grafana dashboard on top of graphite monitoring which was really nice looking but kind of complicated to set up.
>>
>>
>> Even later I successfully just dropped in a newrelic java agent which had solr monitors and a dashboard right out of the box, but it costs money for the full tamale.
>>
>>
>> For basic JVM health and Solr QPS and time percentiles, does anyone have any favorites or other alternative suggestions?
>>
>>
>> Thanks in advance!
>>
>> Robi
>>
>> ________________________________
>>
>> This communication is confidential. Frontier only sends and receives email on the basis of the terms set out at http://www.frontier.com/email_disclaimer.
>

Reply | Threaded
Open this post in threaded view
|

Re: Anyone have any comments on current solr monitoring favorites?

WebsterHomer
My company uses Dynatrace for most everything in production. They have a
plugin for Solr that works with 6.*

On Thu, Nov 2, 2017 at 4:05 PM, Emir Arnautović <
[hidden email]> wrote:

> Hi Robi,
> Did you try Sematext’s SPM? It provides host, JVM and Solr metrics and
> more. We use it for monitoring our Solr instances and for consulting.
>
> Disclaimer - see signature :)
>
> Emir
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>
>
>
> > On 2 Nov 2017, at 19:35, Walter Underwood <[hidden email]> wrote:
> >
> > We use New Relic for JVM, CPU, and disk monitoring.
> >
> > I tried the built-in metrics support in 6.4, but it just didn’t do what
> we want. We want rates and percentiles for each request handler. That gives
> us 95th percentile for textbooks suggest or for homework search results
> page, etc. The Solr metrics didn’t do that. The Jetty metrics didn’t do
> that.
> >
> > We built a dedicated servlet filter that goes in front of the Solr
> webapp and reports metrics. It has some special hacks to handle some weird
> behavior in SolrJ. A request to the “/srp” handler is sent as
> “/select?qt=/srp”, so we normalize that.
> >
> > The metrics start with the cluster name, the hostname, and the
> collection. The rest is generated like this:
> >
> > URL: GET /solr/textbooks/select?q=foo&qt=/auto
> > Metric: textbooks.GET./auto
> >
> > URL: GET /solr/textbooks/select?q=foo
> > Metric: textbooks.GET./select
> >
> > URL: GET /solr/questions/auto
> > Metric: questions.GET./auto
> >
> > So a full metric for the cluster “solr-cloud” and the host “search01"
> would look like “solr-cloud.search01.solr.textbooks.GET./auto.m1_rate”.
> >
> > We send all that to InfluxDB. We’ve configured a template so that each
> part of the metric name is mapped to a field, so we can write efficient
> queries in InfluxQL.
> >
> > Metrics are graphed in Grafana. We have dashboards that mix Cloudwatch
> (for the load balancer) and InfluxDB.
> >
> > I’m still working out the kinks in some of the more complicated queries,
> but the data is all there. I also want to expand the servlet filter to
> report HTTP response codes.
> >
> > wunder
> > Walter Underwood
> > [hidden email]
> > http://observer.wunderwood.org/  (my blog)
> >
> >
> >> On Nov 2, 2017, at 9:30 AM, Petersen, Robert (Contr) <
> [hidden email]> wrote:
> >>
> >> OK I'm probably going to open a can of worms here...  lol
> >>
> >>
> >> In the old old days I used PSI probe to monitor solr running on tomcat
> which worked ok on a machine by machine basis.
> >>
> >>
> >> Later I had a grafana dashboard on top of graphite monitoring which was
> really nice looking but kind of complicated to set up.
> >>
> >>
> >> Even later I successfully just dropped in a newrelic java agent which
> had solr monitors and a dashboard right out of the box, but it costs money
> for the full tamale.
> >>
> >>
> >> For basic JVM health and Solr QPS and time percentiles, does anyone
> have any favorites or other alternative suggestions?
> >>
> >>
> >> Thanks in advance!
> >>
> >> Robi
> >>
> >> ________________________________
> >>
> >> This communication is confidential. Frontier only sends and receives
> email on the basis of the terms set out at http://www.frontier.com/email_
> disclaimer.
> >
>
>

--


This message and any attachment are confidential and may be privileged or
otherwise protected from disclosure. If you are not the intended recipient,
you must not copy this message or attachment or disclose the contents to
any other person. If you have received this transmission in error, please
notify the sender immediately and delete the message and any attachment
from your system. Merck KGaA, Darmstadt, Germany and any of its
subsidiaries do not accept liability for any omissions or errors in this
message which may arise as a result of E-Mail-transmission or for damages
resulting from any unauthorized changes of the content of this message and
any attachment thereto. Merck KGaA, Darmstadt, Germany and any of its
subsidiaries do not guarantee that this message is free of viruses and does
not accept liability for any damages caused by any virus transmitted
therewith.

Click http://www.emdgroup.com/disclaimer to access the German, French,
Spanish and Portuguese versions of this disclaimer.
Reply | Threaded
Open this post in threaded view
|

Re: Anyone have any comments on current solr monitoring favorites?

Petersen, Robert (Contr)
In reply to this post by Emir Arnautović
PS I knew sematext would be required to chime in here!  😊


Is there a non-expiring dev version I could experiment with? I think I did sign up for a trial years ago from a different company... I was actually wondering about hooking it up to my personal AWS based solr cloud instance.


Thanks

Robi

________________________________
From: Emir Arnautović <[hidden email]>
Sent: Thursday, November 2, 2017 2:05:10 PM
To: [hidden email]
Subject: Re: Anyone have any comments on current solr monitoring favorites?

Hi Robi,
Did you try Sematext’s SPM? It provides host, JVM and Solr metrics and more. We use it for monitoring our Solr instances and for consulting.

Disclaimer - see signature :)

Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/



> On 2 Nov 2017, at 19:35, Walter Underwood <[hidden email]> wrote:
>
> We use New Relic for JVM, CPU, and disk monitoring.
>
> I tried the built-in metrics support in 6.4, but it just didn’t do what we want. We want rates and percentiles for each request handler. That gives us 95th percentile for textbooks suggest or for homework search results page, etc. The Solr metrics didn’t do that. The Jetty metrics didn’t do that.
>
> We built a dedicated servlet filter that goes in front of the Solr webapp and reports metrics. It has some special hacks to handle some weird behavior in SolrJ. A request to the “/srp” handler is sent as “/select?qt=/srp”, so we normalize that.
>
> The metrics start with the cluster name, the hostname, and the collection. The rest is generated like this:
>
> URL: GET /solr/textbooks/select?q=foo&qt=/auto
> Metric: textbooks.GET./auto
>
> URL: GET /solr/textbooks/select?q=foo
> Metric: textbooks.GET./select
>
> URL: GET /solr/questions/auto
> Metric: questions.GET./auto
>
> So a full metric for the cluster “solr-cloud” and the host “search01" would look like “solr-cloud.search01.solr.textbooks.GET./auto.m1_rate”.
>
> We send all that to InfluxDB. We’ve configured a template so that each part of the metric name is mapped to a field, so we can write efficient queries in InfluxQL.
>
> Metrics are graphed in Grafana. We have dashboards that mix Cloudwatch (for the load balancer) and InfluxDB.
>
> I’m still working out the kinks in some of the more complicated queries, but the data is all there. I also want to expand the servlet filter to report HTTP response codes.
>
> wunder
> Walter Underwood
> [hidden email]
> http://observer.wunderwood.org/  (my blog)
>
>
>> On Nov 2, 2017, at 9:30 AM, Petersen, Robert (Contr) <[hidden email]> wrote:
>>
>> OK I'm probably going to open a can of worms here...  lol
>>
>>
>> In the old old days I used PSI probe to monitor solr running on tomcat which worked ok on a machine by machine basis.
>>
>>
>> Later I had a grafana dashboard on top of graphite monitoring which was really nice looking but kind of complicated to set up.
>>
>>
>> Even later I successfully just dropped in a newrelic java agent which had solr monitors and a dashboard right out of the box, but it costs money for the full tamale.
>>
>>
>> For basic JVM health and Solr QPS and time percentiles, does anyone have any favorites or other alternative suggestions?
>>
>>
>> Thanks in advance!
>>
>> Robi
>>
>> ________________________________
>>
>> This communication is confidential. Frontier only sends and receives email on the basis of the terms set out at http://www.frontier.com/email_disclaimer.
>

Reply | Threaded
Open this post in threaded view
|

Re: Anyone have any comments on current solr monitoring favorites?

Daniel Ortega
Hi Robert,

We use the following stack:

- Prometheus to scrape metrics (https://prometheus.io/)
- Prometheus node exporter to export "machine metrics" (Disk, network
usage, etc.) (https://github.com/prometheus/node_exporter)
- Prometheus JMX exporter to export "Solr metrics" (Cache usage, QPS,
Response times...) (https://github.com/prometheus/jmx_exporter)
- Grafana to visualize all the data scrapped by Prometheus (
https://grafana.com/)

Best regards
Daniel Ortega

2017-11-06 20:13 GMT+01:00 Petersen, Robert (Contr) <
[hidden email]>:

> PS I knew sematext would be required to chime in here!  😊
>
>
> Is there a non-expiring dev version I could experiment with? I think I did
> sign up for a trial years ago from a different company... I was actually
> wondering about hooking it up to my personal AWS based solr cloud instance.
>
>
> Thanks
>
> Robi
>
> ________________________________
> From: Emir Arnautović <[hidden email]>
> Sent: Thursday, November 2, 2017 2:05:10 PM
> To: [hidden email]
> Subject: Re: Anyone have any comments on current solr monitoring favorites?
>
> Hi Robi,
> Did you try Sematext’s SPM? It provides host, JVM and Solr metrics and
> more. We use it for monitoring our Solr instances and for consulting.
>
> Disclaimer - see signature :)
>
> Emir
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>
>
>
> > On 2 Nov 2017, at 19:35, Walter Underwood <[hidden email]> wrote:
> >
> > We use New Relic for JVM, CPU, and disk monitoring.
> >
> > I tried the built-in metrics support in 6.4, but it just didn’t do what
> we want. We want rates and percentiles for each request handler. That gives
> us 95th percentile for textbooks suggest or for homework search results
> page, etc. The Solr metrics didn’t do that. The Jetty metrics didn’t do
> that.
> >
> > We built a dedicated servlet filter that goes in front of the Solr
> webapp and reports metrics. It has some special hacks to handle some weird
> behavior in SolrJ. A request to the “/srp” handler is sent as
> “/select?qt=/srp”, so we normalize that.
> >
> > The metrics start with the cluster name, the hostname, and the
> collection. The rest is generated like this:
> >
> > URL: GET /solr/textbooks/select?q=foo&qt=/auto
> > Metric: textbooks.GET./auto
> >
> > URL: GET /solr/textbooks/select?q=foo
> > Metric: textbooks.GET./select
> >
> > URL: GET /solr/questions/auto
> > Metric: questions.GET./auto
> >
> > So a full metric for the cluster “solr-cloud” and the host “search01"
> would look like “solr-cloud.search01.solr.textbooks.GET./auto.m1_rate”.
> >
> > We send all that to InfluxDB. We’ve configured a template so that each
> part of the metric name is mapped to a field, so we can write efficient
> queries in InfluxQL.
> >
> > Metrics are graphed in Grafana. We have dashboards that mix Cloudwatch
> (for the load balancer) and InfluxDB.
> >
> > I’m still working out the kinks in some of the more complicated queries,
> but the data is all there. I also want to expand the servlet filter to
> report HTTP response codes.
> >
> > wunder
> > Walter Underwood
> > [hidden email]
> > http://observer.wunderwood.org/  (my blog)
> >
> >
> >> On Nov 2, 2017, at 9:30 AM, Petersen, Robert (Contr) <
> [hidden email]> wrote:
> >>
> >> OK I'm probably going to open a can of worms here...  lol
> >>
> >>
> >> In the old old days I used PSI probe to monitor solr running on tomcat
> which worked ok on a machine by machine basis.
> >>
> >>
> >> Later I had a grafana dashboard on top of graphite monitoring which was
> really nice looking but kind of complicated to set up.
> >>
> >>
> >> Even later I successfully just dropped in a newrelic java agent which
> had solr monitors and a dashboard right out of the box, but it costs money
> for the full tamale.
> >>
> >>
> >> For basic JVM health and Solr QPS and time percentiles, does anyone
> have any favorites or other alternative suggestions?
> >>
> >>
> >> Thanks in advance!
> >>
> >> Robi
> >>
> >> ________________________________
> >>
> >> This communication is confidential. Frontier only sends and receives
> email on the basis of the terms set out at http://www.frontier.com/email_
> disclaimer.
> >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Anyone have any comments on current solr monitoring favorites?

Petersen, Robert (Contr)
Interesting! Finally a Grafana user... Thanks Daniel, I will follow your links. That looks promising.


Is anyone using Grafana over Graphite?


Thanks

Robi

________________________________
From: Daniel Ortega <[hidden email]>
Sent: Monday, November 6, 2017 11:19:10 AM
To: [hidden email]
Subject: Re: Anyone have any comments on current solr monitoring favorites?

Hi Robert,

We use the following stack:

- Prometheus to scrape metrics (https://prometheus.io/)
- Prometheus node exporter to export "machine metrics" (Disk, network
usage, etc.) (https://github.com/prometheus/node_exporter)
- Prometheus JMX exporter to export "Solr metrics" (Cache usage, QPS,
Response times...) (https://github.com/prometheus/jmx_exporter)
- Grafana to visualize all the data scrapped by Prometheus (
https://grafana.com/)

Best regards
Daniel Ortega

2017-11-06 20:13 GMT+01:00 Petersen, Robert (Contr) <
[hidden email]>:

> PS I knew sematext would be required to chime in here!  😊
>
>
> Is there a non-expiring dev version I could experiment with? I think I did
> sign up for a trial years ago from a different company... I was actually
> wondering about hooking it up to my personal AWS based solr cloud instance.
>
>
> Thanks
>
> Robi
>
> ________________________________
> From: Emir Arnautović <[hidden email]>
> Sent: Thursday, November 2, 2017 2:05:10 PM
> To: [hidden email]
> Subject: Re: Anyone have any comments on current solr monitoring favorites?
>
> Hi Robi,
> Did you try Sematext’s SPM? It provides host, JVM and Solr metrics and
> more. We use it for monitoring our Solr instances and for consulting.
>
> Disclaimer - see signature :)
>
> Emir
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>
>
>
> > On 2 Nov 2017, at 19:35, Walter Underwood <[hidden email]> wrote:
> >
> > We use New Relic for JVM, CPU, and disk monitoring.
> >
> > I tried the built-in metrics support in 6.4, but it just didn’t do what
> we want. We want rates and percentiles for each request handler. That gives
> us 95th percentile for textbooks suggest or for homework search results
> page, etc. The Solr metrics didn’t do that. The Jetty metrics didn’t do
> that.
> >
> > We built a dedicated servlet filter that goes in front of the Solr
> webapp and reports metrics. It has some special hacks to handle some weird
> behavior in SolrJ. A request to the “/srp” handler is sent as
> “/select?qt=/srp”, so we normalize that.
> >
> > The metrics start with the cluster name, the hostname, and the
> collection. The rest is generated like this:
> >
> > URL: GET /solr/textbooks/select?q=foo&qt=/auto
> > Metric: textbooks.GET./auto
> >
> > URL: GET /solr/textbooks/select?q=foo
> > Metric: textbooks.GET./select
> >
> > URL: GET /solr/questions/auto
> > Metric: questions.GET./auto
> >
> > So a full metric for the cluster “solr-cloud” and the host “search01"
> would look like “solr-cloud.search01.solr.textbooks.GET./auto.m1_rate”.
> >
> > We send all that to InfluxDB. We’ve configured a template so that each
> part of the metric name is mapped to a field, so we can write efficient
> queries in InfluxQL.
> >
> > Metrics are graphed in Grafana. We have dashboards that mix Cloudwatch
> (for the load balancer) and InfluxDB.
> >
> > I’m still working out the kinks in some of the more complicated queries,
> but the data is all there. I also want to expand the servlet filter to
> report HTTP response codes.
> >
> > wunder
> > Walter Underwood
> > [hidden email]
> > http://observer.wunderwood.org/  (my blog)
> >
> >
> >> On Nov 2, 2017, at 9:30 AM, Petersen, Robert (Contr) <
> [hidden email]> wrote:
> >>
> >> OK I'm probably going to open a can of worms here...  lol
> >>
> >>
> >> In the old old days I used PSI probe to monitor solr running on tomcat
> which worked ok on a machine by machine basis.
> >>
> >>
> >> Later I had a grafana dashboard on top of graphite monitoring which was
> really nice looking but kind of complicated to set up.
> >>
> >>
> >> Even later I successfully just dropped in a newrelic java agent which
> had solr monitors and a dashboard right out of the box, but it costs money
> for the full tamale.
> >>
> >>
> >> For basic JVM health and Solr QPS and time percentiles, does anyone
> have any favorites or other alternative suggestions?
> >>
> >>
> >> Thanks in advance!
> >>
> >> Robi
> >>
> >> ________________________________
> >>
> >> This communication is confidential. Frontier only sends and receives
> email on the basis of the terms set out at http://www.frontier.com/email_
> disclaimer.
> >
>
>

________________________________

This communication is confidential. Frontier only sends and receives email on the basis of the terms set out at http://www.frontier.com/email_disclaimer.
Reply | Threaded
Open this post in threaded view
|

Re: Anyone have any comments on current solr monitoring favorites?

Walter Underwood
Look back down the string to my post. We use Grafana.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)


> On Nov 6, 2017, at 11:23 AM, Petersen, Robert (Contr) <[hidden email]> wrote:
>
> Interesting! Finally a Grafana user... Thanks Daniel, I will follow your links. That looks promising.
>
>
> Is anyone using Grafana over Graphite?
>
>
> Thanks
>
> Robi
>
> ________________________________
> From: Daniel Ortega <[hidden email]>
> Sent: Monday, November 6, 2017 11:19:10 AM
> To: [hidden email]
> Subject: Re: Anyone have any comments on current solr monitoring favorites?
>
> Hi Robert,
>
> We use the following stack:
>
> - Prometheus to scrape metrics (https://prometheus.io/)
> - Prometheus node exporter to export "machine metrics" (Disk, network
> usage, etc.) (https://github.com/prometheus/node_exporter)
> - Prometheus JMX exporter to export "Solr metrics" (Cache usage, QPS,
> Response times...) (https://github.com/prometheus/jmx_exporter)
> - Grafana to visualize all the data scrapped by Prometheus (
> https://grafana.com/)
>
> Best regards
> Daniel Ortega
>
> 2017-11-06 20:13 GMT+01:00 Petersen, Robert (Contr) <
> [hidden email]>:
>
>> PS I knew sematext would be required to chime in here!  😊
>>
>>
>> Is there a non-expiring dev version I could experiment with? I think I did
>> sign up for a trial years ago from a different company... I was actually
>> wondering about hooking it up to my personal AWS based solr cloud instance.
>>
>>
>> Thanks
>>
>> Robi
>>
>> ________________________________
>> From: Emir Arnautović <[hidden email]>
>> Sent: Thursday, November 2, 2017 2:05:10 PM
>> To: [hidden email]
>> Subject: Re: Anyone have any comments on current solr monitoring favorites?
>>
>> Hi Robi,
>> Did you try Sematext’s SPM? It provides host, JVM and Solr metrics and
>> more. We use it for monitoring our Solr instances and for consulting.
>>
>> Disclaimer - see signature :)
>>
>> Emir
>> --
>> Monitoring - Log Management - Alerting - Anomaly Detection
>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>
>>
>>
>>> On 2 Nov 2017, at 19:35, Walter Underwood <[hidden email]> wrote:
>>>
>>> We use New Relic for JVM, CPU, and disk monitoring.
>>>
>>> I tried the built-in metrics support in 6.4, but it just didn’t do what
>> we want. We want rates and percentiles for each request handler. That gives
>> us 95th percentile for textbooks suggest or for homework search results
>> page, etc. The Solr metrics didn’t do that. The Jetty metrics didn’t do
>> that.
>>>
>>> We built a dedicated servlet filter that goes in front of the Solr
>> webapp and reports metrics. It has some special hacks to handle some weird
>> behavior in SolrJ. A request to the “/srp” handler is sent as
>> “/select?qt=/srp”, so we normalize that.
>>>
>>> The metrics start with the cluster name, the hostname, and the
>> collection. The rest is generated like this:
>>>
>>> URL: GET /solr/textbooks/select?q=foo&qt=/auto
>>> Metric: textbooks.GET./auto
>>>
>>> URL: GET /solr/textbooks/select?q=foo
>>> Metric: textbooks.GET./select
>>>
>>> URL: GET /solr/questions/auto
>>> Metric: questions.GET./auto
>>>
>>> So a full metric for the cluster “solr-cloud” and the host “search01"
>> would look like “solr-cloud.search01.solr.textbooks.GET./auto.m1_rate”.
>>>
>>> We send all that to InfluxDB. We’ve configured a template so that each
>> part of the metric name is mapped to a field, so we can write efficient
>> queries in InfluxQL.
>>>
>>> Metrics are graphed in Grafana. We have dashboards that mix Cloudwatch
>> (for the load balancer) and InfluxDB.
>>>
>>> I’m still working out the kinks in some of the more complicated queries,
>> but the data is all there. I also want to expand the servlet filter to
>> report HTTP response codes.
>>>
>>> wunder
>>> Walter Underwood
>>> [hidden email]
>>> http://observer.wunderwood.org/  (my blog)
>>>
>>>
>>>> On Nov 2, 2017, at 9:30 AM, Petersen, Robert (Contr) <
>> [hidden email]> wrote:
>>>>
>>>> OK I'm probably going to open a can of worms here...  lol
>>>>
>>>>
>>>> In the old old days I used PSI probe to monitor solr running on tomcat
>> which worked ok on a machine by machine basis.
>>>>
>>>>
>>>> Later I had a grafana dashboard on top of graphite monitoring which was
>> really nice looking but kind of complicated to set up.
>>>>
>>>>
>>>> Even later I successfully just dropped in a newrelic java agent which
>> had solr monitors and a dashboard right out of the box, but it costs money
>> for the full tamale.
>>>>
>>>>
>>>> For basic JVM health and Solr QPS and time percentiles, does anyone
>> have any favorites or other alternative suggestions?
>>>>
>>>>
>>>> Thanks in advance!
>>>>
>>>> Robi
>>>>
>>>> ________________________________
>>>>
>>>> This communication is confidential. Frontier only sends and receives
>> email on the basis of the terms set out at http://www.frontier.com/email_
>> disclaimer.
>>>
>>
>>
>
> ________________________________
>
> This communication is confidential. Frontier only sends and receives email on the basis of the terms set out at http://www.frontier.com/email_disclaimer.

Reply | Threaded
Open this post in threaded view
|

Re: Anyone have any comments on current solr monitoring favorites?

Emir Arnautović
In reply to this post by Petersen, Robert (Contr)
Hi Robert,
There is a free plan with limited number of nodes and 30 min retention. It should be straight forward to install it on AWS based solr cloud instance, but if you run into some issues you can used built in chat to get in touch with somebody to help you set it up.

Regards,
Emir
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/



> On 6 Nov 2017, at 20:13, Petersen, Robert (Contr) <[hidden email]> wrote:
>
> PS I knew sematext would be required to chime in here!  😊
>
>
> Is there a non-expiring dev version I could experiment with? I think I did sign up for a trial years ago from a different company... I was actually wondering about hooking it up to my personal AWS based solr cloud instance.
>
>
> Thanks
>
> Robi
>
> ________________________________
> From: Emir Arnautović <[hidden email]>
> Sent: Thursday, November 2, 2017 2:05:10 PM
> To: [hidden email]
> Subject: Re: Anyone have any comments on current solr monitoring favorites?
>
> Hi Robi,
> Did you try Sematext’s SPM? It provides host, JVM and Solr metrics and more. We use it for monitoring our Solr instances and for consulting.
>
> Disclaimer - see signature :)
>
> Emir
> --
> Monitoring - Log Management - Alerting - Anomaly Detection
> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>
>
>
>> On 2 Nov 2017, at 19:35, Walter Underwood <[hidden email]> wrote:
>>
>> We use New Relic for JVM, CPU, and disk monitoring.
>>
>> I tried the built-in metrics support in 6.4, but it just didn’t do what we want. We want rates and percentiles for each request handler. That gives us 95th percentile for textbooks suggest or for homework search results page, etc. The Solr metrics didn’t do that. The Jetty metrics didn’t do that.
>>
>> We built a dedicated servlet filter that goes in front of the Solr webapp and reports metrics. It has some special hacks to handle some weird behavior in SolrJ. A request to the “/srp” handler is sent as “/select?qt=/srp”, so we normalize that.
>>
>> The metrics start with the cluster name, the hostname, and the collection. The rest is generated like this:
>>
>> URL: GET /solr/textbooks/select?q=foo&qt=/auto
>> Metric: textbooks.GET./auto
>>
>> URL: GET /solr/textbooks/select?q=foo
>> Metric: textbooks.GET./select
>>
>> URL: GET /solr/questions/auto
>> Metric: questions.GET./auto
>>
>> So a full metric for the cluster “solr-cloud” and the host “search01" would look like “solr-cloud.search01.solr.textbooks.GET./auto.m1_rate”.
>>
>> We send all that to InfluxDB. We’ve configured a template so that each part of the metric name is mapped to a field, so we can write efficient queries in InfluxQL.
>>
>> Metrics are graphed in Grafana. We have dashboards that mix Cloudwatch (for the load balancer) and InfluxDB.
>>
>> I’m still working out the kinks in some of the more complicated queries, but the data is all there. I also want to expand the servlet filter to report HTTP response codes.
>>
>> wunder
>> Walter Underwood
>> [hidden email]
>> http://observer.wunderwood.org/  (my blog)
>>
>>
>>> On Nov 2, 2017, at 9:30 AM, Petersen, Robert (Contr) <[hidden email]> wrote:
>>>
>>> OK I'm probably going to open a can of worms here...  lol
>>>
>>>
>>> In the old old days I used PSI probe to monitor solr running on tomcat which worked ok on a machine by machine basis.
>>>
>>>
>>> Later I had a grafana dashboard on top of graphite monitoring which was really nice looking but kind of complicated to set up.
>>>
>>>
>>> Even later I successfully just dropped in a newrelic java agent which had solr monitors and a dashboard right out of the box, but it costs money for the full tamale.
>>>
>>>
>>> For basic JVM health and Solr QPS and time percentiles, does anyone have any favorites or other alternative suggestions?
>>>
>>>
>>> Thanks in advance!
>>>
>>> Robi
>>>
>>> ________________________________
>>>
>>> This communication is confidential. Frontier only sends and receives email on the basis of the terms set out at http://www.frontier.com/email_disclaimer.
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Anyone have any comments on current solr monitoring favorites?

Petersen, Robert (Contr)
In reply to this post by Walter Underwood
Hi Walter,


Yes, now I see it. I'm wondering about using Grafana and New Relic at the same time since New Relic has a dashboard and also costs money for corporate use. I guess after a reread you are using Grafana to visualize the influxDB data and New Relic just for JVM right?  Did this give you more control over the solr metrics you are monitoring? (PS I've never heard of influxDB)


Thanks

Robi

________________________________
From: Walter Underwood <[hidden email]>
Sent: Monday, November 6, 2017 11:26:07 AM
To: [hidden email]
Subject: Re: Anyone have any comments on current solr monitoring favorites?

Look back down the string to my post. We use Grafana.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)


> On Nov 6, 2017, at 11:23 AM, Petersen, Robert (Contr) <[hidden email]> wrote:
>
> Interesting! Finally a Grafana user... Thanks Daniel, I will follow your links. That looks promising.
>
>
> Is anyone using Grafana over Graphite?
>
>
> Thanks
>
> Robi
>
> ________________________________
> From: Daniel Ortega <[hidden email]>
> Sent: Monday, November 6, 2017 11:19:10 AM
> To: [hidden email]
> Subject: Re: Anyone have any comments on current solr monitoring favorites?
>
> Hi Robert,
>
> We use the following stack:
>
> - Prometheus to scrape metrics (https://prometheus.io/)
> - Prometheus node exporter to export "machine metrics" (Disk, network
> usage, etc.) (https://github.com/prometheus/node_exporter)
> - Prometheus JMX exporter to export "Solr metrics" (Cache usage, QPS,
> Response times...) (https://github.com/prometheus/jmx_exporter)
> - Grafana to visualize all the data scrapped by Prometheus (
> https://grafana.com/)
>
> Best regards
> Daniel Ortega
>
> 2017-11-06 20:13 GMT+01:00 Petersen, Robert (Contr) <
> [hidden email]>:
>
>> PS I knew sematext would be required to chime in here!  😊
>>
>>
>> Is there a non-expiring dev version I could experiment with? I think I did
>> sign up for a trial years ago from a different company... I was actually
>> wondering about hooking it up to my personal AWS based solr cloud instance.
>>
>>
>> Thanks
>>
>> Robi
>>
>> ________________________________
>> From: Emir Arnautović <[hidden email]>
>> Sent: Thursday, November 2, 2017 2:05:10 PM
>> To: [hidden email]
>> Subject: Re: Anyone have any comments on current solr monitoring favorites?
>>
>> Hi Robi,
>> Did you try Sematext’s SPM? It provides host, JVM and Solr metrics and
>> more. We use it for monitoring our Solr instances and for consulting.
>>
>> Disclaimer - see signature :)
>>
>> Emir
>> --
>> Monitoring - Log Management - Alerting - Anomaly Detection
>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>
>>
>>
>>> On 2 Nov 2017, at 19:35, Walter Underwood <[hidden email]> wrote:
>>>
>>> We use New Relic for JVM, CPU, and disk monitoring.
>>>
>>> I tried the built-in metrics support in 6.4, but it just didn’t do what
>> we want. We want rates and percentiles for each request handler. That gives
>> us 95th percentile for textbooks suggest or for homework search results
>> page, etc. The Solr metrics didn’t do that. The Jetty metrics didn’t do
>> that.
>>>
>>> We built a dedicated servlet filter that goes in front of the Solr
>> webapp and reports metrics. It has some special hacks to handle some weird
>> behavior in SolrJ. A request to the “/srp” handler is sent as
>> “/select?qt=/srp”, so we normalize that.
>>>
>>> The metrics start with the cluster name, the hostname, and the
>> collection. The rest is generated like this:
>>>
>>> URL: GET /solr/textbooks/select?q=foo&qt=/auto
>>> Metric: textbooks.GET./auto
>>>
>>> URL: GET /solr/textbooks/select?q=foo
>>> Metric: textbooks.GET./select
>>>
>>> URL: GET /solr/questions/auto
>>> Metric: questions.GET./auto
>>>
>>> So a full metric for the cluster “solr-cloud” and the host “search01"
>> would look like “solr-cloud.search01.solr.textbooks.GET./auto.m1_rate”.
>>>
>>> We send all that to InfluxDB. We’ve configured a template so that each
>> part of the metric name is mapped to a field, so we can write efficient
>> queries in InfluxQL.
>>>
>>> Metrics are graphed in Grafana. We have dashboards that mix Cloudwatch
>> (for the load balancer) and InfluxDB.
>>>
>>> I’m still working out the kinks in some of the more complicated queries,
>> but the data is all there. I also want to expand the servlet filter to
>> report HTTP response codes.
>>>
>>> wunder
>>> Walter Underwood
>>> [hidden email]
>>> http://observer.wunderwood.org/  (my blog)
>>>
>>>
>>>> On Nov 2, 2017, at 9:30 AM, Petersen, Robert (Contr) <
>> [hidden email]> wrote:
>>>>
>>>> OK I'm probably going to open a can of worms here...  lol
>>>>
>>>>
>>>> In the old old days I used PSI probe to monitor solr running on tomcat
>> which worked ok on a machine by machine basis.
>>>>
>>>>
>>>> Later I had a grafana dashboard on top of graphite monitoring which was
>> really nice looking but kind of complicated to set up.
>>>>
>>>>
>>>> Even later I successfully just dropped in a newrelic java agent which
>> had solr monitors and a dashboard right out of the box, but it costs money
>> for the full tamale.
>>>>
>>>>
>>>> For basic JVM health and Solr QPS and time percentiles, does anyone
>> have any favorites or other alternative suggestions?
>>>>
>>>>
>>>> Thanks in advance!
>>>>
>>>> Robi
>>>>
>>>> ________________________________
>>>>
>>>> This communication is confidential. Frontier only sends and receives
>> email on the basis of the terms set out at http://www.frontier.com/email_
>> disclaimer.
>>>
>>
>>
>
> ________________________________
>
> This communication is confidential. Frontier only sends and receives email on the basis of the terms set out at http://www.frontier.com/email_disclaimer.

Reply | Threaded
Open this post in threaded view
|

Re: Anyone have any comments on current solr monitoring favorites?

Walter Underwood
We use New Relic across the site, but it doesn’t split out traffic to different endpoints. It also cannot distinguish between search traffic to the cluster and intra-cluster traffic. With four shards, the total traffic is 4X bigger than the incoming traffic.

We have a bunch of business metrics (orders) and other stuff that is currently in Graphite. We’ll almost certainly move all that to InfluxDB and Grafana.

The Solr metrics were overloading the Graphite database, so we’re the first service that is trying InfluxDB.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)


> On Nov 6, 2017, at 1:31 PM, Petersen, Robert (Contr) <[hidden email]> wrote:
>
> Hi Walter,
>
>
> Yes, now I see it. I'm wondering about using Grafana and New Relic at the same time since New Relic has a dashboard and also costs money for corporate use. I guess after a reread you are using Grafana to visualize the influxDB data and New Relic just for JVM right?  Did this give you more control over the solr metrics you are monitoring? (PS I've never heard of influxDB)
>
>
> Thanks
>
> Robi
>
> ________________________________
> From: Walter Underwood <[hidden email]>
> Sent: Monday, November 6, 2017 11:26:07 AM
> To: [hidden email]
> Subject: Re: Anyone have any comments on current solr monitoring favorites?
>
> Look back down the string to my post. We use Grafana.
>
> wunder
> Walter Underwood
> [hidden email]
> http://observer.wunderwood.org/  (my blog)
>
>
>> On Nov 6, 2017, at 11:23 AM, Petersen, Robert (Contr) <[hidden email]> wrote:
>>
>> Interesting! Finally a Grafana user... Thanks Daniel, I will follow your links. That looks promising.
>>
>>
>> Is anyone using Grafana over Graphite?
>>
>>
>> Thanks
>>
>> Robi
>>
>> ________________________________
>> From: Daniel Ortega <[hidden email]>
>> Sent: Monday, November 6, 2017 11:19:10 AM
>> To: [hidden email]
>> Subject: Re: Anyone have any comments on current solr monitoring favorites?
>>
>> Hi Robert,
>>
>> We use the following stack:
>>
>> - Prometheus to scrape metrics (https://prometheus.io/)
>> - Prometheus node exporter to export "machine metrics" (Disk, network
>> usage, etc.) (https://github.com/prometheus/node_exporter)
>> - Prometheus JMX exporter to export "Solr metrics" (Cache usage, QPS,
>> Response times...) (https://github.com/prometheus/jmx_exporter)
>> - Grafana to visualize all the data scrapped by Prometheus (
>> https://grafana.com/)
>>
>> Best regards
>> Daniel Ortega
>>
>> 2017-11-06 20:13 GMT+01:00 Petersen, Robert (Contr) <
>> [hidden email]>:
>>
>>> PS I knew sematext would be required to chime in here!  😊
>>>
>>>
>>> Is there a non-expiring dev version I could experiment with? I think I did
>>> sign up for a trial years ago from a different company... I was actually
>>> wondering about hooking it up to my personal AWS based solr cloud instance.
>>>
>>>
>>> Thanks
>>>
>>> Robi
>>>
>>> ________________________________
>>> From: Emir Arnautović <[hidden email]>
>>> Sent: Thursday, November 2, 2017 2:05:10 PM
>>> To: [hidden email]
>>> Subject: Re: Anyone have any comments on current solr monitoring favorites?
>>>
>>> Hi Robi,
>>> Did you try Sematext’s SPM? It provides host, JVM and Solr metrics and
>>> more. We use it for monitoring our Solr instances and for consulting.
>>>
>>> Disclaimer - see signature :)
>>>
>>> Emir
>>> --
>>> Monitoring - Log Management - Alerting - Anomaly Detection
>>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>>
>>>
>>>
>>>> On 2 Nov 2017, at 19:35, Walter Underwood <[hidden email]> wrote:
>>>>
>>>> We use New Relic for JVM, CPU, and disk monitoring.
>>>>
>>>> I tried the built-in metrics support in 6.4, but it just didn’t do what
>>> we want. We want rates and percentiles for each request handler. That gives
>>> us 95th percentile for textbooks suggest or for homework search results
>>> page, etc. The Solr metrics didn’t do that. The Jetty metrics didn’t do
>>> that.
>>>>
>>>> We built a dedicated servlet filter that goes in front of the Solr
>>> webapp and reports metrics. It has some special hacks to handle some weird
>>> behavior in SolrJ. A request to the “/srp” handler is sent as
>>> “/select?qt=/srp”, so we normalize that.
>>>>
>>>> The metrics start with the cluster name, the hostname, and the
>>> collection. The rest is generated like this:
>>>>
>>>> URL: GET /solr/textbooks/select?q=foo&qt=/auto
>>>> Metric: textbooks.GET./auto
>>>>
>>>> URL: GET /solr/textbooks/select?q=foo
>>>> Metric: textbooks.GET./select
>>>>
>>>> URL: GET /solr/questions/auto
>>>> Metric: questions.GET./auto
>>>>
>>>> So a full metric for the cluster “solr-cloud” and the host “search01"
>>> would look like “solr-cloud.search01.solr.textbooks.GET./auto.m1_rate”.
>>>>
>>>> We send all that to InfluxDB. We’ve configured a template so that each
>>> part of the metric name is mapped to a field, so we can write efficient
>>> queries in InfluxQL.
>>>>
>>>> Metrics are graphed in Grafana. We have dashboards that mix Cloudwatch
>>> (for the load balancer) and InfluxDB.
>>>>
>>>> I’m still working out the kinks in some of the more complicated queries,
>>> but the data is all there. I also want to expand the servlet filter to
>>> report HTTP response codes.
>>>>
>>>> wunder
>>>> Walter Underwood
>>>> [hidden email]
>>>> http://observer.wunderwood.org/  (my blog)
>>>>
>>>>
>>>>> On Nov 2, 2017, at 9:30 AM, Petersen, Robert (Contr) <
>>> [hidden email]> wrote:
>>>>>
>>>>> OK I'm probably going to open a can of worms here...  lol
>>>>>
>>>>>
>>>>> In the old old days I used PSI probe to monitor solr running on tomcat
>>> which worked ok on a machine by machine basis.
>>>>>
>>>>>
>>>>> Later I had a grafana dashboard on top of graphite monitoring which was
>>> really nice looking but kind of complicated to set up.
>>>>>
>>>>>
>>>>> Even later I successfully just dropped in a newrelic java agent which
>>> had solr monitors and a dashboard right out of the box, but it costs money
>>> for the full tamale.
>>>>>
>>>>>
>>>>> For basic JVM health and Solr QPS and time percentiles, does anyone
>>> have any favorites or other alternative suggestions?
>>>>>
>>>>>
>>>>> Thanks in advance!
>>>>>
>>>>> Robi
>>>>>
>>>>> ________________________________
>>>>>
>>>>> This communication is confidential. Frontier only sends and receives
>>> email on the basis of the terms set out at http://www.frontier.com/email_
>>> disclaimer.
>>>>
>>>
>>>
>>
>> ________________________________
>>
>> This communication is confidential. Frontier only sends and receives email on the basis of the terms set out at http://www.frontier.com/email_disclaimer.
>

Reply | Threaded
Open this post in threaded view
|

Re: Anyone have any comments on current solr monitoring favorites?

Petersen, Robert (Contr)
Hi Walter,


OK now that sounds really interesting. I actually just turned on logging in Jetty and yes did see all the intra-cluster traffic there. I'm pushing our ELK team to pick out the get search requests across the cluster and aggregate them for me. We'll see how that looks but that would just be for user query analysis and not for real time analysis. Still looking for something to monitor real time since apparently my company has all it's new relic licenses tied up with other level one processes and doesn't want to buy any more of them at this time... 😞 lol


And yes when I looked directly at the Graphite data backing Grafana at my last position it was just scary!


Thanks

Robi


PS early adapter for influxDB in general or just for this use case?

________________________________
From: Walter Underwood <[hidden email]>
Sent: Monday, November 6, 2017 1:44:01 PM
To: [hidden email]
Subject: Re: Anyone have any comments on current solr monitoring favorites?

We use New Relic across the site, but it doesn’t split out traffic to different endpoints. It also cannot distinguish between search traffic to the cluster and intra-cluster traffic. With four shards, the total traffic is 4X bigger than the incoming traffic.

We have a bunch of business metrics (orders) and other stuff that is currently in Graphite. We’ll almost certainly move all that to InfluxDB and Grafana.

The Solr metrics were overloading the Graphite database, so we’re the first service that is trying InfluxDB.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)


> On Nov 6, 2017, at 1:31 PM, Petersen, Robert (Contr) <[hidden email]> wrote:
>
> Hi Walter,
>
>
> Yes, now I see it. I'm wondering about using Grafana and New Relic at the same time since New Relic has a dashboard and also costs money for corporate use. I guess after a reread you are using Grafana to visualize the influxDB data and New Relic just for JVM right?  Did this give you more control over the solr metrics you are monitoring? (PS I've never heard of influxDB)
>
>
> Thanks
>
> Robi
>
> ________________________________
> From: Walter Underwood <[hidden email]>
> Sent: Monday, November 6, 2017 11:26:07 AM
> To: [hidden email]
> Subject: Re: Anyone have any comments on current solr monitoring favorites?
>
> Look back down the string to my post. We use Grafana.
>
> wunder
> Walter Underwood
> [hidden email]
> http://observer.wunderwood.org/  (my blog)
>
>
>> On Nov 6, 2017, at 11:23 AM, Petersen, Robert (Contr) <[hidden email]> wrote:
>>
>> Interesting! Finally a Grafana user... Thanks Daniel, I will follow your links. That looks promising.
>>
>>
>> Is anyone using Grafana over Graphite?
>>
>>
>> Thanks
>>
>> Robi
>>
>> ________________________________
>> From: Daniel Ortega <[hidden email]>
>> Sent: Monday, November 6, 2017 11:19:10 AM
>> To: [hidden email]
>> Subject: Re: Anyone have any comments on current solr monitoring favorites?
>>
>> Hi Robert,
>>
>> We use the following stack:
>>
>> - Prometheus to scrape metrics (https://prometheus.io/)
>> - Prometheus node exporter to export "machine metrics" (Disk, network
>> usage, etc.) (https://github.com/prometheus/node_exporter)
>> - Prometheus JMX exporter to export "Solr metrics" (Cache usage, QPS,
>> Response times...) (https://github.com/prometheus/jmx_exporter)
>> - Grafana to visualize all the data scrapped by Prometheus (
>> https://grafana.com/)
>>
>> Best regards
>> Daniel Ortega
>>
>> 2017-11-06 20:13 GMT+01:00 Petersen, Robert (Contr) <
>> [hidden email]>:
>>
>>> PS I knew sematext would be required to chime in here!  😊
>>>
>>>
>>> Is there a non-expiring dev version I could experiment with? I think I did
>>> sign up for a trial years ago from a different company... I was actually
>>> wondering about hooking it up to my personal AWS based solr cloud instance.
>>>
>>>
>>> Thanks
>>>
>>> Robi
>>>
>>> ________________________________
>>> From: Emir Arnautović <[hidden email]>
>>> Sent: Thursday, November 2, 2017 2:05:10 PM
>>> To: [hidden email]
>>> Subject: Re: Anyone have any comments on current solr monitoring favorites?
>>>
>>> Hi Robi,
>>> Did you try Sematext’s SPM? It provides host, JVM and Solr metrics and
>>> more. We use it for monitoring our Solr instances and for consulting.
>>>
>>> Disclaimer - see signature :)
>>>
>>> Emir
>>> --
>>> Monitoring - Log Management - Alerting - Anomaly Detection
>>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>>
>>>
>>>
>>>> On 2 Nov 2017, at 19:35, Walter Underwood <[hidden email]> wrote:
>>>>
>>>> We use New Relic for JVM, CPU, and disk monitoring.
>>>>
>>>> I tried the built-in metrics support in 6.4, but it just didn’t do what
>>> we want. We want rates and percentiles for each request handler. That gives
>>> us 95th percentile for textbooks suggest or for homework search results
>>> page, etc. The Solr metrics didn’t do that. The Jetty metrics didn’t do
>>> that.
>>>>
>>>> We built a dedicated servlet filter that goes in front of the Solr
>>> webapp and reports metrics. It has some special hacks to handle some weird
>>> behavior in SolrJ. A request to the “/srp” handler is sent as
>>> “/select?qt=/srp”, so we normalize that.
>>>>
>>>> The metrics start with the cluster name, the hostname, and the
>>> collection. The rest is generated like this:
>>>>
>>>> URL: GET /solr/textbooks/select?q=foo&qt=/auto
>>>> Metric: textbooks.GET./auto
>>>>
>>>> URL: GET /solr/textbooks/select?q=foo
>>>> Metric: textbooks.GET./select
>>>>
>>>> URL: GET /solr/questions/auto
>>>> Metric: questions.GET./auto
>>>>
>>>> So a full metric for the cluster “solr-cloud” and the host “search01"
>>> would look like “solr-cloud.search01.solr.textbooks.GET./auto.m1_rate”.
>>>>
>>>> We send all that to InfluxDB. We’ve configured a template so that each
>>> part of the metric name is mapped to a field, so we can write efficient
>>> queries in InfluxQL.
>>>>
>>>> Metrics are graphed in Grafana. We have dashboards that mix Cloudwatch
>>> (for the load balancer) and InfluxDB.
>>>>
>>>> I’m still working out the kinks in some of the more complicated queries,
>>> but the data is all there. I also want to expand the servlet filter to
>>> report HTTP response codes.
>>>>
>>>> wunder
>>>> Walter Underwood
>>>> [hidden email]
>>>> http://observer.wunderwood.org/  (my blog)
>>>>
>>>>
>>>>> On Nov 2, 2017, at 9:30 AM, Petersen, Robert (Contr) <
>>> [hidden email]> wrote:
>>>>>
>>>>> OK I'm probably going to open a can of worms here...  lol
>>>>>
>>>>>
>>>>> In the old old days I used PSI probe to monitor solr running on tomcat
>>> which worked ok on a machine by machine basis.
>>>>>
>>>>>
>>>>> Later I had a grafana dashboard on top of graphite monitoring which was
>>> really nice looking but kind of complicated to set up.
>>>>>
>>>>>
>>>>> Even later I successfully just dropped in a newrelic java agent which
>>> had solr monitors and a dashboard right out of the box, but it costs money
>>> for the full tamale.
>>>>>
>>>>>
>>>>> For basic JVM health and Solr QPS and time percentiles, does anyone
>>> have any favorites or other alternative suggestions?
>>>>>
>>>>>
>>>>> Thanks in advance!
>>>>>
>>>>> Robi
>>>>>
>>>>> ________________________________
>>>>>
>>>>> This communication is confidential. Frontier only sends and receives
>>> email on the basis of the terms set out at http://www.frontier.com/email_
>>> disclaimer.
>>>>
>>>
>>>
>>
>> ________________________________
>>
>> This communication is confidential. Frontier only sends and receives email on the basis of the terms set out at http://www.frontier.com/email_disclaimer.
>

Reply | Threaded
Open this post in threaded view
|

RE: Anyone have any comments on current solr monitoring favorites?

Davis, Daniel (NIH/NLM) [C]
I have used Java Melody for this purpose on past Java based servers, but I haven't tried to embed it in Jetty.

-----Original Message-----
From: Petersen, Robert (Contr) [mailto:[hidden email]]
Sent: Monday, November 06, 2017 4:50 PM
To: [hidden email]
Subject: Re: Anyone have any comments on current solr monitoring favorites?

Hi Walter,


OK now that sounds really interesting. I actually just turned on logging in Jetty and yes did see all the intra-cluster traffic there. I'm pushing our ELK team to pick out the get search requests across the cluster and aggregate them for me. We'll see how that looks but that would just be for user query analysis and not for real time analysis. Still looking for something to monitor real time since apparently my company has all it's new relic licenses tied up with other level one processes and doesn't want to buy any more of them at this time... 😞 lol


And yes when I looked directly at the Graphite data backing Grafana at my last position it was just scary!


Thanks

Robi


PS early adapter for influxDB in general or just for this use case?

________________________________
From: Walter Underwood <[hidden email]>
Sent: Monday, November 6, 2017 1:44:01 PM
To: [hidden email]
Subject: Re: Anyone have any comments on current solr monitoring favorites?

We use New Relic across the site, but it doesn’t split out traffic to different endpoints. It also cannot distinguish between search traffic to the cluster and intra-cluster traffic. With four shards, the total traffic is 4X bigger than the incoming traffic.

We have a bunch of business metrics (orders) and other stuff that is currently in Graphite. We’ll almost certainly move all that to InfluxDB and Grafana.

The Solr metrics were overloading the Graphite database, so we’re the first service that is trying InfluxDB.

wunder
Walter Underwood
[hidden email]
http://observer.wunderwood.org/  (my blog)


> On Nov 6, 2017, at 1:31 PM, Petersen, Robert (Contr) <[hidden email]> wrote:
>
> Hi Walter,
>
>
> Yes, now I see it. I'm wondering about using Grafana and New Relic at
> the same time since New Relic has a dashboard and also costs money for
> corporate use. I guess after a reread you are using Grafana to
> visualize the influxDB data and New Relic just for JVM right?  Did
> this give you more control over the solr metrics you are monitoring?
> (PS I've never heard of influxDB)
>
>
> Thanks
>
> Robi
>
> ________________________________
> From: Walter Underwood <[hidden email]>
> Sent: Monday, November 6, 2017 11:26:07 AM
> To: [hidden email]
> Subject: Re: Anyone have any comments on current solr monitoring favorites?
>
> Look back down the string to my post. We use Grafana.
>
> wunder
> Walter Underwood
> [hidden email]
> http://observer.wunderwood.org/  (my blog)
>
>
>> On Nov 6, 2017, at 11:23 AM, Petersen, Robert (Contr) <[hidden email]> wrote:
>>
>> Interesting! Finally a Grafana user... Thanks Daniel, I will follow your links. That looks promising.
>>
>>
>> Is anyone using Grafana over Graphite?
>>
>>
>> Thanks
>>
>> Robi
>>
>> ________________________________
>> From: Daniel Ortega <[hidden email]>
>> Sent: Monday, November 6, 2017 11:19:10 AM
>> To: [hidden email]
>> Subject: Re: Anyone have any comments on current solr monitoring favorites?
>>
>> Hi Robert,
>>
>> We use the following stack:
>>
>> - Prometheus to scrape metrics (https://prometheus.io/)
>> - Prometheus node exporter to export "machine metrics" (Disk, network
>> usage, etc.) (https://github.com/prometheus/node_exporter)
>> - Prometheus JMX exporter to export "Solr metrics" (Cache usage, QPS,
>> Response times...) (https://github.com/prometheus/jmx_exporter)
>> - Grafana to visualize all the data scrapped by Prometheus (
>> https://grafana.com/)
>>
>> Best regards
>> Daniel Ortega
>>
>> 2017-11-06 20:13 GMT+01:00 Petersen, Robert (Contr) <
>> [hidden email]>:
>>
>>> PS I knew sematext would be required to chime in here!  😊
>>>
>>>
>>> Is there a non-expiring dev version I could experiment with? I think
>>> I did sign up for a trial years ago from a different company... I
>>> was actually wondering about hooking it up to my personal AWS based solr cloud instance.
>>>
>>>
>>> Thanks
>>>
>>> Robi
>>>
>>> ________________________________
>>> From: Emir Arnautović <[hidden email]>
>>> Sent: Thursday, November 2, 2017 2:05:10 PM
>>> To: [hidden email]
>>> Subject: Re: Anyone have any comments on current solr monitoring favorites?
>>>
>>> Hi Robi,
>>> Did you try Sematext’s SPM? It provides host, JVM and Solr metrics
>>> and more. We use it for monitoring our Solr instances and for consulting.
>>>
>>> Disclaimer - see signature :)
>>>
>>> Emir
>>> --
>>> Monitoring - Log Management - Alerting - Anomaly Detection Solr &
>>> Elasticsearch Consulting Support Training - http://sematext.com/
>>>
>>>
>>>
>>>> On 2 Nov 2017, at 19:35, Walter Underwood <[hidden email]> wrote:
>>>>
>>>> We use New Relic for JVM, CPU, and disk monitoring.
>>>>
>>>> I tried the built-in metrics support in 6.4, but it just didn’t do
>>>> what
>>> we want. We want rates and percentiles for each request handler.
>>> That gives us 95th percentile for textbooks suggest or for homework
>>> search results page, etc. The Solr metrics didn’t do that. The Jetty
>>> metrics didn’t do that.
>>>>
>>>> We built a dedicated servlet filter that goes in front of the Solr
>>> webapp and reports metrics. It has some special hacks to handle some
>>> weird behavior in SolrJ. A request to the “/srp” handler is sent as
>>> “/select?qt=/srp”, so we normalize that.
>>>>
>>>> The metrics start with the cluster name, the hostname, and the
>>> collection. The rest is generated like this:
>>>>
>>>> URL: GET /solr/textbooks/select?q=foo&qt=/auto
>>>> Metric: textbooks.GET./auto
>>>>
>>>> URL: GET /solr/textbooks/select?q=foo
>>>> Metric: textbooks.GET./select
>>>>
>>>> URL: GET /solr/questions/auto
>>>> Metric: questions.GET./auto
>>>>
>>>> So a full metric for the cluster “solr-cloud” and the host “search01"
>>> would look like “solr-cloud.search01.solr.textbooks.GET./auto.m1_rate”.
>>>>
>>>> We send all that to InfluxDB. We’ve configured a template so that
>>>> each
>>> part of the metric name is mapped to a field, so we can write
>>> efficient queries in InfluxQL.
>>>>
>>>> Metrics are graphed in Grafana. We have dashboards that mix
>>>> Cloudwatch
>>> (for the load balancer) and InfluxDB.
>>>>
>>>> I’m still working out the kinks in some of the more complicated
>>>> queries,
>>> but the data is all there. I also want to expand the servlet filter
>>> to report HTTP response codes.
>>>>
>>>> wunder
>>>> Walter Underwood
>>>> [hidden email]
>>>> http://observer.wunderwood.org/  (my blog)
>>>>
>>>>
>>>>> On Nov 2, 2017, at 9:30 AM, Petersen, Robert (Contr) <
>>> [hidden email]> wrote:
>>>>>
>>>>> OK I'm probably going to open a can of worms here...  lol
>>>>>
>>>>>
>>>>> In the old old days I used PSI probe to monitor solr running on
>>>>> tomcat
>>> which worked ok on a machine by machine basis.
>>>>>
>>>>>
>>>>> Later I had a grafana dashboard on top of graphite monitoring
>>>>> which was
>>> really nice looking but kind of complicated to set up.
>>>>>
>>>>>
>>>>> Even later I successfully just dropped in a newrelic java agent
>>>>> which
>>> had solr monitors and a dashboard right out of the box, but it costs
>>> money for the full tamale.
>>>>>
>>>>>
>>>>> For basic JVM health and Solr QPS and time percentiles, does
>>>>> anyone
>>> have any favorites or other alternative suggestions?
>>>>>
>>>>>
>>>>> Thanks in advance!
>>>>>
>>>>> Robi
>>>>>
>>>>> ________________________________
>>>>>
>>>>> This communication is confidential. Frontier only sends and
>>>>> receives
>>> email on the basis of the terms set out at
>>> http://www.frontier.com/email_ disclaimer.
>>>>
>>>
>>>
>>
>> ________________________________
>>
>> This communication is confidential. Frontier only sends and receives email on the basis of the terms set out at http://www.frontier.com/email_disclaimer.
>

Reply | Threaded
Open this post in threaded view
|

Re: Anyone have any comments on current solr monitoring favorites?

Atita Arora
In reply to this post by Daniel Ortega
Hi @Daniel ,

What version of Solr are you using ?
We gave prometheus + Jolokia + InfluxDB + Grafana a try , that came out
well.
With Solr 6.6 the metrics are explosed through the /metrics api, but how do
we go about for the earlier versions , please guide ?
Specifically the cache monitoring.

Thanks in advance,
Atita

On Mon, Nov 6, 2017 at 2:19 PM, Daniel Ortega <[hidden email]>
wrote:

> Hi Robert,
>
> We use the following stack:
>
> - Prometheus to scrape metrics (https://prometheus.io/)
> - Prometheus node exporter to export "machine metrics" (Disk, network
> usage, etc.) (https://github.com/prometheus/node_exporter)
> - Prometheus JMX exporter to export "Solr metrics" (Cache usage, QPS,
> Response times...) (https://github.com/prometheus/jmx_exporter)
> - Grafana to visualize all the data scrapped by Prometheus (
> https://grafana.com/)
>
> Best regards
> Daniel Ortega
>
> 2017-11-06 20:13 GMT+01:00 Petersen, Robert (Contr) <
> [hidden email]>:
>
> > PS I knew sematext would be required to chime in here!  😊
> >
> >
> > Is there a non-expiring dev version I could experiment with? I think I
> did
> > sign up for a trial years ago from a different company... I was actually
> > wondering about hooking it up to my personal AWS based solr cloud
> instance.
> >
> >
> > Thanks
> >
> > Robi
> >
> > ________________________________
> > From: Emir Arnautović <[hidden email]>
> > Sent: Thursday, November 2, 2017 2:05:10 PM
> > To: [hidden email]
> > Subject: Re: Anyone have any comments on current solr monitoring
> favorites?
> >
> > Hi Robi,
> > Did you try Sematext’s SPM? It provides host, JVM and Solr metrics and
> > more. We use it for monitoring our Solr instances and for consulting.
> >
> > Disclaimer - see signature :)
> >
> > Emir
> > --
> > Monitoring - Log Management - Alerting - Anomaly Detection
> > Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> >
> >
> >
> > > On 2 Nov 2017, at 19:35, Walter Underwood <[hidden email]>
> wrote:
> > >
> > > We use New Relic for JVM, CPU, and disk monitoring.
> > >
> > > I tried the built-in metrics support in 6.4, but it just didn’t do what
> > we want. We want rates and percentiles for each request handler. That
> gives
> > us 95th percentile for textbooks suggest or for homework search results
> > page, etc. The Solr metrics didn’t do that. The Jetty metrics didn’t do
> > that.
> > >
> > > We built a dedicated servlet filter that goes in front of the Solr
> > webapp and reports metrics. It has some special hacks to handle some
> weird
> > behavior in SolrJ. A request to the “/srp” handler is sent as
> > “/select?qt=/srp”, so we normalize that.
> > >
> > > The metrics start with the cluster name, the hostname, and the
> > collection. The rest is generated like this:
> > >
> > > URL: GET /solr/textbooks/select?q=foo&qt=/auto
> > > Metric: textbooks.GET./auto
> > >
> > > URL: GET /solr/textbooks/select?q=foo
> > > Metric: textbooks.GET./select
> > >
> > > URL: GET /solr/questions/auto
> > > Metric: questions.GET./auto
> > >
> > > So a full metric for the cluster “solr-cloud” and the host “search01"
> > would look like “solr-cloud.search01.solr.textbooks.GET./auto.m1_rate”.
> > >
> > > We send all that to InfluxDB. We’ve configured a template so that each
> > part of the metric name is mapped to a field, so we can write efficient
> > queries in InfluxQL.
> > >
> > > Metrics are graphed in Grafana. We have dashboards that mix Cloudwatch
> > (for the load balancer) and InfluxDB.
> > >
> > > I’m still working out the kinks in some of the more complicated
> queries,
> > but the data is all there. I also want to expand the servlet filter to
> > report HTTP response codes.
> > >
> > > wunder
> > > Walter Underwood
> > > [hidden email]
> > > http://observer.wunderwood.org/  (my blog)
> > >
> > >
> > >> On Nov 2, 2017, at 9:30 AM, Petersen, Robert (Contr) <
> > [hidden email]> wrote:
> > >>
> > >> OK I'm probably going to open a can of worms here...  lol
> > >>
> > >>
> > >> In the old old days I used PSI probe to monitor solr running on tomcat
> > which worked ok on a machine by machine basis.
> > >>
> > >>
> > >> Later I had a grafana dashboard on top of graphite monitoring which
> was
> > really nice looking but kind of complicated to set up.
> > >>
> > >>
> > >> Even later I successfully just dropped in a newrelic java agent which
> > had solr monitors and a dashboard right out of the box, but it costs
> money
> > for the full tamale.
> > >>
> > >>
> > >> For basic JVM health and Solr QPS and time percentiles, does anyone
> > have any favorites or other alternative suggestions?
> > >>
> > >>
> > >> Thanks in advance!
> > >>
> > >> Robi
> > >>
> > >> ________________________________
> > >>
> > >> This communication is confidential. Frontier only sends and receives
> > email on the basis of the terms set out at
> http://www.frontier.com/email_
> > disclaimer.
> > >
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Anyone have any comments on current solr monitoring favorites?

Daniel Ortega
Hi @Atita,

We are using the latear version (Solr 7.1.0).

As the metrics are exposed with MBeans via JMX, you could use the
Prometheus JMX exportar to take the values of that metrics and expose them.
You could use it to monitor caches, response times, number of errors in all
the handlers you have defined.

To configure JMX in a Solr instance follow this link:
https://lucene.apache.org/solr/guide/6_6/using-jmx-with-solr.html

This page explains some of the JMX metrics that Solr exposes:
https://lucene.apache.org/solr/guide/6_6/performance-statistics-reference.html

Basically the JMX exporter is an Embedded Jetty server that read values
exposed using JMX (in localhost or in a remote instance), parse that values
and exposes them using the format that Prometheus could scrap.

Best regards,
Daniel

El El mar, 7 nov 2017 a las 2:43, Atita Arora <[hidden email]>
escribió:

> Hi @Daniel ,
>
> What version of Solr are you using ?
> We gave prometheus + Jolokia + InfluxDB + Grafana a try , that came out
> well.
> With Solr 6.6 the metrics are explosed through the /metrics api, but how do
> we go about for the earlier versions , please guide ?
> Specifically the cache monitoring.
>
> Thanks in advance,
> Atita
>
> On Mon, Nov 6, 2017 at 2:19 PM, Daniel Ortega <[hidden email]
> >
> wrote:
>
> > Hi Robert,
> >
> > We use the following stack:
> >
> > - Prometheus to scrape metrics (https://prometheus.io/)
> > - Prometheus node exporter to export "machine metrics" (Disk, network
> > usage, etc.) (https://github.com/prometheus/node_exporter)
> > - Prometheus JMX exporter to export "Solr metrics" (Cache usage, QPS,
> > Response times...) (https://github.com/prometheus/jmx_exporter)
> > - Grafana to visualize all the data scrapped by Prometheus (
> > https://grafana.com/)
> >
> > Best regards
> > Daniel Ortega
> >
> > 2017-11-06 20:13 GMT+01:00 Petersen, Robert (Contr) <
> > [hidden email]>:
> >
> > > PS I knew sematext would be required to chime in here!  😊
> > >
> > >
> > > Is there a non-expiring dev version I could experiment with? I think I
> > did
> > > sign up for a trial years ago from a different company... I was
> actually
> > > wondering about hooking it up to my personal AWS based solr cloud
> > instance.
> > >
> > >
> > > Thanks
> > >
> > > Robi
> > >
> > > ________________________________
> > > From: Emir Arnautović <[hidden email]>
> > > Sent: Thursday, November 2, 2017 2:05:10 PM
> > > To: [hidden email]
> > > Subject: Re: Anyone have any comments on current solr monitoring
> > favorites?
> > >
> > > Hi Robi,
> > > Did you try Sematext’s SPM? It provides host, JVM and Solr metrics and
> > > more. We use it for monitoring our Solr instances and for consulting.
> > >
> > > Disclaimer - see signature :)
> > >
> > > Emir
> > > --
> > > Monitoring - Log Management - Alerting - Anomaly Detection
> > > Solr & Elasticsearch Consulting Support Training -
> http://sematext.com/
> > >
> > >
> > >
> > > > On 2 Nov 2017, at 19:35, Walter Underwood <[hidden email]>
> > wrote:
> > > >
> > > > We use New Relic for JVM, CPU, and disk monitoring.
> > > >
> > > > I tried the built-in metrics support in 6.4, but it just didn’t do
> what
> > > we want. We want rates and percentiles for each request handler. That
> > gives
> > > us 95th percentile for textbooks suggest or for homework search results
> > > page, etc. The Solr metrics didn’t do that. The Jetty metrics didn’t do
> > > that.
> > > >
> > > > We built a dedicated servlet filter that goes in front of the Solr
> > > webapp and reports metrics. It has some special hacks to handle some
> > weird
> > > behavior in SolrJ. A request to the “/srp” handler is sent as
> > > “/select?qt=/srp”, so we normalize that.
> > > >
> > > > The metrics start with the cluster name, the hostname, and the
> > > collection. The rest is generated like this:
> > > >
> > > > URL: GET /solr/textbooks/select?q=foo&qt=/auto
> > > > Metric: textbooks.GET./auto
> > > >
> > > > URL: GET /solr/textbooks/select?q=foo
> > > > Metric: textbooks.GET./select
> > > >
> > > > URL: GET /solr/questions/auto
> > > > Metric: questions.GET./auto
> > > >
> > > > So a full metric for the cluster “solr-cloud” and the host “search01"
> > > would look like “solr-cloud.search01.solr.textbooks.GET./auto.m1_rate”.
> > > >
> > > > We send all that to InfluxDB. We’ve configured a template so that
> each
> > > part of the metric name is mapped to a field, so we can write efficient
> > > queries in InfluxQL.
> > > >
> > > > Metrics are graphed in Grafana. We have dashboards that mix
> Cloudwatch
> > > (for the load balancer) and InfluxDB.
> > > >
> > > > I’m still working out the kinks in some of the more complicated
> > queries,
> > > but the data is all there. I also want to expand the servlet filter to
> > > report HTTP response codes.
> > > >
> > > > wunder
> > > > Walter Underwood
> > > > [hidden email]
> > > > http://observer.wunderwood.org/  (my blog)
> > > >
> > > >
> > > >> On Nov 2, 2017, at 9:30 AM, Petersen, Robert (Contr) <
> > > [hidden email]> wrote:
> > > >>
> > > >> OK I'm probably going to open a can of worms here...  lol
> > > >>
> > > >>
> > > >> In the old old days I used PSI probe to monitor solr running on
> tomcat
> > > which worked ok on a machine by machine basis.
> > > >>
> > > >>
> > > >> Later I had a grafana dashboard on top of graphite monitoring which
> > was
> > > really nice looking but kind of complicated to set up.
> > > >>
> > > >>
> > > >> Even later I successfully just dropped in a newrelic java agent
> which
> > > had solr monitors and a dashboard right out of the box, but it costs
> > money
> > > for the full tamale.
> > > >>
> > > >>
> > > >> For basic JVM health and Solr QPS and time percentiles, does anyone
> > > have any favorites or other alternative suggestions?
> > > >>
> > > >>
> > > >> Thanks in advance!
> > > >>
> > > >> Robi
> > > >>
> > > >> ________________________________
> > > >>
> > > >> This communication is confidential. Frontier only sends and receives
> > > email on the basis of the terms set out at
> > http://www.frontier.com/email_
> > > disclaimer.
> > > >
> > >
> > >
> >
>