Why solr sends a request for a metrics every minute?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Why solr sends a request for a metrics every minute?

levtannen
Hello Solr community,

My solrcloud system consists of 3 machines, each running a zookeeper and a
solr server. It manages about 200 collections with 1 shard each.
When I run it, I see  that every minutes samebody sends a request for some
metrics to my system. Because nobody can sent requests to my development
system I assume that it is solr sends these requests by itself to itself.
Could anybody explain me, what is going on and how can I controll such
requests?
Bellow is an example of  log messages that are produced by these requests.  

Regards
Lev Tannen

Message example:
2019-02-04 16:39:13.487 INFO  (qtp817348612-16) [   ] o.a.s.s.HttpSolrCall
[admin] webapp=null path=/admin/metrics
params={wt=javabin&version=2&key=solr.core.OKN-A-documents.shard1.replica_n4:QUERY./select.requests&key=solr.core.CA5-  
B-documents.shard1.replica_n4:UPDATE./update.requests&key=solr.core.LAM-B-cases.shard1.replica_n4:UPDATE./update.requests&key=solr.core.NV-A-documents.shard1.replica_n2:QUERY./select.requests&key=solr.core.FLN-B-documents.shard1.repl  
ica_n1:INDEX.sizeInBytes&key=solr.core.GU-B-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.CO-B-documents.shard1.replica_n4:UPDATE./update.requests&key=solr.core.MD-A-documents.shard1.replica_n1:QUERY./select.requests&ke  
y=solr.core.INS-B-documents.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.MD-A-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.INS-A-cases.shard1.replica_n4:QUERY./select.requests&key=solr.core.ILS-A-documents.shard1.  
replica_n1:QUERY./select.requests&key=solr.core.VAW-B-cases.shard1.replica_n4:QUERY./select.requests&key=solr.core.ARW-A-cases.shard1.replica_n2:UPDATE./update.requests&key=solr.core.OKW-A-documents.shard1.replica_n1:QUERY./select.re  
quests&key=solr.core.KS-A-cases.shard1.replica_n1:QUERY./select.requests&key=solr.core.NYN-B-cases.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.MA-B-cases.shard1.replica_n4:INDEX.sizeInBytes&key=solr.core.FLM-B-cases.shard1.repl  
ica_n2:INDEX.sizeInBytes&key=solr.core.NYE-A-cases.shard1.replica_n1:QUERY./select.requests&key=solr.core.MN-A-documents.shard1.replica_n2:QUERY./select.requests&key=solr.core.NYS-B-cases.shard1.replica_n1:UPDATE./update.requests&key  
=solr.core.MIE-B-documents.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.ILS-A-cases.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.CAN-B-cases.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.ARE-A-documents.shard1.replica_n  
4:UPDATE./update.requests&key=solr.core.MOW-A-documents.shard1.replica_n5:QUERY./select.requests&key=solr.core.INS-B-cases.shard1.replica_n4:QUERY./select.requests&key=solr.core.ND-B-cases.shard1.replica_n1:UPDATE./update.requests&ke  
y=solr.core.OHN-A-documents.shard1.replica_n1:UPDATE./update.requests&key=solr.core.UT-B-documents.shard1.replica_n2:UPDATE./update.requests&key=solr.core.TNE-B-cases.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.OHS-B-cases.shar  
d1.replica_n2:UPDATE./update.requests&key=solr.core.KYW-B-documents.shard1.replica_n3:UPDATE./update.requests&key=solr.core.TXE-A-cases.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.ND-A-cases.shard1.replica_n4:UPDATE./update.req  
uests&key=solr.core.NV-A-documents.shard1.replica_n2:UPDATE./update.requests&key=solr.core.WVN-A-documents.shard1.replica_n1:QUERY./select.requests&key=solr.core.UT-B-documents.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.TXN-B-  
documents.shard1.replica_n1:UPDATE./update.requests&key=solr.core.CA4-B-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.HI-B-cases.shard1.replica_n2:QUERY./select.requests&key=solr.core.TNM-A-documents.shard1.replica_n2:U  
PDATE./update.requests&key=solr.core.CA3-B-documents.shard1.replica_n4:UPDATE./update.requests&key=solr.core.NE-B-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.NCM-B-cases.shard1.replica_n2:UPDATE./update.requests&key=s  
olr.core.CAS-B-documents.shard1.replica_n4:QUERY./select.requests&key=solr.core.GU-B-cases.shard1.replica_n4:UPDATE./update.requests&key=solr.core.CAC-A-cases.shard1.replica_n4:INDEX.sizeInBytes&key=solr.core.KS-A-documents.shard1.re  
plica_n2:INDEX.sizeInBytes&key=solr.core.LAE-A-documents.shard1.replica_n2:UPDATE./update.requests&key=solr.core.CA7-B-documents.shard1.replica_n3:INDEX.sizeInBytes&key=solr.core.PAM-A-cases.shard1.replica_n4:QUERY./select.requests&k  
ey=solr.core.HI-A-cases.shard1.replica_n2:QUERY./select.requests&key=solr.core.MIW-B-cases.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.LAE-B-documents.shard1.replica_n4:QUERY./select.requests&key=solr.core.CA10-B-documents.shar  
d1.replica_n1:INDEX.sizeInBytes&key=solr.core.CAN-B-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.FLS-A-cases.shard1.replica_n2:QUERY./select.requests&key=solr.core.NCW-A-cases.shard1.replica_n4:QUERY./select.requests&k  
ey=solr.core.VT-A-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.LAE-B-cases.shard1.replica_n4:INDEX.sizeInBytes&key=solr.core.SD-B-documents.shard1.replica_n4:INDEX.sizeInBytes&key=solr.core.GAM-A-documents.shard1.repli  
ca_n4:UPDATE./update.requests&key=solr.core.FLN-B-cases.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.FLS-B-documents.shard1.replica_n4:QUERY./select.requests&key=solr.core.VI-B-cases.shard1.replica_n1:INDEX.sizeInBytes&key=solr.  
core.VAW-A-documents.shard1.replica_n1:UPDATE./update.requests&key=solr.core.ALN-B-cases.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.RI-B-documents.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.NCE-A-cases.shard1.replica_n4  
:QUERY./select.requests&key=solr.core.VAE-B-documents.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.TXW-B-documents.shard1.replica_n4:QUERY./select.requests&key=solr.core.NV-B-cases.shard1.replica_n1:UPDATE./update.requests&key=s  
olr.core.UT-A-documents.shard1.replica_n1:QUERY./select.requests&key=solr.core.NM-B-documents.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.NE-A-documents.shard1.replica_n1:QUERY./select.requests&key=solr.core.CA9-A-documents.sha
<and so on>



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Why solr sends a request for a metrics every minute?

Jan Høydahl / Cominvent
Yes, it is the Metrics History Handler that does this to collect some key metrics in a central place.
Read more here https://lucene.apache.org/solr/guide/7_6/metrics-history.html

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

> 4. feb. 2019 kl. 18:45 skrev levtannen <[hidden email]>:
>
> Hello Solr community,
>
> My solrcloud system consists of 3 machines, each running a zookeeper and a
> solr server. It manages about 200 collections with 1 shard each.
> When I run it, I see  that every minutes samebody sends a request for some
> metrics to my system. Because nobody can sent requests to my development
> system I assume that it is solr sends these requests by itself to itself.
> Could anybody explain me, what is going on and how can I controll such
> requests?
> Bellow is an example of  log messages that are produced by these requests.  
>
> Regards
> Lev Tannen
>
> Message example:
> 2019-02-04 16:39:13.487 INFO  (qtp817348612-16) [   ] o.a.s.s.HttpSolrCall
> [admin] webapp=null path=/admin/metrics
> params={wt=javabin&version=2&key=solr.core.OKN-A-documents.shard1.replica_n4:QUERY./select.requests&key=solr.core.CA5-  
> B-documents.shard1.replica_n4:UPDATE./update.requests&key=solr.core.LAM-B-cases.shard1.replica_n4:UPDATE./update.requests&key=solr.core.NV-A-documents.shard1.replica_n2:QUERY./select.requests&key=solr.core.FLN-B-documents.shard1.repl  
> ica_n1:INDEX.sizeInBytes&key=solr.core.GU-B-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.CO-B-documents.shard1.replica_n4:UPDATE./update.requests&key=solr.core.MD-A-documents.shard1.replica_n1:QUERY./select.requests&ke  
> y=solr.core.INS-B-documents.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.MD-A-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.INS-A-cases.shard1.replica_n4:QUERY./select.requests&key=solr.core.ILS-A-documents.shard1.  
> replica_n1:QUERY./select.requests&key=solr.core.VAW-B-cases.shard1.replica_n4:QUERY./select.requests&key=solr.core.ARW-A-cases.shard1.replica_n2:UPDATE./update.requests&key=solr.core.OKW-A-documents.shard1.replica_n1:QUERY./select.re  
> quests&key=solr.core.KS-A-cases.shard1.replica_n1:QUERY./select.requests&key=solr.core.NYN-B-cases.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.MA-B-cases.shard1.replica_n4:INDEX.sizeInBytes&key=solr.core.FLM-B-cases.shard1.repl  
> ica_n2:INDEX.sizeInBytes&key=solr.core.NYE-A-cases.shard1.replica_n1:QUERY./select.requests&key=solr.core.MN-A-documents.shard1.replica_n2:QUERY./select.requests&key=solr.core.NYS-B-cases.shard1.replica_n1:UPDATE./update.requests&key  
> =solr.core.MIE-B-documents.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.ILS-A-cases.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.CAN-B-cases.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.ARE-A-documents.shard1.replica_n  
> 4:UPDATE./update.requests&key=solr.core.MOW-A-documents.shard1.replica_n5:QUERY./select.requests&key=solr.core.INS-B-cases.shard1.replica_n4:QUERY./select.requests&key=solr.core.ND-B-cases.shard1.replica_n1:UPDATE./update.requests&ke  
> y=solr.core.OHN-A-documents.shard1.replica_n1:UPDATE./update.requests&key=solr.core.UT-B-documents.shard1.replica_n2:UPDATE./update.requests&key=solr.core.TNE-B-cases.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.OHS-B-cases.shar  
> d1.replica_n2:UPDATE./update.requests&key=solr.core.KYW-B-documents.shard1.replica_n3:UPDATE./update.requests&key=solr.core.TXE-A-cases.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.ND-A-cases.shard1.replica_n4:UPDATE./update.req  
> uests&key=solr.core.NV-A-documents.shard1.replica_n2:UPDATE./update.requests&key=solr.core.WVN-A-documents.shard1.replica_n1:QUERY./select.requests&key=solr.core.UT-B-documents.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.TXN-B-  
> documents.shard1.replica_n1:UPDATE./update.requests&key=solr.core.CA4-B-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.HI-B-cases.shard1.replica_n2:QUERY./select.requests&key=solr.core.TNM-A-documents.shard1.replica_n2:U  
> PDATE./update.requests&key=solr.core.CA3-B-documents.shard1.replica_n4:UPDATE./update.requests&key=solr.core.NE-B-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.NCM-B-cases.shard1.replica_n2:UPDATE./update.requests&key=s  
> olr.core.CAS-B-documents.shard1.replica_n4:QUERY./select.requests&key=solr.core.GU-B-cases.shard1.replica_n4:UPDATE./update.requests&key=solr.core.CAC-A-cases.shard1.replica_n4:INDEX.sizeInBytes&key=solr.core.KS-A-documents.shard1.re  
> plica_n2:INDEX.sizeInBytes&key=solr.core.LAE-A-documents.shard1.replica_n2:UPDATE./update.requests&key=solr.core.CA7-B-documents.shard1.replica_n3:INDEX.sizeInBytes&key=solr.core.PAM-A-cases.shard1.replica_n4:QUERY./select.requests&k  
> ey=solr.core.HI-A-cases.shard1.replica_n2:QUERY./select.requests&key=solr.core.MIW-B-cases.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.LAE-B-documents.shard1.replica_n4:QUERY./select.requests&key=solr.core.CA10-B-documents.shar  
> d1.replica_n1:INDEX.sizeInBytes&key=solr.core.CAN-B-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.FLS-A-cases.shard1.replica_n2:QUERY./select.requests&key=solr.core.NCW-A-cases.shard1.replica_n4:QUERY./select.requests&k  
> ey=solr.core.VT-A-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.LAE-B-cases.shard1.replica_n4:INDEX.sizeInBytes&key=solr.core.SD-B-documents.shard1.replica_n4:INDEX.sizeInBytes&key=solr.core.GAM-A-documents.shard1.repli  
> ca_n4:UPDATE./update.requests&key=solr.core.FLN-B-cases.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.FLS-B-documents.shard1.replica_n4:QUERY./select.requests&key=solr.core.VI-B-cases.shard1.replica_n1:INDEX.sizeInBytes&key=solr.  
> core.VAW-A-documents.shard1.replica_n1:UPDATE./update.requests&key=solr.core.ALN-B-cases.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.RI-B-documents.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.NCE-A-cases.shard1.replica_n4  
> :QUERY./select.requests&key=solr.core.VAE-B-documents.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.TXW-B-documents.shard1.replica_n4:QUERY./select.requests&key=solr.core.NV-B-cases.shard1.replica_n1:UPDATE./update.requests&key=s  
> olr.core.UT-A-documents.shard1.replica_n1:QUERY./select.requests&key=solr.core.NM-B-documents.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.NE-A-documents.shard1.replica_n1:QUERY./select.requests&key=solr.core.CA9-A-documents.sha
> <and so on>
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Reply | Threaded
Open this post in threaded view
|

Re: Why solr sends a request for a metrics every minute?

levtannen
Thank you Jan.
Now, when I know what it is, I probably will not try to suppress the metrics
itself, but instead will suppress the log message in log4j2.xml using an
appropriate filter. This way I will have metrics in case I will figure out
how to use it and will not clog the log. I hope this will cause performance
degradation.
 



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: Why solr sends a request for a metrics every minute?

levtannen
Jan,

After I suppress the metrics messages,
I found that there are other messages. They come also once a minute, but
only on the one out of 3 computers. Could you please explain me what do
these messages mean and why they are produced by only one computer?

Best wishes.

2019-02-07 20:18:37.089 INFO  (commitScheduler-18-thread-1) [   ]
o.a.s.u.SolrIndexWriter Calling setCommitData with
IW:org.apache.solr.update.SolrIndexWriter@28a1b6e5
commitCommandVersion:1624842664242315264
2019-02-07 20:18:37.093 INFO  (commitScheduler-16-thread-1) [   ]
o.a.s.u.DirectUpdateHandler2 start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
2019-02-07 20:18:37.093 INFO  (qtp689401025-19) [c:.system s:shard1
r:core_node4 x:.system_shard1_replica_n3]
o.a.s.u.p.LogUpdateProcessorFactory [.system_shard1_replica_n3]
webapp=/solr path=/update
params={update.distrib=FROMLEADER&distrib.from=http://usahubslvcvw122.usa.doj.gov:8983/solr/.system_shard1_replica_n1/&wt=javabin&version=2}{add=[rrd|solr.collection.NJ-A-documents
(1624842664240218112)]} 0 2
2019-02-07 20:18:37.093 INFO  (commitScheduler-16-thread-1) [   ]
o.a.s.u.SolrIndexWriter Calling setCommitData with
IW:org.apache.solr.update.SolrIndexWriter@4107d3ff commitCommandVersion:0
2019-02-07 20:18:37.093 INFO  (qtp689401025-547) [c:.system s:shard1
r:core_node2 x:.system_shard1_replica_n1]
o.a.s.u.p.LogUpdateProcessorFactory [.system_shard1_replica_n1]
webapp=/solr path=/update
params={wt=javabin&version=2}{add=[rrd|solr.collection.NJ-A-documents
(1624842664240218112)]} 0 7
2019-02-07 20:18:37.096 INFO  (qtp689401025-19) [c:.system s:shard1
r:core_node4 x:.system_shard1_replica_n3]
o.a.s.u.p.LogUpdateProcessorFactory [.system_shard1_replica_n3]
webapp=/solr path=/update
params={update.distrib=FROMLEADER&distrib.from=http://usahubslvcvw122.usa.doj.gov:8983/solr/.system_shard1_replica_n1/&wt=javabin&version=2}{add=[rrd|solr.collection.OHN-B-cases
(1624842664248606720)]} 0 0
2019-02-07 20:18:37.096 INFO  (qtp689401025-576) [c:.system s:shard1
r:core_node2 x:.system_shard1_replica_n1]
o.a.s.u.p.LogUpdateProcessorFactory [.system_shard1_replica_n1]
webapp=/solr path=/update
params={wt=javabin&version=2}{add=[rrd|solr.collection.OHN-B-cases
(1624842664248606720)]} 0 2
2019-02-07 20:18:37.099 INFO  (qtp689401025-19) [c:.system s:shard1
r:core_node4 x:.system_shard1_replica_n3]
o.a.s.u.p.LogUpdateProcessorFactory [.system_shard1_replica_n3]
webapp=/solr path=/update
params={update.distrib=FROMLEADER&distrib.from=http://usahubslvcvw122.usa.doj.gov:8983/solr/.system_shard1_replica_n1/&wt=javabin&version=2}{add=[rrd|solr.collection.NM-A-documents
(1624842664251752448)]} 0 0
2019-02-07 20:18:37.099 INFO  (qtp689401025-580) [c:.system s:shard1
r:core_node2 x:.system_shard1_replica_n1]
o.a.s.u.p.LogUpdateProcessorFactory [.system_shard1_replica_n1]
webapp=/solr path=/update
params={wt=javabin&version=2}{add=[rrd|solr.collection.NM-A-documents
(1624842664251752448)]} 0 1
2019-02-07 20:18:37.101 INFO  (qtp689401025-19) [c:.system s:shard1
r:core_node4 x:.system_shard1_replica_n3]
o.a.s.u.p.LogUpdateProcessorFactory [.system_shard1_replica_n3]
webapp=/solr path=/update
params={update.distrib=FROMLEADER&distrib.from=http://usahubslvcvw122.usa.doj.gov:8983/solr/.system_shard1_replica_n1/&wt=javabin&version=2}{add=[rrd|solr.collection.KYE-B-documents
(1624842664253849600)]} 0 0
2019-02-07 20:18:37.101 INFO  (qtp689401025-21) [c:.system s:shard1
r:core_node2 x:.system_shard1_replica_n1]
o.a.s.u.p.LogUpdateProcessorFactory [.system_shard1_replica_n1]
webapp=/solr path=/update
params={wt=javabin&version=2}{add=[rrd|solr.collection.KYE-B-documents
(1624842664253849600)]} 0 1
<this is repeated for all collections>

 



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html