solr cloud updatehandler stats mismatch

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

solr cloud updatehandler stats mismatch

weiwang19
Hi,

I use the following api to track the number of update requests:

/solr/collection1/admin/mbeans?cat=UPDATE&stats=true&wt=json


Result:


   - class: "org.apache.solr.handler.UpdateRequestHandler",
   - version: "6.4.2.1",
   - description: "Add documents using XML (with XSLT), CSV, JSON, or
   javabin",
   - src: null,
   - stats:
   {
      - handlerStart: 1509824945436,
      - requests: 106062,
      - ...


I am quite confused that the number of requests reported above is quite
different from the count from solr access logs. A few times the handler
stats is much higher: handler reports ~100k requests but in the access log
there are only 5k update requests. What could be the possible cause?

Thanks,
Wei
Reply | Threaded
Open this post in threaded view
|

Re: solr cloud updatehandler stats mismatch

kamaci
Hi Wei,

Do you compare it with files which are under /var/solr/logs by default?

Kind Regards,
Furkan KAMACI

On Sun, Nov 5, 2017 at 6:59 PM, Wei <[hidden email]> wrote:

> Hi,
>
> I use the following api to track the number of update requests:
>
> /solr/collection1/admin/mbeans?cat=UPDATE&stats=true&wt=json
>
>
> Result:
>
>
>    - class: "org.apache.solr.handler.UpdateRequestHandler",
>    - version: "6.4.2.1",
>    - description: "Add documents using XML (with XSLT), CSV, JSON, or
>    javabin",
>    - src: null,
>    - stats:
>    {
>       - handlerStart: 1509824945436,
>       - requests: 106062,
>       - ...
>
>
> I am quite confused that the number of requests reported above is quite
> different from the count from solr access logs. A few times the handler
> stats is much higher: handler reports ~100k requests but in the access log
> there are only 5k update requests. What could be the possible cause?
>
> Thanks,
> Wei
>
Reply | Threaded
Open this post in threaded view
|

Re: solr cloud updatehandler stats mismatch

Amrit Sarkar
Wei,

Are the requests coming through to collection has multiple shards and
replicas. Please mind a update request is received by a node, redirected to
particular shard the doc belong, and then distributed to replicas of the
collection. On each replica, each core, update request is played.

Can be a probable reason b/w mismatch between Mbeans stats and manual
counting in logs, as not everything gets logged. Need to check that once.

Amrit Sarkar
Search Engineer
Lucidworks, Inc.
415-589-9269
www.lucidworks.com
Twitter http://twitter.com/lucidworks
LinkedIn: https://www.linkedin.com/in/sarkaramrit2
Medium: https://medium.com/@sarkaramrit2

On Thu, Nov 9, 2017 at 4:34 PM, Furkan KAMACI <[hidden email]>
wrote:

> Hi Wei,
>
> Do you compare it with files which are under /var/solr/logs by default?
>
> Kind Regards,
> Furkan KAMACI
>
> On Sun, Nov 5, 2017 at 6:59 PM, Wei <[hidden email]> wrote:
>
> > Hi,
> >
> > I use the following api to track the number of update requests:
> >
> > /solr/collection1/admin/mbeans?cat=UPDATE&stats=true&wt=json
> >
> >
> > Result:
> >
> >
> >    - class: "org.apache.solr.handler.UpdateRequestHandler",
> >    - version: "6.4.2.1",
> >    - description: "Add documents using XML (with XSLT), CSV, JSON, or
> >    javabin",
> >    - src: null,
> >    - stats:
> >    {
> >       - handlerStart: 1509824945436,
> >       - requests: 106062,
> >       - ...
> >
> >
> > I am quite confused that the number of requests reported above is quite
> > different from the count from solr access logs. A few times the handler
> > stats is much higher: handler reports ~100k requests but in the access
> log
> > there are only 5k update requests. What could be the possible cause?
> >
> > Thanks,
> > Wei
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: solr cloud updatehandler stats mismatch

weiwang19
Thanks Amrit. Can you explain a bit more what kind of requests won't be
logged?  Is that something configurable for solr?

Best,
Wei

On Thu, Nov 9, 2017 at 3:12 AM, Amrit Sarkar <[hidden email]> wrote:

> Wei,
>
> Are the requests coming through to collection has multiple shards and
> replicas. Please mind a update request is received by a node, redirected to
> particular shard the doc belong, and then distributed to replicas of the
> collection. On each replica, each core, update request is played.
>
> Can be a probable reason b/w mismatch between Mbeans stats and manual
> counting in logs, as not everything gets logged. Need to check that once.
>
> Amrit Sarkar
> Search Engineer
> Lucidworks, Inc.
> 415-589-9269
> www.lucidworks.com
> Twitter http://twitter.com/lucidworks
> LinkedIn: https://www.linkedin.com/in/sarkaramrit2
> Medium: https://medium.com/@sarkaramrit2
>
> On Thu, Nov 9, 2017 at 4:34 PM, Furkan KAMACI <[hidden email]>
> wrote:
>
> > Hi Wei,
> >
> > Do you compare it with files which are under /var/solr/logs by default?
> >
> > Kind Regards,
> > Furkan KAMACI
> >
> > On Sun, Nov 5, 2017 at 6:59 PM, Wei <[hidden email]> wrote:
> >
> > > Hi,
> > >
> > > I use the following api to track the number of update requests:
> > >
> > > /solr/collection1/admin/mbeans?cat=UPDATE&stats=true&wt=json
> > >
> > >
> > > Result:
> > >
> > >
> > >    - class: "org.apache.solr.handler.UpdateRequestHandler",
> > >    - version: "6.4.2.1",
> > >    - description: "Add documents using XML (with XSLT), CSV, JSON, or
> > >    javabin",
> > >    - src: null,
> > >    - stats:
> > >    {
> > >       - handlerStart: 1509824945436,
> > >       - requests: 106062,
> > >       - ...
> > >
> > >
> > > I am quite confused that the number of requests reported above is quite
> > > different from the count from solr access logs. A few times the handler
> > > stats is much higher: handler reports ~100k requests but in the access
> > log
> > > there are only 5k update requests. What could be the possible cause?
> > >
> > > Thanks,
> > > Wei
> > >
> >
>