Dumping solr requests for indexing

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Dumping solr requests for indexing

T. Kuro Kurosaka
Is there any way to dump all incoming requests to Solr
into a file?

My customer is seeing a strange problem of disappearing
docs from index and I'd like to ask them to capture all
incoming requests.

Thanks.

-kuro
Reply | Threaded
Open this post in threaded view
|

Re: Dumping solr requests for indexing

Otis Gospodnetic-2
The solr log, as well as the servlet container log should have them all.

Otis
--
Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch



----- Original Message ----

> From: Teruhiko Kurosaka <[hidden email]>
> To: "[hidden email]" <[hidden email]>
> Sent: Fri, December 4, 2009 2:23:17 PM
> Subject: Dumping solr requests for indexing
>
> Is there any way to dump all incoming requests to Solr
> into a file?
>
> My customer is seeing a strange problem of disappearing
> docs from index and I'd like to ask them to capture all
> incoming requests.
>
> Thanks.
>
> -kuro

Reply | Threaded
Open this post in threaded view
|

RE: Dumping solr requests for indexing

T. Kuro Kurosaka
Log only tells high-level descriptions of what were done.
I'd like to capture the exact XML requests with data, so that
I could re-feed it to Solr to reproduce the issue my
customer is encountering.

-kuro  

> -----Original Message-----
> From: Otis Gospodnetic [mailto:[hidden email]]
> Sent: Friday, December 04, 2009 11:41 AM
> To: [hidden email]
> Subject: Re: Dumping solr requests for indexing
>
> The solr log, as well as the servlet container log should
> have them all.
>
> Otis
> --
> Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch
>
>
>
> ----- Original Message ----
> > From: Teruhiko Kurosaka <[hidden email]>
> > To: "[hidden email]" <[hidden email]>
> > Sent: Fri, December 4, 2009 2:23:17 PM
> > Subject: Dumping solr requests for indexing
> >
> > Is there any way to dump all incoming requests to Solr into a file?
> >
> > My customer is seeing a strange problem of disappearing docs from
> > index and I'd like to ask them to capture all incoming requests.
> >
> > Thanks.
> >
> > -kuro
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Dumping solr requests for indexing

Otis Gospodnetic-2
Aha!
Sounds like a job for a simple, custom UpdateRequestProcessor.  Actually, I think URP doesn't get access to the actual XML, but what it has access may be enough for you: http://wiki.apache.org/solr/UpdateRequestProcessor

Alternatively, unpack the war, add a custom logging servlet filter, chain it in web.xml and that might do the trick.

Otis
--
Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch



----- Original Message ----

> From: Teruhiko Kurosaka <[hidden email]>
> To: "[hidden email]" <[hidden email]>
> Sent: Fri, December 4, 2009 3:05:57 PM
> Subject: RE: Dumping solr requests for indexing
>
> Log only tells high-level descriptions of what were done.
> I'd like to capture the exact XML requests with data, so that
> I could re-feed it to Solr to reproduce the issue my
> customer is encountering.
>
> -kuro  
>
> > -----Original Message-----
> > From: Otis Gospodnetic [mailto:[hidden email]]
> > Sent: Friday, December 04, 2009 11:41 AM
> > To: [hidden email]
> > Subject: Re: Dumping solr requests for indexing
> >
> > The solr log, as well as the servlet container log should
> > have them all.
> >
> > Otis
> > --
> > Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch
> >
> >
> >
> > ----- Original Message ----
> > > From: Teruhiko Kurosaka
> > > To: "[hidden email]"
> > > Sent: Fri, December 4, 2009 2:23:17 PM
> > > Subject: Dumping solr requests for indexing
> > >
> > > Is there any way to dump all incoming requests to Solr into a file?
> > >
> > > My customer is seeing a strange problem of disappearing docs from
> > > index and I'd like to ask them to capture all incoming requests.
> > >
> > > Thanks.
> > >
> > > -kuro
> >
> >

Reply | Threaded
Open this post in threaded view
|

RE: Dumping solr requests for indexing

T. Kuro Kurosaka

> Aha!
> Sounds like a job for a simple, custom
> UpdateRequestProcessor.  Actually, I think URP doesn't get
> access to the actual XML, but what it has access may be
> enough for you: http://wiki.apache.org/solr/UpdateRequestProcessor

I added this to solrconfig.xml but I don't see any extra output
in the log file.

  <updateRequestProcessorChain>
    <processor class="solr.LogUpdateProcessorFactory" />
    <processor class="solr.RunUpdateProcessorFactory" />
  </updateRequestProcessorChain>

Do I need to do something else to make this effective?

The commented-out example in solrconfic.xml has 'name="dedupe"'
attribute. Do I have to specify a name? If so, how do I use
that name in the request?

Kuro
Reply | Threaded
Open this post in threaded view
|

Re: Dumping solr requests for indexing

Mark Miller-3
Teruhiko Kurosaka wrote:

>> Aha!
>> Sounds like a job for a simple, custom
>> UpdateRequestProcessor.  Actually, I think URP doesn't get
>> access to the actual XML, but what it has access may be
>> enough for you: http://wiki.apache.org/solr/UpdateRequestProcessor
>>    
>
> I added this to solrconfig.xml but I don't see any extra output
> in the log file.
>
>   <updateRequestProcessorChain>
>     <processor class="solr.LogUpdateProcessorFactory" />
>     <processor class="solr.RunUpdateProcessorFactory" />
>   </updateRequestProcessorChain>
>
> Do I need to do something else to make this effective?
>
> The commented-out example in solrconfic.xml has 'name="dedupe"'
> attribute. Do I have to specify a name? If so, how do I use
> that name in the request?
>
> Kuro
>  
Look at the comment above the dedupe declaration:

       You have to link the chain to an update handler above to use it ie:
         <requestHandler name="/update
"class="solr.XmlUpdateRequestHandler">
           <lst name="defaults">
             <str name="update.processor">dedupe</str>
           </lst>
         </requestHandler>

--
- Mark

http://www.lucidimagination.com