Analysing SOLR logfiles

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Analysing SOLR logfiles

Jay Flattery
Hi there,


Just wondering what tools people use to analyse SOLR log files.

We're looking to do things like extracting common queries, calculating averaging

Qtime and hits, returning particularly slow/expensive queries, etc.

Would prefer not to code something (completely) from scratch.

Thanks!


     

Reply | Threaded
Open this post in threaded view
|

Re: Analysing SOLR logfiles

Jan Høydahl / Cominvent
Have a look at www.splunk.com

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
Training in Europe - www.solrtraining.com

On 11. aug. 2010, at 19.34, Jay Flattery wrote:

> Hi there,
>
>
> Just wondering what tools people use to analyse SOLR log files.
>
> We're looking to do things like extracting common queries, calculating averaging
>
> Qtime and hits, returning particularly slow/expensive queries, etc.
>
> Would prefer not to code something (completely) from scratch.
>
> Thanks!
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Analysing SOLR logfiles

Jay Flattery
Thanks - splunk looks overkill.
We're extremely small scale - were hoping for something open source :-)


----- Original Message ----
From: Jan Høydahl / Cominvent <[hidden email]>
To: [hidden email]
Sent: Wed, August 11, 2010 11:14:37 PM
Subject: Re: Analysing SOLR logfiles

Have a look at www.splunk.com

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
Training in Europe - www.solrtraining.com

On 11. aug. 2010, at 19.34, Jay Flattery wrote:

> Hi there,
>
>
> Just wondering what tools people use to analyse SOLR log files.
>
> We're looking to do things like extracting common queries, calculating
>averaging
>
>
> Qtime and hits, returning particularly slow/expensive queries, etc.
>
> Would prefer not to code something (completely) from scratch.
>
> Thanks!
>
>
>
>




Reply | Threaded
Open this post in threaded view
|

Re: Analysing SOLR logfiles

Rebecca Watson
we've just started using awstats - as suggested by the solr 1.4 book.

its open source!:
http://awstats.sourceforge.net/

On 12 August 2010 18:18, Jay Flattery <[hidden email]> wrote:

> Thanks - splunk looks overkill.
> We're extremely small scale - were hoping for something open source :-)
>
>
> ----- Original Message ----
> From: Jan Høydahl / Cominvent <[hidden email]>
> To: [hidden email]
> Sent: Wed, August 11, 2010 11:14:37 PM
> Subject: Re: Analysing SOLR logfiles
>
> Have a look at www.splunk.com
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
> Training in Europe - www.solrtraining.com
>
> On 11. aug. 2010, at 19.34, Jay Flattery wrote:
>
>> Hi there,
>>
>>
>> Just wondering what tools people use to analyse SOLR log files.
>>
>> We're looking to do things like extracting common queries, calculating
>>averaging
>>
>>
>> Qtime and hits, returning particularly slow/expensive queries, etc.
>>
>> Would prefer not to code something (completely) from scratch.
>>
>> Thanks!
>>
>>
>>
>>
>
>
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Analysing SOLR logfiles

Peter

I wonder too, that there shouldn't be a special tool which analyzes solr
logfiles (e.g. parses qtime, the parameters q, fq, ...)

Because there are some other open source log analyzers out there:
http://yaala.org/ http://www.mrunix.net/webalizer/

Another free tool is newrelic.com (you will submit your query data to
this site, same as for google analytics). Setup is easy.

For traffic on our site which triggers the solr search we use piwik and
common queries can be extracted easily. Setup was done in 5 minutes.

Regards,
Peter.

> we've just started using awstats - as suggested by the solr 1.4 book.
>
> its open source!:
> http://awstats.sourceforge.net/
>
> On 12 August 2010 18:18, Jay Flattery <[hidden email]> wrote:
>  
>> Thanks - splunk looks overkill.
>> We're extremely small scale - were hoping for something open source :-)
>>
>>
>> ----- Original Message ----
>> From: Jan Høydahl / Cominvent <[hidden email]>
>> To: [hidden email]
>> Sent: Wed, August 11, 2010 11:14:37 PM
>> Subject: Re: Analysing SOLR logfiles
>>
>> Have a look at www.splunk.com
>>
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>> Training in Europe - www.solrtraining.com
>>
>> On 11. aug. 2010, at 19.34, Jay Flattery wrote:
>>
>>    
>>> Hi there,
>>>
>>>
>>> Just wondering what tools people use to analyse SOLR log files.
>>>
>>> We're looking to do things like extracting common queries, calculating
>>> averaging
>>>
>>>
>>> Qtime and hits, returning particularly slow/expensive queries, etc.
>>>
>>> Would prefer not to code something (completely) from scratch.
>>>
>>> Thanks!
>>>
>>>
>>>