Question bout I/O monitoring

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Question bout I/O monitoring

edwardyf
Is there any means to monitor the I/O of lucene?
i.e. say i am searching in a FSDirectory, i wanna know the number of pages read from disk
Reply | Threaded
Open this post in threaded view
|

Re: Question bout I/O monitoring

Ted Dunning
Try this:

http://lucene.grantingersoll.com/2009/08/25/lucid-imagination-%C2%BB-lucid-gaze-for-lucene/

On Thu, Sep 10, 2009 at 8:39 PM, edwardyf <[hidden email]> wrote:

Is there any means to monitor the I/O of lucene?
i.e. say i am searching in a FSDirectory, i wanna know the number of pages
read from disk

Reply | Threaded
Open this post in threaded view
|

Re: Question bout I/O monitoring

edwardyf
Thanks for the reply, just checked the Lucid Gaze package, it only collects the stats at
function call level, no I/O stats


Ted Dunning wrote
Try this:

http://lucene.grantingersoll.com/2009/08/25/lucid-imagination-%C2%BB-lucid-gaze-for-lucene/

On Thu, Sep 10, 2009 at 8:39 PM, edwardyf <edwardyf@gmail.com> wrote:

>
> Is there any means to monitor the I/O of lucene?
> i.e. say i am searching in a FSDirectory, i wanna know the number of pages
> read from disk
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Question bout I/O monitoring

Brian Pinkerton-2-2
If you're on a Mac or Solaris, dtrace will tell you everything you  
want to know (and more.)  If you're not familiar with dtrace,  
iosnoop.d is a good start for this kind of measurement.

At Technorati, I used dtrace to build a trace file of all the read  
requests made by a big lucene app, recording timestamp, file, offset,  
and number of bytes read.  Among other things, the measurement led me  
to decrease the heap size of the JVM so that more of the index would  
fit in the file cache.

bri

On Sep 10, 2009, at 9:39 PM, edwardyf wrote:

>
> Thanks for the reply, just checked the Lucid Gaze package, it only  
> collects
> the stats at
> function call level, no I/O stats


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Question bout I/O monitoring

edwardyf
unfortunately i am running a Red hat enterprise version, i am doing an
academic experiment which need number of disk I/Os.

just looked at SystemTap, which is said to be the iosnoop for linux.
but my kernel is too old for it....(2.4.21-47.0.1.EL)


Brian Pinkerton-2 wrote
If you're on a Mac or Solaris, dtrace will tell you everything you  
want to know (and more.)  If you're not familiar with dtrace,  
iosnoop.d is a good start for this kind of measurement.

At Technorati, I used dtrace to build a trace file of all the read  
requests made by a big lucene app, recording timestamp, file, offset,  
and number of bytes read.  Among other things, the measurement led me  
to decrease the heap size of the JVM so that more of the index would  
fit in the file cache.

bri

On Sep 10, 2009, at 9:39 PM, edwardyf wrote:

>
> Thanks for the reply, just checked the Lucid Gaze package, it only  
> collects
> the stats at
> function call level, no I/O stats


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Reply | Threaded
Open this post in threaded view
|

Re: Question bout I/O monitoring

Ted Dunning

You can also integrate the results from iostat on an otherwise idle machine.

On Thu, Sep 10, 2009 at 10:24 PM, edwardyf <[hidden email]> wrote:

unfortunately i am running a Red hat enterprise version, i am doing an
academic experiment which need number of disk I/Os.

I am now looking at SystemTap, which is said to be the iosnoop for linux.
thanks for the guide



Brian Pinkerton-2 wrote:
>
> If you're on a Mac or Solaris, dtrace will tell you everything you
> want to know (and more.)  If you're not familiar with dtrace,
> iosnoop.d is a good start for this kind of measurement.
>
> At Technorati, I used dtrace to build a trace file of all the read
> requests made by a big lucene app, recording timestamp, file, offset,
> and number of bytes read.  Among other things, the measurement led me
> to decrease the heap size of the JVM so that more of the index would
> fit in the file cache.
>
> bri
>
> On Sep 10, 2009, at 9:39 PM, edwardyf wrote:
>
>>
>> Thanks for the reply, just checked the Lucid Gaze package, it only
>> collects
>> the stats at
>> function call level, no I/O stats
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
>

--
View this message in context: http://www.nabble.com/Question-bout-I-O-monitoring-tp25394566p25395172.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]




--
Ted Dunning, CTO
DeepDyve