possible to disable internal caching?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

possible to disable internal caching?

jmuguruza
Hi,

That last thread about caching reminded me of something. Me need is
actually the opposite...

I use lucene to search in hundreds/thousands of indexes. Doing a
lucene query on a set of the indexes is only one of the steps involved
in my 'queries', and some of the other steps take longer than lucene.
Also, my queries are offline, so I can live with the query lasting 20
seconds or 10 minutes.

So my question, is it possible to disable some of the caching lucene
does so the memory consumption will be smaller (I am a bit concerned
on the memory usage side)? Or the memory savings would not pay off?

thanks,
javi

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: possible to disable internal caching?

Karl Wettin

14 feb 2007 kl. 17.12 skrev jm:
>
> So my question, is it possible to disable some of the caching lucene
> does so the memory consumption will be smaller (I am a bit concerned
> on the memory usage side)? Or the memory savings would not pay off?

You could try to create a new Searcher for each query, and dispose of
the instance when you have accessed the results.

--
karl

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: possible to disable internal caching?

Daniel Naber-5
In reply to this post by jmuguruza
On Wednesday 14 February 2007 17:12, jm wrote:

> So my question, is it possible to disable some of the caching lucene
> does so the memory consumption will be smaller (I am a bit concerned
> on the memory usage side)? Or the memory savings would not pay off?

You could set IndexWriter.setTermIndexInterval() to a value larger than the
default (128). Thus Lucene will use less memory for its term list index in
memory.

Regards
 Daniel

--
http://www.danielnaber.de

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: possible to disable internal caching?

jmuguruza
Thanks Karl and Daniel

I am already disponing of the Searchers I am using. And regarding
IndexWriter.setTermIndexInterval(), I need the indexing to be as fast
as possible, is the searches where I dont need any speed and prefer to
keep the memory low.

javier

On 2/14/07, Daniel Naber <[hidden email]> wrote:

> On Wednesday 14 February 2007 17:12, jm wrote:
>
> > So my question, is it possible to disable some of the caching lucene
> > does so the memory consumption will be smaller (I am a bit concerned
> > on the memory usage side)? Or the memory savings would not pay off?
>
> You could set IndexWriter.setTermIndexInterval() to a value larger than the
> default (128). Thus Lucene will use less memory for its term list index in
> memory.
>
> Regards
>  Daniel
>
> --
> http://www.danielnaber.de
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]