Can changes on an index be visible to an open IndexSearcher without reopening it?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Can changes on an index be visible to an open IndexSearcher without reopening it?

Enrique Lamas
Hi,
I have an application using Lucene 2.2.0 that opens an IndexSearcher only once to optimize performance, because opening the index is a heavy operation. My question is, if I modify the index with an IndexWriter or IndexModifier, is there any way for the changes to be visible to the opened IndexSearcher without reopening it?

Thanks
Reply | Threaded
Open this post in threaded view
|

Re: Can changes on an index be visible to an open IndexSearcher without reopening it?

Michael McCandless-2

Unfortunately, no.  Once open, the IndexReader/IndexSearcher searches
a frozen "point in time" snapshot of the index as it existed when it
was first opened.

You'll have to open a new searcher in order to see the changes.

However, there is work underway now to add a "reopen" method to
IndexReader that somewhat lowers the cost of opening a reader (not yet
clear by how much).  This should be part of the next release (2.3).
See here for details:

  https://issues.apache.org/jira/browse/LUCENE-743

Mike

"Enrique Lamas" <[hidden email]> wrote:
> Hi,
> I have an application using Lucene 2.2.0 that opens an IndexSearcher only
> once to optimize performance, because opening the index is a heavy
> operation. My question is, if I modify the index with an IndexWriter or
> IndexModifier, is there any way for the changes to be visible to the
> opened IndexSearcher without reopening it?
>
> Thanks

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Can changes on an index be visible to an open IndexSearcher without reopening it?

Scott-123
No plan for IndexSearcher.reopen?
I don't know about cost of creating IndexSearcher instance.
But we almost need IndexSearcher.reopen instead of IndexReader.reopen.


----
IndexReader reader = IndexReader.open(...);
IndexSearcher searcher = new IndexSearcher(reader);
searcher.search(...);

...for reopen

IndexReader newReader = reader.reopen();
if (newReader != reader) {
    searcher.close();
    reader.close();
    reader = newReader;
    newReader = null;
    searcher = new IndexSearcher(reader);
} else {
    newReader = null;
}
searcher.search(...);

...want simply...

IndexSearcher newSearcher = searcher.reopen();
if (newSearcher != searcher) {
    searcher.close();
    searcher = newSearcher;
    newSearcher = null;
}

2007/11/5, Michael McCandless <[hidden email]>:

>
> Unfortunately, no.  Once open, the IndexReader/IndexSearcher searches
> a frozen "point in time" snapshot of the index as it existed when it
> was first opened.
>
> You'll have to open a new searcher in order to see the changes.
>
> However, there is work underway now to add a "reopen" method to
> IndexReader that somewhat lowers the cost of opening a reader (not yet
> clear by how much).  This should be part of the next release (2.3).
> See here for details:
>
>   https://issues.apache.org/jira/browse/LUCENE-743
>
> Mike
>
> "Enrique Lamas" <[hidden email]> wrote:
> > Hi,
> > I have an application using Lucene 2.2.0 that opens an IndexSearcher only
> > once to optimize performance, because opening the index is a heavy
> > operation. My question is, if I modify the index with an IndexWriter or
> > IndexModifier, is there any way for the changes to be visible to the
> > opened IndexSearcher without reopening it?
> >
> > Thanks
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Can changes on an index be visible to an open IndexSearcher without reopening it?

Michael McCandless-2
This is an excellent question.

You are right, what we really need is efficient reopening of an
IndexSearcher.  Creating & warming a new IndexSearcher can be
expensive due [at least] to populating the FieldCache.

This has been discussed before, eg here:

     http://www.gossamer-threads.com/lists/lucene/java-dev/53852#53852

And there is at least one Jira issue opened, here:

     https://issues.apache.org/jira/browse/LUCENE-831

But this is clearly still a "work in progress".  Stay tuned (on
java-dev)...

Mike

Scott Tiger wrote:

> No plan for IndexSearcher.reopen?
> I don't know about cost of creating IndexSearcher instance.
> But we almost need IndexSearcher.reopen instead of IndexReader.reopen.
>
>
> ----
> IndexReader reader = IndexReader.open(...);
> IndexSearcher searcher = new IndexSearcher(reader);
> searcher.search(...);
>
> ...for reopen
>
> IndexReader newReader = reader.reopen();
> if (newReader != reader) {
>     searcher.close();
>     reader.close();
>     reader = newReader;
>     newReader = null;
>     searcher = new IndexSearcher(reader);
> } else {
>     newReader = null;
> }
> searcher.search(...);
>
> ...want simply...
>
> IndexSearcher newSearcher = searcher.reopen();
> if (newSearcher != searcher) {
>     searcher.close();
>     searcher = newSearcher;
>     newSearcher = null;
> }
>
> 2007/11/5, Michael McCandless <[hidden email]>:
>>
>> Unfortunately, no.  Once open, the IndexReader/IndexSearcher searches
>> a frozen "point in time" snapshot of the index as it existed when it
>> was first opened.
>>
>> You'll have to open a new searcher in order to see the changes.
>>
>> However, there is work underway now to add a "reopen" method to
>> IndexReader that somewhat lowers the cost of opening a reader (not  
>> yet
>> clear by how much).  This should be part of the next release (2.3).
>> See here for details:
>>
>>   https://issues.apache.org/jira/browse/LUCENE-743
>>
>> Mike
>>
>> "Enrique Lamas" <[hidden email]> wrote:
>>> Hi,
>>> I have an application using Lucene 2.2.0 that opens an  
>>> IndexSearcher only
>>> once to optimize performance, because opening the index is a heavy
>>> operation. My question is, if I modify the index with an  
>>> IndexWriter or
>>> IndexModifier, is there any way for the changes to be visible to the
>>> opened IndexSearcher without reopening it?
>>>
>>> Thanks
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]