How to open IndexWriter to append document?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

How to open IndexWriter to append document?

David-317
Hi all:
    I want first erase the original index and then create an index for
appending, I use the following python code using ports pyLucene.

   def store(doc)
        store = PyLucene.FSDirectory.getDirectory("index", True)
        writer = PyLucene.IndexWriter(store, StandardAnalyzer, False ) #
error occurs in this line!
        writer.addDocument(doc);
        writer.optimize()
        writer.close()
   if __name__ == '__main__':
        store(doc1)
        store(doc2)

    But I get the following errors:

 File "D:\work\contentstore\contentstore\backends\lucene\base.py", line 30,
in store
    writer = PyLucene.IndexWriter(store, self.analyzer, False )
PyLucene.JavaError: java.io.FileNotFoundException:
D:\work\contentstore\contentstore\backends\lucene\index\segments: The system
cannot find the file specified.

    How to fix it?
Appreciate your help!

--
David
Reply | Threaded
Open this post in threaded view
|

Re: How to open IndexWriter to append document?

Doron Cohen
You should instantiate the writer with create=true :

      IndexWriter(store, StandardAnalyzer, True )

Because with create=False, an existing index is expected.

David <[hidden email]> wrote on 14/01/2007 23:56:23:

> Hi all:
>     I want first erase the original index and then create an index for
> appending, I use the following python code using ports pyLucene.
>
>    def store(doc)
>         store = PyLucene.FSDirectory.getDirectory("index", True)
>         writer = PyLucene.IndexWriter(store, StandardAnalyzer, False ) #
> error occurs in this line!
>         writer.addDocument(doc);
>         writer.optimize()
>         writer.close()
>    if __name__ == '__main__':
>         store(doc1)
>         store(doc2)
>
>     But I get the following errors:
>
>  File "D:\work\contentstore\contentstore\backends\lucene\base.py", line
30,
> in store
>     writer = PyLucene.IndexWriter(store, self.analyzer, False )
> PyLucene.JavaError: java.io.FileNotFoundException:
> D:\work\contentstore\contentstore\backends\lucene\index\segments: The
system
> cannot find the file specified.
>
>     How to fix it?
> Appreciate your help!
>
> --
> David


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to open IndexWriter to append document?

David-317
Thanks, I think I did not describe my problem exactly.
What I want to do is:
     if the index file exist, append document
     if the index file does not exist, create a new , empty index file.
How can I implement that?
Appreciate your help!


2007/1/15, Doron Cohen <[hidden email]>:

>
> You should instantiate the writer with create=true :
>
>       IndexWriter(store, StandardAnalyzer, True )
>
> Because with create=False, an existing index is expected.
>
> David <[hidden email]> wrote on 14/01/2007 23:56:23:
>
> > Hi all:
> >     I want first erase the original index and then create an index for
> > appending, I use the following python code using ports pyLucene.
> >
> >    def store(doc)
> >         store = PyLucene.FSDirectory.getDirectory("index", True)
> >         writer = PyLucene.IndexWriter(store, StandardAnalyzer, False ) #
> > error occurs in this line!
> >         writer.addDocument(doc);
> >         writer.optimize()
> >         writer.close()
> >    if __name__ == '__main__':
> >         store(doc1)
> >         store(doc2)
> >
> >     But I get the following errors:
> >
> >  File "D:\work\contentstore\contentstore\backends\lucene\base.py", line
> 30,
> > in store
> >     writer = PyLucene.IndexWriter(store, self.analyzer, False )
> > PyLucene.JavaError: java.io.FileNotFoundException:
> > D:\work\contentstore\contentstore\backends\lucene\index\segments: The
> system
> > cannot find the file specified.
> >
> >     How to fix it?
> > Appreciate your help!
> >
> > --
> > David
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>


--
David
Reply | Threaded
Open this post in threaded view
|

Re: How to open IndexWriter to append document?

Doron Cohen
David <[hidden email]> wrote on 15/01/2007 00:36:28:

> Thanks, I think I did not describe my problem exactly.
> What I want to do is:
>      if the index file exist, append document
>      if the index file does not exist, create a new , empty index file.
> How can I implement that?

If this is what you need, this (Lucene Java) conostructor would do:
   IndexWriter(File path, Analyzer a)
          Constructs an IndexWriter for the index in path, creating it
first if it does not already exist, otherwise appending to the existing
index.

Your current code really cannot work, becuase for FSDirectory.get(),
create=true, which erases everything first, and for IndexWriter(),
create=false, which expects an existing index.

> Appreciate your help!
>
>
> 2007/1/15, Doron Cohen <[hidden email]>:
> >
> > You should instantiate the writer with create=true :
> >
> >       IndexWriter(store, StandardAnalyzer, True )
> >
> > Because with create=False, an existing index is expected.
> >
> > David <[hidden email]> wrote on 14/01/2007 23:56:23:
> >
> > > Hi all:
> > >     I want first erase the original index and then create an index
for
> > > appending, I use the following python code using ports pyLucene.
> > >
> > >    def store(doc)
> > >         store = PyLucene.FSDirectory.getDirectory("index", True)
> > >         writer = PyLucene.IndexWriter(store, StandardAnalyzer, False
) #

> > > error occurs in this line!
> > >         writer.addDocument(doc);
> > >         writer.optimize()
> > >         writer.close()
> > >    if __name__ == '__main__':
> > >         store(doc1)
> > >         store(doc2)
> > >
> > >     But I get the following errors:
> > >
> > >  File "D:\work\contentstore\contentstore\backends\lucene\base.py",
line

> > 30,
> > > in store
> > >     writer = PyLucene.IndexWriter(store, self.analyzer, False )
> > > PyLucene.JavaError: java.io.FileNotFoundException:
> > > D:\work\contentstore\contentstore\backends\lucene\index\segments: The
> > system
> > > cannot find the file specified.
> > >
> > >     How to fix it?
> > > Appreciate your help!
> > >
> > > --
> > > David
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
> >
>
>
> --
> David


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to open IndexWriter to append document?

Bhavin Pandya
In reply to this post by David-317
Hi david,

> What I want to do is:
>     if the index file exist, append document
>     if the index file does not exist, create a new , empty index file.

Please check Lucene api for IndexReader...

It has one method which u can use before opening
IndexWriter..."indexExists(Directory directory)"

- Bhavin pandya


----- Original Message -----
From: "David" <[hidden email]>
To: <[hidden email]>
Sent: Monday, January 15, 2007 2:06 PM
Subject: Re: How to open IndexWriter to append document?


> Thanks, I think I did not describe my problem exactly.
> What I want to do is:
>     if the index file exist, append document
>     if the index file does not exist, create a new , empty index file.
> How can I implement that?
> Appreciate your help!
>
>
> 2007/1/15, Doron Cohen <[hidden email]>:
>>
>> You should instantiate the writer with create=true :
>>
>>       IndexWriter(store, StandardAnalyzer, True )
>>
>> Because with create=False, an existing index is expected.
>>
>> David <[hidden email]> wrote on 14/01/2007 23:56:23:
>>
>> > Hi all:
>> >     I want first erase the original index and then create an index for
>> > appending, I use the following python code using ports pyLucene.
>> >
>> >    def store(doc)
>> >         store = PyLucene.FSDirectory.getDirectory("index", True)
>> >         writer = PyLucene.IndexWriter(store, StandardAnalyzer, False )
>> > #
>> > error occurs in this line!
>> >         writer.addDocument(doc);
>> >         writer.optimize()
>> >         writer.close()
>> >    if __name__ == '__main__':
>> >         store(doc1)
>> >         store(doc2)
>> >
>> >     But I get the following errors:
>> >
>> >  File "D:\work\contentstore\contentstore\backends\lucene\base.py", line
>> 30,
>> > in store
>> >     writer = PyLucene.IndexWriter(store, self.analyzer, False )
>> > PyLucene.JavaError: java.io.FileNotFoundException:
>> > D:\work\contentstore\contentstore\backends\lucene\index\segments: The
>> system
>> > cannot find the file specified.
>> >
>> >     How to fix it?
>> > Appreciate your help!
>> >
>> > --
>> > David
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>
>
> --
> David
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to open IndexWriter to append document?

Michael McCandless-2
Bhavin Pandya wrote:
>> What I want to do is:
>>     if the index file exist, append document
>>     if the index file does not exist, create a new , empty index file.
>
> Please check Lucene api for IndexReader...
>
> It has one method which u can use before opening
> IndexWriter..."indexExists(Directory directory)"

Also note that in the unreleased version of Lucene there
are now constructors for IndexWriter (one for String, File
and Directory) that do this exact logic for you.  This was
the fix for JIRA issue LUCENE-301:

     http://issues.apache.org/jira/browse/LUCENE-301

So the next release of Lucene will have this functionality.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to open IndexWriter to append document?

Michael McCandless-2
In reply to this post by Doron Cohen
Doron Cohen wrote:

> David <[hidden email]> wrote on 15/01/2007 00:36:28:
>
>> Thanks, I think I did not describe my problem exactly.
>> What I want to do is:
>>      if the index file exist, append document
>>      if the index file does not exist, create a new , empty index file.
>> How can I implement that?
>
> If this is what you need, this (Lucene Java) conostructor would do:
>    IndexWriter(File path, Analyzer a)
>           Constructs an IndexWriter for the index in path, creating it
> first if it does not already exist, otherwise appending to the existing
> index.

But please note these are not yet part of any release (ie, only
committed to the "trunk" right now).

> Your current code really cannot work, becuase for FSDirectory.get(),
> create=true, which erases everything first, and for IndexWriter(),
> create=false, which expects an existing index.

See also JIRA issue LUCENE-773:

     http://issues.apache.org/jira/browse/LUCENE-773

The fact that there are two "create" options (one at the FSDirectory
level and one at the IndexWriter level) has been a source of confusion
/ mistakes / bugs in the past.  So the plan is to deprecate
FSDirectory's create and only use IndexWriter's create going forward.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to open IndexWriter to append document?

Doron Cohen
Michael McCandless <[hidden email]> wrote on 15/01/2007
03:10:03:

> Doron Cohen wrote:
> > David <[hidden email]> wrote on 15/01/2007 00:36:28:
> >
> >> Thanks, I think I did not describe my problem exactly.
> >> What I want to do is:
> >>      if the index file exist, append document
> >>      if the index file does not exist, create a new , empty index
file.

> >> How can I implement that?
> >
> > If this is what you need, this (Lucene Java) conostructor would do:
> >    IndexWriter(File path, Analyzer a)
> >           Constructs an IndexWriter for the index in path, creating it
> > first if it does not already exist, otherwise appending to the existing
> > index.
>
> But please note these are not yet part of any release (ie, only
> committed to the "trunk" right now).

Yes, I was referring to the most recent code here.

>
> > Your current code really cannot work, becuase for FSDirectory.get(),
> > create=true, which erases everything first, and for IndexWriter(),
> > create=false, which expects an existing index.
>
> See also JIRA issue LUCENE-773:
>
>      http://issues.apache.org/jira/browse/LUCENE-773
>
> The fact that there are two "create" options (one at the FSDirectory
> level and one at the IndexWriter level) has been a source of confusion
> / mistakes / bugs in the past.  So the plan is to deprecate
> FSDirectory's create and only use IndexWriter's create going forward.
>
> Mike
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]