IOException - The handle is invalid

classic Classic list List threaded Threaded
4 messages Options
adb
Reply | Threaded
Open this post in threaded view
|

IOException - The handle is invalid

adb
Hi,

I'm running load tests with Lucene 2.0, SUN's JDK 6 on Windows XP2, dual core
CPU.  I have 8 worker threads adding a few hundred K documents, split between
two Lucene indexes, I've started getting

java.io.IOException: The handle is invalid in places like

java.io.RandomAccessFile.writeBytes(Native Method)
java.io.RandomAccessFile.write(RandomAccessFile.java:466)
org.apache.lucene.store.FSIndexOutput.flushBuffer(FSDirectory.java:503)
org.apache.lucene.store.BufferedIndexOutput.flush(BufferedIndexOutput.java:84)
org.apache.lucene.store.BufferedIndexOutput.close(BufferedIndexOutput.java:98)
org.apache.lucene.store.FSIndexOutput.close(FSDirectory.java:506)
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:415)
org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:90)
org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:709)
org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:686)
org.apache.lucene.index.IndexWriter.maybeMergeSegments(IndexWriter.java:674)
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:479)
com.teamware.office.lucene.index.TWOIndexWriter.addDocument(TWOIndexWriter.java:170)

java.io.RandomAccessFile.close0(Native Method)
java.io.RandomAccessFile.close(RandomAccessFile.java:543)
org.apache.lucene.store.FSIndexInput.close(FSDirectory.java:465)
org.apache.lucene.index.CompoundFileWriter.copyFile(CompoundFileWriter.java:231)
org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java:168)
org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:145)
org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:735)
org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:686)
org.apache.lucene.index.IndexWriter.maybeMergeSegments(IndexWriter.java:674)
org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:479)
com.teamware.office.lucene.index.TWOIndexWriter.addDocument(TWOIndexWriter.java:170)

I saw Mike McCandless JIRA issue

http://issues.apache.org/jira/browse/LUCENE-669

Is the patch referenced there useful for a 2.0 system.  I would like to use the
lockless commit stuff, but am waiting until I get the core system working well.

I am also getting IOException in some of my classes, but from the JIRA comments,
it seems that Lucene may be the culprit.

No urgency, I won't get a chance to test it until the New Year, but it's easy to
reproduce, although intermittent.

Antony


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: IOException - The handle is invalid

Michael McCandless-2
Antony Bowesman wrote:

> Hi,
>
> I'm running load tests with Lucene 2.0, SUN's JDK 6 on Windows XP2, dual
> core CPU.  I have 8 worker threads adding a few hundred K documents,
> split between two Lucene indexes, I've started getting
>
> java.io.IOException: The handle is invalid in places like
>
> java.io.RandomAccessFile.writeBytes(Native Method)
> java.io.RandomAccessFile.write(RandomAccessFile.java:466)
> org.apache.lucene.store.FSIndexOutput.flushBuffer(FSDirectory.java:503)
> org.apache.lucene.store.BufferedIndexOutput.flush(BufferedIndexOutput.java:84)
>
> org.apache.lucene.store.BufferedIndexOutput.close(BufferedIndexOutput.java:98)
>
> org.apache.lucene.store.FSIndexOutput.close(FSDirectory.java:506)
> org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:415)
> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:90)
> org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:709)
> org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:686)
> org.apache.lucene.index.IndexWriter.maybeMergeSegments(IndexWriter.java:674)
>
> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:479)
> com.teamware.office.lucene.index.TWOIndexWriter.addDocument(TWOIndexWriter.java:170)
>
>
> java.io.RandomAccessFile.close0(Native Method)
> java.io.RandomAccessFile.close(RandomAccessFile.java:543)
> org.apache.lucene.store.FSIndexInput.close(FSDirectory.java:465)
> org.apache.lucene.index.CompoundFileWriter.copyFile(CompoundFileWriter.java:231)
>
> org.apache.lucene.index.CompoundFileWriter.close(CompoundFileWriter.java:168)
>
> org.apache.lucene.index.SegmentMerger.createCompoundFile(SegmentMerger.java:145)
>
> org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:735)
> org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:686)
> org.apache.lucene.index.IndexWriter.maybeMergeSegments(IndexWriter.java:674)
>
> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:479)
> com.teamware.office.lucene.index.TWOIndexWriter.addDocument(TWOIndexWriter.java:170)
>
>
> I saw Mike McCandless JIRA issue
>
> http://issues.apache.org/jira/browse/LUCENE-669
>
> Is the patch referenced there useful for a 2.0 system.  I would like to
> use the lockless commit stuff, but am waiting until I get the core
> system working well.
>
> I am also getting IOException in some of my classes, but from the JIRA
> comments, it seems that Lucene may be the culprit.

This does sound very much like LUCENE-669 (and that bug is indeed
present in Lucene 2.0).  That patch was fairly simple; it may apply
cleanly or require only small fixes.  I would recommend try it &
seeing if it resolves your IOExceptions?

If you still have more IOExceptions even after that, maybe test the
trunk.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

adb
Reply | Threaded
Open this post in threaded view
|

Re: IOException - The handle is invalid

adb
Hi Mike,

>> I saw Mike McCandless JIRA issue
>>
>> http://issues.apache.org/jira/browse/LUCENE-669
>>
>> Is the patch referenced there useful for a 2.0 system.  I would like
>> to use the lockless commit stuff, but am waiting until I get the core
>> system working well.
>>
>> I am also getting IOException in some of my classes, but from the JIRA
>> comments, it seems that Lucene may be the culprit.
>
> This does sound very much like LUCENE-669 (and that bug is indeed
> present in Lucene 2.0).  That patch was fairly simple; it may apply
> cleanly or require only small fixes.  I would recommend try it &
> seeing if it resolves your IOExceptions?

I made the patch, re-run the tests and the IOExceptions have gone away, so this
fix looks good, though with Christmas and New Year between the previous test and
these ones, I'm not 100% sure I had the same setup...

I'm glad you fixed this one already ;) !

Thanks
Antony



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: IOException - The handle is invalid

Michael McCandless-2
Antony Bowesman wrote:

> Hi Mike,
>
>>> I saw Mike McCandless JIRA issue
>>>
>>> http://issues.apache.org/jira/browse/LUCENE-669
>>>
>>> Is the patch referenced there useful for a 2.0 system.  I would like
>>> to use the lockless commit stuff, but am waiting until I get the core
>>> system working well.
>>>
>>> I am also getting IOException in some of my classes, but from the
>>> JIRA comments, it seems that Lucene may be the culprit.
>>
>> This does sound very much like LUCENE-669 (and that bug is indeed
>> present in Lucene 2.0).  That patch was fairly simple; it may apply
>> cleanly or require only small fixes.  I would recommend try it &
>> seeing if it resolves your IOExceptions?
>
> I made the patch, re-run the tests and the IOExceptions have gone away,
> so this fix looks good, though with Christmas and New Year between the
> previous test and these ones, I'm not 100% sure I had the same setup...
>
> I'm glad you fixed this one already ;) !

Awesome, I'm glad to hear that!  Let's tentatively hope that indeed
you were hitting that bug and now it's resolved :)

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]