Could not obtain block

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Could not obtain block

lists-82
I'm using nutch-0.8, hadoop 0.4.0 and getting the following exception in
my app while attempting to merge indexes:

Exception in thread "main" java.io.IOException: Could not obtain block:
blk_-5101718459437792052
file=/post/segments/20061002050004/index/part-00000/_wn0.f3 offset=0
        at
org.apache.hadoop.dfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:561)
        at
org.apache.hadoop.dfs.DFSClient$DFSInputStream.read(DFSClient.java:673)
        at
org.apache.hadoop.fs.FSDataInputStream$Checker.read(FSDataInputStream.java:84)
        at
org.apache.hadoop.fs.FSDataInputStream$PositionCache.read(FSDataInputStream.java:159)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:254)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
        at java.io.DataInputStream.read(DataInputStream.java:134)
        at
org.apache.nutch.indexer.FsDirectory$DfsIndexInput.readInternal(FsDirectory.java:182)
        at
org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:45)
        at
org.apache.lucene.index.SegmentReader.norms(SegmentReader.java:507)
        at
org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:406)
        at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:90)
        at
org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:681)
        at
org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:549)


However, I've verifed the block does exist:
[nutch@server data]$ ls |grep 5101718459437792052
blk_-5101718459437792052


Furthermore, I'm able to cat the file:
hadoop dfs -cat /post/segments/20061002050004/index/part-00000/_wn0.f3
<outputs contents of file>

I'm running a 2-node DFS, and both operations are successful on both nodes.

What am I missing here, and how do I rectify the problem?

thanks
Kelvin