ndfs / Lost connection to namenode

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

ndfs / Lost connection to namenode

Mr. Udatny
i use the nutch 0.7.1 distribution

starting namenode seems to be ok (as i can see from the logs)

051122 161211 parsing
file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml
051122 161212 parsing
file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
051122 161212 Server listener on port 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting


when trying to start ndfs datanode i get the following error:
(i changed the code to show the exception stacktrace)

051122 161220 parsing
file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml
051122 161220 parsing
file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
051122 161220 Using [/tmp/nutch/ndfs/data] directory for data storage.
051122 161220 Opened server at 7000
051122 161220 Lost connection to namenode [/10.100.100.181:8009].  
Retrying...
java.util.NoSuchElementException
        at java.util.StringTokenizer.nextToken(StringTokenizer.java:332)
        at org.apache.nutch.ndfs.DF.<init>(DF.java:51)
        at org.apache.nutch.ndfs.FSDataset.getCapacity(FSDataset.java:204)
        at org.apache.nutch.ndfs.NDFS$DataNode.offerService(NDFS.java:476)
        at org.apache.nutch.ndfs.NDFS$DataNode.main(NDFS.java:922)
051122 161225 Lost connection to namenode [/10.100.100.181:8009].  
Retrying...
java.util.NoSuchElementException
        at java.util.StringTokenizer.nextToken(StringTokenizer.java:332)
        at org.apache.nutch.ndfs.DF.<init>(DF.java:51)
        at org.apache.nutch.ndfs.FSDataset.getCapacity(FSDataset.java:204)
        at org.apache.nutch.ndfs.NDFS$DataNode.offerService(NDFS.java:476)
        at org.apache.nutch.ndfs.NDFS$DataNode.main(NDFS.java:922)


i could connect to the namenode by just telnetting there:
telnet 10.100.100.181 8009
Trying 10.100.100.181...
Connected to 10.100.100.181 (10.100.100.181).
Escape character is '^]'.
...


further i can use the testclient to create directories and list the
contents of the ndfs. if it comes to a "-put" another exception arises:
[rude@ws-rude nutch-0.7.1]$ ./bin/nutch org.apache.nutch.fs.TestClient -du /
051122 161956 parsing
file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml051122
161956 parsing file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
051122 161956 No FS indicated, using default:10.100.100.181:8009
051122 161956 Client connection to 10.100.100.181:8009: starting
Found 1 items
/test   0
[rude@ws-rude nutch-0.7.1]$ ./bin/nutch org.apache.nutch.fs.TestClient
-put ./nutch-0.7.1.jar /testfile
051122 162013 parsing
file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml051122
162013 parsing file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
051122 162013 No FS indicated, using default:10.100.100.181:8009
051122 162013 Client connection to 10.100.100.181:8009: starting
Exception in thread "main" java.io.IOException: Could not obtain new
output block for file /testfile
        at
org.apache.nutch.ndfs.NDFSClient$NameNodeCaller.getNewOutputBlock(NDFSClient.java:921)
        at
org.apache.nutch.ndfs.NDFSClient$NDFSOutputStream.nextBlockOutputStream(NDFSClient.java:616)
        at
org.apache.nutch.ndfs.NDFSClient$NDFSOutputStream.<init>(NDFSClient.java:597)
        at org.apache.nutch.ndfs.NDFSClient.create(NDFSClient.java:85)
        at org.apache.nutch.fs.NDFSFileSystem.create(NDFSFileSystem.java:76)
        at org.apache.nutch.fs.NDFSFileSystem.create(NDFSFileSystem.java:71)
        at
org.apache.nutch.fs.NDFSFileSystem.doFromLocalFile(NDFSFileSystem.java:183)
        at
org.apache.nutch.fs.NDFSFileSystem.copyFromLocalFile(NDFSFileSystem.java:158)
        at org.apache.nutch.fs.TestClient.copyFromLocal(TestClient.java:46)
        at org.apache.nutch.fs.TestClient.main(TestClient.java:234)
[rude@ws-rude nutch-0.7.1]$



who tells me what i do wrong?

regards,

ud


Reply | Threaded
Open this post in threaded view
|

Re: ndfs / Lost connection to namenode

Mr. Udatny
i checked out the latest mapred branch version from svn
and it seems to work!

java -cp ./:./classes/:./conf org.apache.nutch.ndfs.NameNode &
java -cp ./:./classes/:./conf org.apache.nutch.ndfs.DataNode &
java -cp ./:./classes/:./conf  org.apache.nutch.fs.TestClient -put
somebigfile /testfile
java -cp ./:./classes/:./conf  org.apache.nutch.fs.TestClient -report

cool.

ud



Mr. Udatny wrote:

> i use the nutch 0.7.1 distribution
>
> starting namenode seems to be ok (as i can see from the logs)
>
> 051122 161211 parsing
> file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml
> 051122 161212 parsing
> file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
> 051122 161212 Server listener on port 8009: starting
> 051122 161212 Server handler on 8009: starting
> 051122 161212 Server handler on 8009: starting
> 051122 161212 Server handler on 8009: starting
> 051122 161212 Server handler on 8009: starting
> 051122 161212 Server handler on 8009: starting
> 051122 161212 Server handler on 8009: starting
> 051122 161212 Server handler on 8009: starting
> 051122 161212 Server handler on 8009: starting
> 051122 161212 Server handler on 8009: starting
> 051122 161212 Server handler on 8009: starting
>
>
> when trying to start ndfs datanode i get the following error:
> (i changed the code to show the exception stacktrace)
>
> 051122 161220 parsing
> file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml
> 051122 161220 parsing
> file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
> 051122 161220 Using [/tmp/nutch/ndfs/data] directory for data storage.
> 051122 161220 Opened server at 7000
> 051122 161220 Lost connection to namenode [/10.100.100.181:8009].  
> Retrying...
> java.util.NoSuchElementException
>        at java.util.StringTokenizer.nextToken(StringTokenizer.java:332)
>        at org.apache.nutch.ndfs.DF.<init>(DF.java:51)
>        at org.apache.nutch.ndfs.FSDataset.getCapacity(FSDataset.java:204)
>        at org.apache.nutch.ndfs.NDFS$DataNode.offerService(NDFS.java:476)
>        at org.apache.nutch.ndfs.NDFS$DataNode.main(NDFS.java:922)
> 051122 161225 Lost connection to namenode [/10.100.100.181:8009].  
> Retrying...
> java.util.NoSuchElementException
>        at java.util.StringTokenizer.nextToken(StringTokenizer.java:332)
>        at org.apache.nutch.ndfs.DF.<init>(DF.java:51)
>        at org.apache.nutch.ndfs.FSDataset.getCapacity(FSDataset.java:204)
>        at org.apache.nutch.ndfs.NDFS$DataNode.offerService(NDFS.java:476)
>        at org.apache.nutch.ndfs.NDFS$DataNode.main(NDFS.java:922)
>
>
> i could connect to the namenode by just telnetting there:
> telnet 10.100.100.181 8009
> Trying 10.100.100.181...
> Connected to 10.100.100.181 (10.100.100.181).
> Escape character is '^]'.
> ...
>
>
> further i can use the testclient to create directories and list the
> contents of the ndfs. if it comes to a "-put" another exception arises:
> [rude@ws-rude nutch-0.7.1]$ ./bin/nutch org.apache.nutch.fs.TestClient
> -du /
> 051122 161956 parsing
> file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml051122
> 161956 parsing file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
> 051122 161956 No FS indicated, using default:10.100.100.181:8009
> 051122 161956 Client connection to 10.100.100.181:8009: starting
> Found 1 items
> /test   0
> [rude@ws-rude nutch-0.7.1]$ ./bin/nutch org.apache.nutch.fs.TestClient
> -put ./nutch-0.7.1.jar /testfile
> 051122 162013 parsing
> file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml051122
> 162013 parsing file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
> 051122 162013 No FS indicated, using default:10.100.100.181:8009
> 051122 162013 Client connection to 10.100.100.181:8009: starting
> Exception in thread "main" java.io.IOException: Could not obtain new
> output block for file /testfile
>        at
> org.apache.nutch.ndfs.NDFSClient$NameNodeCaller.getNewOutputBlock(NDFSClient.java:921)
>
>        at
> org.apache.nutch.ndfs.NDFSClient$NDFSOutputStream.nextBlockOutputStream(NDFSClient.java:616)
>
>        at
> org.apache.nutch.ndfs.NDFSClient$NDFSOutputStream.<init>(NDFSClient.java:597)
>
>        at org.apache.nutch.ndfs.NDFSClient.create(NDFSClient.java:85)
>        at
> org.apache.nutch.fs.NDFSFileSystem.create(NDFSFileSystem.java:76)
>        at
> org.apache.nutch.fs.NDFSFileSystem.create(NDFSFileSystem.java:71)
>        at
> org.apache.nutch.fs.NDFSFileSystem.doFromLocalFile(NDFSFileSystem.java:183)
>
>        at
> org.apache.nutch.fs.NDFSFileSystem.copyFromLocalFile(NDFSFileSystem.java:158)
>
>        at
> org.apache.nutch.fs.TestClient.copyFromLocal(TestClient.java:46)
>        at org.apache.nutch.fs.TestClient.main(TestClient.java:234)
> [rude@ws-rude nutch-0.7.1]$
>
>
>
> who tells me what i do wrong?
>
> regards,
>
> ud
>
>