Problem Getting Started With Hadoop

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem Getting Started With Hadoop

Pedro Guedes
Hello!

I'm having trouble following your tutorial on Getting started with
hadoop [GSWH] (from
http://wiki.apache.org/lucene-hadoop/GettingStartedWithHadoop). Is it
not up to date?

I'm using hadoop-0.12.2 that i've just downloaded today...

The GSWH guide says: "Set the variable fs.default.name to the Namenode's
intended host:port", but the defaults in hadoop-default.xml say it is
file:/// and describe it as "The name of the default file system.  A URI
whose scheme and authority determine the FileSystem implementation" so I
left it has file:///...

Next it says: "Set the variable mapred.job.tracker to the jobtrackers
intended host:port" and in the description from hadoop-default.xml I
read: "The host and port that the MapReduce job tracker runs at.  If
"local", then jobs are run in-process as a single map and reduce task."
so I left it as "local".

Then i changed the following data paths: dfs.name.dir, dfs.data.dir,
dfs.client.buffer.dir and mapred.local.dir to use a directory i specify
instead of $(hadoop.tmp.dir).

I "ensure that the dfs.name.dir directory exists" and format the
namenode with '$HADOOP_INSTALL/hadoop/bin/hadoop namenode -format'...
all goes well (or at least i get no errors)

When i try to start my single node cluster with:
'$HADOOP_INSTALL/hadoop/bin/start-all.sh' I get the following output:

$ hadoop/bin/start-all.sh
starting namenode, logging to
/home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-namenode-ps-guedes-desktop.out
ciclope@localhost's password:
localhost: starting datanode, logging to
/home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-datanode-ps-guedes-desktop.out
ciclope@localhost's password:
localhost: starting secondarynamenode, logging to
/home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-secondarynamenode-ps-guedes-desktop.out
localhost: Exception in thread "main"
java.lang.IllegalArgumentException: port out of range:-1
localhost:      at
java.net.InetSocketAddress.<init>(InetSocketAddress.java:118)
localhost:      at
org.apache.hadoop.dfs.DataNode.createSocketAddr(DataNode.java:106)
localhost:      at
org.apache.hadoop.dfs.SecondaryNameNode.<init>(SecondaryNameNode.java:93)
localhost:      at
org.apache.hadoop.dfs.SecondaryNameNode.main(SecondaryNameNode.java:474)
starting jobtracker, logging to
/home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-jobtracker-ps-guedes-desktop.out
ciclope@localhost's password:
localhost: starting tasktracker, logging to
/home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-tasktracker-ps-guedes-desktop.out

and from the logs:
ERROR org.apache.hadoop.dfs.NameNode:
java.lang.IllegalArgumentException: port out of range:-1
ERROR org.apache.hadoop.dfs.DataNode:
java.lang.IllegalArgumentException: port out of range:-1
ERROR org.apache.hadoop.mapred.JobTracker: java.lang.RuntimeException:
Bad mapred.job.tracker: local
ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker
because java.lang.RuntimeException: Bad mapred.job.tracker: local
and finally for the secondary namenode the erros goes to the .out file:
java.lang.IllegalArgumentException: port out of range:-1

humm... this must be a really stupid mistake... so i'm either being
really dumb or the Guide isn't very well specified... probably both!

Can anybody help? If it's a problem with the wiki i'm very much willing
to help fix it...

Thanks in advance,

Pedro Guedes


Reply | Threaded
Open this post in threaded view
|

RE: Problem Getting Started With Hadoop

Richard Yang-3



-----Original Message-----
From: Pedro Guedes [mailto:[hidden email]]
Sent: Wed 3/28/2007 8:04 AM
To: Hadoop Mailling list
Subject: Problem Getting Started With Hadoop
 
Hello!

I'm having trouble following your tutorial on Getting started with
hadoop [GSWH] (from
http://wiki.apache.org/lucene-hadoop/GettingStartedWithHadoop). Is it
not up to date?

I'm using hadoop-0.12.2 that i've just downloaded today...

The GSWH guide says: "Set the variable fs.default.name to the Namenode's
intended host:port", but the defaults in hadoop-default.xml say it is
file:/// and describe it as "The name of the default file system.  A URI
whose scheme and authority determine the FileSystem implementation" so I
left it has file:///...

>i think with the default setting, you will run everything local

Next it says: "Set the variable mapred.job.tracker to the jobtrackers
intended host:port" and in the description from hadoop-default.xml I
read: "The host and port that the MapReduce job tracker runs at.  If
"local", then jobs are run in-process as a single map and reduce task."
so I left it as "local".

Then i changed the following data paths: dfs.name.dir, dfs.data.dir,
dfs.client.buffer.dir and mapred.local.dir to use a directory i specify
instead of $(hadoop.tmp.dir).

I "ensure that the dfs.name.dir directory exists" and format the
namenode with '$HADOOP_INSTALL/hadoop/bin/hadoop namenode -format'...
all goes well (or at least i get no errors)

When i try to start my single node cluster with:
'$HADOOP_INSTALL/hadoop/bin/start-all.sh' I get the following output:

$ hadoop/bin/start-all.sh
starting namenode, logging to
/home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-namenode-ps-guedes-desktop.out
ciclope@localhost's password:
localhost: starting datanode, logging to
/home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-datanode-ps-guedes-desktop.out
ciclope@localhost's password:
localhost: starting secondarynamenode, logging to
/home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-secondarynamenode-ps-guedes-desktop.out
localhost: Exception in thread "main"
java.lang.IllegalArgumentException: port out of range:-1
localhost:      at
java.net.InetSocketAddress.<init>(InetSocketAddress.java:118)
localhost:      at
org.apache.hadoop.dfs.DataNode.createSocketAddr(DataNode.java:106)
localhost:      at
org.apache.hadoop.dfs.SecondaryNameNode.<init>(SecondaryNameNode.java:93)
localhost:      at
org.apache.hadoop.dfs.SecondaryNameNode.main(SecondaryNameNode.java:474)
starting jobtracker, logging to
/home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-jobtracker-ps-guedes-desktop.out
ciclope@localhost's password:
localhost: starting tasktracker, logging to
/home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-tasktracker-ps-guedes-desktop.out

and from the logs:
ERROR org.apache.hadoop.dfs.NameNode:
java.lang.IllegalArgumentException: port out of range:-1
ERROR org.apache.hadoop.dfs.DataNode:
java.lang.IllegalArgumentException: port out of range:-1
ERROR org.apache.hadoop.mapred.JobTracker: java.lang.RuntimeException:
Bad mapred.job.tracker: local
ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker
because java.lang.RuntimeException: Bad mapred.job.tracker: local
and finally for the secondary namenode the erros goes to the .out file:
java.lang.IllegalArgumentException: port out of range:-1

humm... this must be a really stupid mistake... so i'm either being
really dumb or the Guide isn't very well specified... probably both!

Can anybody help? If it's a problem with the wiki i'm very much willing
to help fix it...

Thanks in advance,

Pedro Guedes

>>i suggest you to google around for more examples and instructions.  i agree that the official instructions are minimal.



Reply | Threaded
Open this post in threaded view
|

Re: Problem Getting Started With Hadoop

Pedro Guedes
Hi...

I was confused by the file://... changed it to (the obvious)
localhost:<some_port> and it works now... sorry for the silly questions,
and keep up the good work!

Pedro

Richard Yang wrote:

>
> -----Original Message-----
> From: Pedro Guedes [mailto:[hidden email]]
> Sent: Wed 3/28/2007 8:04 AM
> To: Hadoop Mailling list
> Subject: Problem Getting Started With Hadoop
>  
> Hello!
>
> I'm having trouble following your tutorial on Getting started with
> hadoop [GSWH] (from
> http://wiki.apache.org/lucene-hadoop/GettingStartedWithHadoop). Is it
> not up to date?
>
> I'm using hadoop-0.12.2 that i've just downloaded today...
>
> The GSWH guide says: "Set the variable fs.default.name to the Namenode's
> intended host:port", but the defaults in hadoop-default.xml say it is
> file:/// and describe it as "The name of the default file system.  A URI
> whose scheme and authority determine the FileSystem implementation" so I
> left it has file:///...
>
>  
>> i think with the default setting, you will run everything local
>>    
>
> Next it says: "Set the variable mapred.job.tracker to the jobtrackers
> intended host:port" and in the description from hadoop-default.xml I
> read: "The host and port that the MapReduce job tracker runs at.  If
> "local", then jobs are run in-process as a single map and reduce task."
> so I left it as "local".
>
> Then i changed the following data paths: dfs.name.dir, dfs.data.dir,
> dfs.client.buffer.dir and mapred.local.dir to use a directory i specify
> instead of $(hadoop.tmp.dir).
>
> I "ensure that the dfs.name.dir directory exists" and format the
> namenode with '$HADOOP_INSTALL/hadoop/bin/hadoop namenode -format'...
> all goes well (or at least i get no errors)
>
> When i try to start my single node cluster with:
> '$HADOOP_INSTALL/hadoop/bin/start-all.sh' I get the following output:
>
> $ hadoop/bin/start-all.sh
> starting namenode, logging to
> /home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-namenode-ps-guedes-desktop.out
> ciclope@localhost's password:
> localhost: starting datanode, logging to
> /home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-datanode-ps-guedes-desktop.out
> ciclope@localhost's password:
> localhost: starting secondarynamenode, logging to
> /home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-secondarynamenode-ps-guedes-desktop.out
> localhost: Exception in thread "main"
> java.lang.IllegalArgumentException: port out of range:-1
> localhost:      at
> java.net.InetSocketAddress.<init>(InetSocketAddress.java:118)
> localhost:      at
> org.apache.hadoop.dfs.DataNode.createSocketAddr(DataNode.java:106)
> localhost:      at
> org.apache.hadoop.dfs.SecondaryNameNode.<init>(SecondaryNameNode.java:93)
> localhost:      at
> org.apache.hadoop.dfs.SecondaryNameNode.main(SecondaryNameNode.java:474)
> starting jobtracker, logging to
> /home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-jobtracker-ps-guedes-desktop.out
> ciclope@localhost's password:
> localhost: starting tasktracker, logging to
> /home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-tasktracker-ps-guedes-desktop.out
>
> and from the logs:
> ERROR org.apache.hadoop.dfs.NameNode:
> java.lang.IllegalArgumentException: port out of range:-1
> ERROR org.apache.hadoop.dfs.DataNode:
> java.lang.IllegalArgumentException: port out of range:-1
> ERROR org.apache.hadoop.mapred.JobTracker: java.lang.RuntimeException:
> Bad mapred.job.tracker: local
> ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker
> because java.lang.RuntimeException: Bad mapred.job.tracker: local
> and finally for the secondary namenode the erros goes to the .out file:
> java.lang.IllegalArgumentException: port out of range:-1
>
> humm... this must be a really stupid mistake... so i'm either being
> really dumb or the Guide isn't very well specified... probably both!
>
> Can anybody help? If it's a problem with the wiki i'm very much willing
> to help fix it...
>
> Thanks in advance,
>
> Pedro Guedes
>
>  
>>> i suggest you to google around for more examples and instructions.  i agree that the official instructions are minimal.
>>>      
>
>
>
>  

Reply | Threaded
Open this post in threaded view
|

RE: Problem Getting Started With Hadoop

Richard Yang-3
In reply to this post by Pedro Guedes

no problem at all. i was confused too. it could be a host name with tcp some port. in this case, your dns must be set up.

otherwise, use a ip and a tcp port.  that's what i have so far still.  i don't want to introduce additional complications to my cluster.

richard

-----Original Message-----
From: Pedro Guedes [mailto:[hidden email]]
Sent: Wed 3/28/2007 9:18 AM
To: [hidden email]
Subject: Re: Problem Getting Started With Hadoop
 
Hi...

I was confused by the file://... changed it to (the obvious)
localhost:<some_port> and it works now... sorry for the silly questions,
and keep up the good work!

Pedro

Richard Yang wrote:

>
> -----Original Message-----
> From: Pedro Guedes [mailto:[hidden email]]
> Sent: Wed 3/28/2007 8:04 AM
> To: Hadoop Mailling list
> Subject: Problem Getting Started With Hadoop
>  
> Hello!
>
> I'm having trouble following your tutorial on Getting started with
> hadoop [GSWH] (from
> http://wiki.apache.org/lucene-hadoop/GettingStartedWithHadoop). Is it
> not up to date?
>
> I'm using hadoop-0.12.2 that i've just downloaded today...
>
> The GSWH guide says: "Set the variable fs.default.name to the Namenode's
> intended host:port", but the defaults in hadoop-default.xml say it is
> file:/// and describe it as "The name of the default file system.  A URI
> whose scheme and authority determine the FileSystem implementation" so I
> left it has file:///...
>
>  
>> i think with the default setting, you will run everything local
>>    
>
> Next it says: "Set the variable mapred.job.tracker to the jobtrackers
> intended host:port" and in the description from hadoop-default.xml I
> read: "The host and port that the MapReduce job tracker runs at.  If
> "local", then jobs are run in-process as a single map and reduce task."
> so I left it as "local".
>
> Then i changed the following data paths: dfs.name.dir, dfs.data.dir,
> dfs.client.buffer.dir and mapred.local.dir to use a directory i specify
> instead of $(hadoop.tmp.dir).
>
> I "ensure that the dfs.name.dir directory exists" and format the
> namenode with '$HADOOP_INSTALL/hadoop/bin/hadoop namenode -format'...
> all goes well (or at least i get no errors)
>
> When i try to start my single node cluster with:
> '$HADOOP_INSTALL/hadoop/bin/start-all.sh' I get the following output:
>
> $ hadoop/bin/start-all.sh
> starting namenode, logging to
> /home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-namenode-ps-guedes-desktop.out
> ciclope@localhost's password:
> localhost: starting datanode, logging to
> /home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-datanode-ps-guedes-desktop.out
> ciclope@localhost's password:
> localhost: starting secondarynamenode, logging to
> /home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-secondarynamenode-ps-guedes-desktop.out
> localhost: Exception in thread "main"
> java.lang.IllegalArgumentException: port out of range:-1
> localhost:      at
> java.net.InetSocketAddress.<init>(InetSocketAddress.java:118)
> localhost:      at
> org.apache.hadoop.dfs.DataNode.createSocketAddr(DataNode.java:106)
> localhost:      at
> org.apache.hadoop.dfs.SecondaryNameNode.<init>(SecondaryNameNode.java:93)
> localhost:      at
> org.apache.hadoop.dfs.SecondaryNameNode.main(SecondaryNameNode.java:474)
> starting jobtracker, logging to
> /home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-jobtracker-ps-guedes-desktop.out
> ciclope@localhost's password:
> localhost: starting tasktracker, logging to
> /home/ciclope/hadoop-install/hadoop/bin/../logs/hadoop-ciclope-tasktracker-ps-guedes-desktop.out
>
> and from the logs:
> ERROR org.apache.hadoop.dfs.NameNode:
> java.lang.IllegalArgumentException: port out of range:-1
> ERROR org.apache.hadoop.dfs.DataNode:
> java.lang.IllegalArgumentException: port out of range:-1
> ERROR org.apache.hadoop.mapred.JobTracker: java.lang.RuntimeException:
> Bad mapred.job.tracker: local
> ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker
> because java.lang.RuntimeException: Bad mapred.job.tracker: local
> and finally for the secondary namenode the erros goes to the .out file:
> java.lang.IllegalArgumentException: port out of range:-1
>
> humm... this must be a really stupid mistake... so i'm either being
> really dumb or the Guide isn't very well specified... probably both!
>
> Can anybody help? If it's a problem with the wiki i'm very much willing
> to help fix it...
>
> Thanks in advance,
>
> Pedro Guedes
>
>  
>>> i suggest you to google around for more examples and instructions.  i agree that the official instructions are minimal.
>>>      
>
>
>
>