Quantcast

Starting datanode in secure mode

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Starting datanode in secure mode

Thomas Weise
I'm configuring a local hadoop cluster in secure mode for development/experimental purposes on Ubuntu 11.04 with the hadoop-0.20.203.0 distribution from apache mirror.

I have the basic Kerberos setup working, can start namenode in secure mode and connect to it with hadoop fs -ls

I'm not able to get the datanode start in secure mode - what do I have to do to make that happen?

The error I get:

11/08/30 18:01:57 INFO security.UserGroupInformation: Login successful for user hduser/[hidden email] using keytab file /opt/hadoop/conf/nn.keytab
11/08/30 18:01:57 ERROR datanode.DataNode: java.lang.RuntimeException: Cannot start secure cluster without privileged resources.
        at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:293)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:268)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1480)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1419)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1437)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1563)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1573)

11/08/30 18:01:57 INFO datanode.DataNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down DataNode at hdev-vm/127.0.1.1

I have not configured the system to use port numbers that require root (yet). All I want is the datanode to run in secure mode with kerberos authentication.

Any pointers would be greatly appreciated!

Thomas

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Starting datanode in secure mode

Ravi Prakash-3
In short you MUST use priviledged resourced.

In long:

Here's what I did to setup a secure single node cluster. I'm sure there's
other ways, but here's how I did it.

    1.    Install krb5-server
    2.    Setup the kerberos configuration (files attached).
/var/kerberos/krb5kdc/kdc.conf and /etc/krb5.conf
http://yahoo.github.com/hadoop-common/installing.html
    3.    To clean up everything :
http://mailman.mit.edu/pipermail/kerberos/2003-June/003312.html
    4.    Create Kerberos database $ sudo kdb5_util create -s
    5.    Start Kerberos $ sudo /etc/rc.d/init.d/kadmin start $ sudo
/etc/rc.d/init.d/krb5kdc start
    6.    Create principal raviprak/localhost.localdomain@localdomain
http://web.mit.edu/kerberos/krb5-1.5/krb5-1.5.4/doc/krb5-admin/Adding-or-Modifying-Principals.html
    7.    Create keytab fiie using “xst -k /home/raviprak/raviprak.keytab
raviprak/localhost.localdomain@localdomain”
    8.    Setup hdfs-site.xml and core-site.xml (files attached)
    9.    sudo hostname localhost.localdomain
    10.    hadoop-daemon.sh start namenode
    11.    sudo bash. Then export HADOOP_SECURE_DN_USER=raviprak . Then
hadoop-daemon.sh start datanode



CORE-SITE.XML
========================================
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
    <property>
        <name>fs.default.name</name>
        <value>hdfs://localhost:9001</value>
    </property>
    <property>
        <name>hadoop.security.authorization</name>
        <value>true</value>
    </property>
    <property>
        <name>hadoop.security.authentication</name>
        <value>kerberos</value>
    </property>
  <property>
    <name>dfs.namenode.kerberos.principal</name>
    <value>raviprak/localhost.localdomain</value>
  </property>
  <property>
    <name>dfs.datanode.kerberos.principal</name>
    <value>raviprak/localhost.localdomain</value>
  </property>
  <property>
    <name>dfs.secondary.namenode.kerberos.principal</name>
    <value>raviprak/localhost.localdomain</value>
  </property>
</configuration>

=========================================================



HDFS-SITE.XML
=========================================================
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>

    <property>
        <name>dfs.name.dir.restore</name>
        <value>false</value>
    </property>

    <property>
        <name>dfs.namenode.checkpoint.period</name>
        <value>10</value>
    </property>

    <property>
        <name>dfs.namenode.keytab.file</name>
        <value>/home/raviprak/raviprak.keytab</value>
    </property>

    <property>
        <name>dfs.secondary.namenode.keytab.file</name>
        <value>/home/raviprak/raviprak.keytab</value>
    </property>

    <property>
        <name>dfs.datanode.keytab.file</name>
        <value>/home/raviprak/raviprak.keytab</value>
    </property>

    <property>
        <name>dfs.datanode.address</name>
        <value>0.0.0.0:1004</value>
    </property>

    <property>
        <name>dfs.datanode.http.address</name>
        <value>0.0.0.0:1006</value>
    </property>

    <property>
        <name>dfs.namenode.kerberos.principal</name>
        <value>raviprak/localhost.localdomain@localdomain</value>
    </property>

    <property>
        <name>dfs.secondary.namenode.kerberos.principal</name>
        <value>raviprak/localhost.localdomain@localdomain</value>
    </property>

    <property>
        <name>dfs.datanode.kerberos.principal</name>
        <value>raviprak/localhost.localdomain@localdomain</value>
    </property>

    <property>
        <name>dfs.namenode.kerberos.https.principal</name>
        <value>raviprak/localhost.localdomain@localdomain</value>
    </property>

    <property>
        <name>dfs.secondary.namenode.kerberos.https.principal</name>
        <value>raviprak/localhost.localdomain@localdomain</value>
    </property>

    <property>
        <name>dfs.datanode.kerberos.https.principal</name>
        <value>raviprak/localhost.localdomain@localdomain</value>
    </property>

</configuration>
=========================================================


On Tue, Aug 30, 2011 at 8:08 PM, Thomas Weise <[hidden email]> wrote:

> I'm configuring a local hadoop cluster in secure mode for
> development/experimental purposes on Ubuntu 11.04 with the hadoop-0.20.203.0
> distribution from apache mirror.
>
> I have the basic Kerberos setup working, can start namenode in secure mode
> and connect to it with hadoop fs -ls
>
> I'm not able to get the datanode start in secure mode - what do I have to
> do to make that happen?
>
> The error I get:
>
> 11/08/30 18:01:57 INFO security.UserGroupInformation: Login successful for
> user hduser/[hidden email] using keytab file
> /opt/hadoop/conf/nn.keytab
> 11/08/30 18:01:57 ERROR datanode.DataNode: java.lang.RuntimeException:
> Cannot start secure cluster without privileged resources.
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:293)
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:268)
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1480)
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1419)
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1437)
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1563)
>        at
> org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1573)
>
> 11/08/30 18:01:57 INFO datanode.DataNode: SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down DataNode at hdev-vm/127.0.1.1
>
> I have not configured the system to use port numbers that require root
> (yet). All I want is the datanode to run in secure mode with kerberos
> authentication.
>
> Any pointers would be greatly appreciated!
>
> Thomas
>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Starting datanode in secure mode

Thomas Weise
Thanks Ravi. This has brought my local hadoop cluster to life!

The two things I was missing:

1) Have to use privileged ports

<!-- secure setup requires privileged ports -->
<property>
  <name>dfs.datanode.address</name>
  <value>0.0.0.0:1004</value>
</property>
<property>
  <name>dfs.datanode.http.address</name>
  <value>0.0.0.0:1006</value>
</property>

2) implied by 1) sudo required to launch datanode

Clearly, this is geared towards the production system. For development, having the ability to run with Kerberos but w/o the need for privileged resources would be desirable.


On Aug 30, 2011, at 9:00 PM, Ravi Prakash wrote:

> In short you MUST use priviledged resourced.
>
> In long:
>
> Here's what I did to setup a secure single node cluster. I'm sure there's
> other ways, but here's how I did it.
>
>    1.    Install krb5-server
>    2.    Setup the kerberos configuration (files attached).
> /var/kerberos/krb5kdc/kdc.conf and /etc/krb5.conf
> http://yahoo.github.com/hadoop-common/installing.html
>    3.    To clean up everything :
> http://mailman.mit.edu/pipermail/kerberos/2003-June/003312.html
>    4.    Create Kerberos database $ sudo kdb5_util create -s
>    5.    Start Kerberos $ sudo /etc/rc.d/init.d/kadmin start $ sudo
> /etc/rc.d/init.d/krb5kdc start
>    6.    Create principal raviprak/localhost.localdomain@localdomain
> http://web.mit.edu/kerberos/krb5-1.5/krb5-1.5.4/doc/krb5-admin/Adding-or-Modifying-Principals.html
>    7.    Create keytab fiie using “xst -k /home/raviprak/raviprak.keytab
> raviprak/localhost.localdomain@localdomain”
>    8.    Setup hdfs-site.xml and core-site.xml (files attached)
>    9.    sudo hostname localhost.localdomain
>    10.    hadoop-daemon.sh start namenode
>    11.    sudo bash. Then export HADOOP_SECURE_DN_USER=raviprak . Then
> hadoop-daemon.sh start datanode
>
>
>
> CORE-SITE.XML
> ========================================
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>
> <!-- Put site-specific property overrides in this file. -->
>
> <configuration>
>    <property>
>        <name>fs.default.name</name>
>        <value>hdfs://localhost:9001</value>
>    </property>
>    <property>
>        <name>hadoop.security.authorization</name>
>        <value>true</value>
>    </property>
>    <property>
>        <name>hadoop.security.authentication</name>
>        <value>kerberos</value>
>    </property>
>  <property>
>    <name>dfs.namenode.kerberos.principal</name>
>    <value>raviprak/localhost.localdomain</value>
>  </property>
>  <property>
>    <name>dfs.datanode.kerberos.principal</name>
>    <value>raviprak/localhost.localdomain</value>
>  </property>
>  <property>
>    <name>dfs.secondary.namenode.kerberos.principal</name>
>    <value>raviprak/localhost.localdomain</value>
>  </property>
> </configuration>
>
> =========================================================
>
>
>
> HDFS-SITE.XML
> =========================================================
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>
> <!-- Put site-specific property overrides in this file. -->
>
> <configuration>
>    <property>
>        <name>dfs.replication</name>
>        <value>1</value>
>    </property>
>
>    <property>
>        <name>dfs.name.dir.restore</name>
>        <value>false</value>
>    </property>
>
>    <property>
>        <name>dfs.namenode.checkpoint.period</name>
>        <value>10</value>
>    </property>
>
>    <property>
>        <name>dfs.namenode.keytab.file</name>
>        <value>/home/raviprak/raviprak.keytab</value>
>    </property>
>
>    <property>
>        <name>dfs.secondary.namenode.keytab.file</name>
>        <value>/home/raviprak/raviprak.keytab</value>
>    </property>
>
>    <property>
>        <name>dfs.datanode.keytab.file</name>
>        <value>/home/raviprak/raviprak.keytab</value>
>    </property>
>
>    <property>
>        <name>dfs.datanode.address</name>
>        <value>0.0.0.0:1004</value>
>    </property>
>
>    <property>
>        <name>dfs.datanode.http.address</name>
>        <value>0.0.0.0:1006</value>
>    </property>
>
>    <property>
>        <name>dfs.namenode.kerberos.principal</name>
>        <value>raviprak/localhost.localdomain@localdomain</value>
>    </property>
>
>    <property>
>        <name>dfs.secondary.namenode.kerberos.principal</name>
>        <value>raviprak/localhost.localdomain@localdomain</value>
>    </property>
>
>    <property>
>        <name>dfs.datanode.kerberos.principal</name>
>        <value>raviprak/localhost.localdomain@localdomain</value>
>    </property>
>
>    <property>
>        <name>dfs.namenode.kerberos.https.principal</name>
>        <value>raviprak/localhost.localdomain@localdomain</value>
>    </property>
>
>    <property>
>        <name>dfs.secondary.namenode.kerberos.https.principal</name>
>        <value>raviprak/localhost.localdomain@localdomain</value>
>    </property>
>
>    <property>
>        <name>dfs.datanode.kerberos.https.principal</name>
>        <value>raviprak/localhost.localdomain@localdomain</value>
>    </property>
>
> </configuration>
> =========================================================
>
>
> On Tue, Aug 30, 2011 at 8:08 PM, Thomas Weise <[hidden email]> wrote:
>
>> I'm configuring a local hadoop cluster in secure mode for
>> development/experimental purposes on Ubuntu 11.04 with the hadoop-0.20.203.0
>> distribution from apache mirror.
>>
>> I have the basic Kerberos setup working, can start namenode in secure mode
>> and connect to it with hadoop fs -ls
>>
>> I'm not able to get the datanode start in secure mode - what do I have to
>> do to make that happen?
>>
>> The error I get:
>>
>> 11/08/30 18:01:57 INFO security.UserGroupInformation: Login successful for
>> user hduser/[hidden email] using keytab file
>> /opt/hadoop/conf/nn.keytab
>> 11/08/30 18:01:57 ERROR datanode.DataNode: java.lang.RuntimeException:
>> Cannot start secure cluster without privileged resources.
>>       at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:293)
>>       at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:268)
>>       at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1480)
>>       at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1419)
>>       at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1437)
>>       at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1563)
>>       at
>> org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1573)
>>
>> 11/08/30 18:01:57 INFO datanode.DataNode: SHUTDOWN_MSG:
>> /************************************************************
>> SHUTDOWN_MSG: Shutting down DataNode at hdev-vm/127.0.1.1
>>
>> I have not configured the system to use port numbers that require root
>> (yet). All I want is the datanode to run in secure mode with kerberos
>> authentication.
>>
>> Any pointers would be greatly appreciated!
>>
>> Thomas
>>
>>

Loading...