Use a URL for the HADOOP_CONF_DIR?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Use a URL for the HADOOP_CONF_DIR?

Niels Basjes
Hi,

When giving users access to an Hadoop cluster they need a few XML config files (like the hadoop-site.xml ).
They put these somewhere on they PC and start running their jobs on the cluster.

Now when you're changing the settings you want those users to use (for example you changed some tcpport) you need them all to update their config files.

My question is: Can you set the HADOOP_CONF_DIR to be a URL on a webserver? 
A while ago I tried this and (back then) it didn't work.

Would this be a useful enhancement?

--
Best regards,

Niels Basjes
Reply | Threaded
Open this post in threaded view
|

Re: Use a URL for the HADOOP_CONF_DIR?

Azuryy Yu

yes. it is useful.

On Jul 16, 2013 5:40 AM, "Niels Basjes" <[hidden email]> wrote:
Hi,

When giving users access to an Hadoop cluster they need a few XML config files (like the hadoop-site.xml ).
They put these somewhere on they PC and start running their jobs on the cluster.

Now when you're changing the settings you want those users to use (for example you changed some tcpport) you need them all to update their config files.

My question is: Can you set the HADOOP_CONF_DIR to be a URL on a webserver? 
A while ago I tried this and (back then) it didn't work.

Would this be a useful enhancement?

--
Best regards,

Niels Basjes