Quantcast

Can I change "hadoo.tmp.dir" for each jon run without formatting

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Can I change "hadoo.tmp.dir" for each jon run without formatting

abhay ratnaparkhi
"hadoop.tmp.dir" points to the directory on local disk to store
intermediate task related data.

It's currently mounted to "/tmp/hadoop" for me. Some of my jobs are running
and Filesystem on which '/tmp' is mounted is getting full.
Is it possible to change "hadoop.tmp.dir" parameter before submitting a new
job?

~Abhay
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Can I change "hadoo.tmp.dir" for each jon run without formatting

Alok Kumar
Hi Abhay,

On Wed, Jul 25, 2012 at 10:44 PM, Abhay Ratnaparkhi
<[hidden email]> wrote:
> "hadoop.tmp.dir" points to the directory on local disk to store
> intermediate task related data.
>
> It's currently mounted to "/tmp/hadoop" for me. Some of my jobs are running
> and Filesystem on which '/tmp' is mounted is getting full.
> Is it possible to change "hadoop.tmp.dir" parameter before submitting a new
> job?

You can override "hadoop.tmp.dir" everytime before submitting your Job.
I tried like this :

Configuration configuration = new Configuration();
config.set("hadoop.tmp.dir", "/home/user/some-other-path");
Job job = new Job(config, "Job1");

It produced same result (I didn't format anything)

Thanks
--
Alok
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Can I change "hadoo.tmp.dir" for each jon run without formatting

in.abdul

Hi abhay ,
As alok mentioned that's a perfect choice to override on runtime . Make sure that properties should not be set as final in configuration file .

Regards
Syed

On Jul 26, 2012 12:16 AM, "Alok Kumar [via Lucene]" <[hidden email]> wrote:
Hi Abhay,

On Wed, Jul 25, 2012 at 10:44 PM, Abhay Ratnaparkhi
<[hidden email]> wrote:
> "hadoop.tmp.dir" points to the directory on local disk to store
> intermediate task related data.
>
> It's currently mounted to "/tmp/hadoop" for me. Some of my jobs are running
> and Filesystem on which '/tmp' is mounted is getting full.
> Is it possible to change "hadoop.tmp.dir" parameter before submitting a new
> job?

You can override "hadoop.tmp.dir" everytime before submitting your Job.
I tried like this :

Configuration configuration = new Configuration();
config.set("hadoop.tmp.dir", "/home/user/some-other-path");
Job job = new Job(config, "Job1");

It produced same result (I didn't format anything)

Thanks
--
Alok



If you reply to this email, your message will be added to the discussion below:
http://lucene.472066.n3.nabble.com/Can-I-change-hadoo-tmp-dir-for-each-jon-run-without-formatting-tp3997287p3997300.html
To unsubscribe from Lucene, click here.
NAML
THANKS AND REGARDS, SYED ABDUL KATHER
Loading...