save job.xml files in history

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

save job.xml files in history

Frédéric Bertin
Hi all,

Has anyone already tackled the problem of saving all submitted jobs
configuration files (job.xml)? AFAIU, the current job history feature
only saves the path to the jobconf, located in mapred.system.dir. But
this folder is deleted whenever the job tracker is restarted.
Is there already a simple way of doing this?

Thanks,
Fred
Reply | Threaded
Open this post in threaded view
|

Re: save job.xml files in history

Jeff Hammerbacher-2
seems like the easiest way to do this would be to do an asynchronous POST at
the end of JobClient.submitJob(); send job.toString() and do whatever
logging you want at the receiving end.

also, it seems as though job.toString() a different format from job.write()
-- does that bother people?

On 9/10/07, Frédéric Bertin <[hidden email]> wrote:

>
> Hi all,
>
> Has anyone already tackled the problem of saving all submitted jobs
> configuration files (job.xml)? AFAIU, the current job history feature
> only saves the path to the jobconf, located in mapred.system.dir. But
> this folder is deleted whenever the job tracker is restarted.
> Is there already a simple way of doing this?
>
> Thanks,
> Fred
>