Logging garbage collection metrics into yarn nodemanager application log directory for mappers in Hadoop

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Logging garbage collection metrics into yarn nodemanager application log directory for mappers in Hadoop

Gopi Krishnan

What is the correct way to log garbage collection metrics into the same location where yarn syslogs/stderr and stdout logs are located for mappers in Hadoop Yarn. Any insights wold be helpful:

Here are the setting I tried in mapred-site.xml, but no gc logs were available in the location ${yarn.nodemanager.log-dirs}/application_${appid} and I did not find any logs pointing out issues for the same:

<property> <name>mapreduce.map.java.opts</name> <value>-Xloggc:${yarn.nodemanager.log-dirs}/application_${appid}/gc-@taskid@.log -verbose:gc -XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintCommandLineFlags</value> </property>