Jprofiler compile options

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Jprofiler compile options

Murat Ali Bayir
Hi eveybody, I have problem in running Jprofiler in remote side.  I am
using DFS and submitting crawl job.
I configure LD library path and hadoop-default.xml in machine which I
run hadoop tasktracker,
in hadoop-default.xml, my java child compile option is given as:

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

<property>
  <name>mapred.child.java.opts</name>
  <value>-Xmx200m
-agentlib:jprofilerti=port=8849,nowait,id=121,config=/home/agmlab/jprofiler4/config/config.xml
-Xbootclasspath/a:/home/agmlab/jprofiler4/bin/agent.jar</value>
  <description>Java opts for the task tracker child processes.  Subsumes
  'mapred.child.heap.size' (If a mapred.child.heap.size value is found
  in a configuration, its maximum heap size will be used and a warning
  emitted that heap.size has been deprecated). Also, the following symbols,
  if present, will be interpolated: @taskid@ is replaced by current TaskID;
  and @port@ will be replaced by mapred.task.tracker.report.port + 1 (A
second
  child will fail with a port-in-use if mapred.tasktracker.tasks.maximum is
  greater than one). Any other occurrences of '@' will go unchanged. For
  example, to enable verbose gc logging to a file named for the taskid in
  /tmp and to set the heap maximum to be a gigabyte, pass a 'value' of:

        -Xmx1024m -verbose:gc -Xloggc:/tmp/@taskid@.gc
  </description>
</property>

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

When I submit a job, I run ps aux | grep java from the shell in
tasktracker, my child process related to crawl operation is given as:

*****************************************************************************************************************************

agmlab    8775  0.0  1.0 495460 22356 pts/1  Rl+  11:39   0:00
/opt/jdk1.5.0_07/jre/bin/java -Xmx200m -classpath
/home/agmlab/source/Hadoop/conf:/opt/jdk1.5.0_07/lib/tools.jar:/home/agmlab/source/Hadoop/build/classes
:/home/agmlab/source/Hadoop/build:/home/agmlab/source/Hadoop/build/test/classes:/home/agmlab/source/Hadoop:
/home/agmlab/source/Hadoop/hadoop-0.2.0-examples.jar:/home/agmlab/source/Hadoop/hadoop-0.2.0-test.jar
:/home/agmlab/source/Hadoop/hadoop-0.2.1.jar:/home/agmlab/source/Hadoop/lib/commons-logging-api-1.0.4.jar:
/home/agmlab/source/Hadoop/lib/jetty-5.1.4.jar:/home/agmlab/source/Hadoop/lib/junit-3.8.1.jar:
/home/agmlab/source/Hadoop/lib/lucene-core-1.9.1.jar:/home/agmlab/source/Hadoop/lib/servlet-api.jar:
/home/agmlab/source/Hadoop/lib/jetty-ext/ant.jar:/home/agmlab/source/Hadoop/lib/jetty-ext/commons-el.jar:
/home/agmlab/source/Hadoop/lib/jetty-ext/jasper-compiler.jar:/home/agmlab/source/Hadoop/lib/jetty-ext/jasper-runtime.jar
:/home/agmlab/source/Hadoop/lib/jetty-ext/jsp-api.jar:
:/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/jetty-ext:/home/agmlab/temp/local/taskTracker/
task_0010_m_000001_0/work/lib/pmd-ext:/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/
work/lib/lucene-core-1.9.1.jar
:/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/xerces-2_6_2-apis.jar
:/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/jetty-5.1.4.jar:/home/agmlab/temp/local/taskTracker
/task_0010_m_000001_0/work/lib/commons-codec-1.3.jar:
/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/xerces-2_6_2.jar
:/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/taglibs-i18n.jar:/home/agmlab/temp/local/
taskTracker/task_0010_m_000001_0/work/lib/commons-logging-api-1.0.4.jar:/home/agmlab/temp/local/taskTracker/
task_0010_m_000001_0/work/lib/je-2.0.83.jar
:/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/lucene-misc-1.9.1.jar:/home/agmlab/temp/local/
taskTracker/task_0010_m_000001_0/work/lib/xmlrpc-2.0.jar:/home/agmlab/temp/local/taskTracker/
task_0010_m_000001_0/work/lib/servlet.jar:/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/
concurrent-1.3.4.jar:
/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/jakarta-oro-2.0.7.jar
:/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/commons-lang-2.1.jar
:/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/servlet-api.jar:/home/agmlab/temp/local/
taskTracker/task_0010_m_000001_0/work/lib/junit-3.8.1.jar:/home/agmlab/
temp/local/taskTracker/task_0010_m_000001_0/work/classes:/home/agmlab/temp/local/taskTracker/
task_0010_m_000001_0/work org.apache.hadoop.mapred.TaskTracker$Child
18850 task_0010_m_000001_0
agmlab    8789  0.0  0.0 51088  664 pts/0    R+   11:39   0:00 grep java

***********************************************************************************************************************************
The options given below in XML document after -Xmx200m  does not work,
what am I doing wrong in this configurations?
-agentlib:jprofilerti=port=8849,nowait,id=121,config=/home/agmlab/jprofiler4/config/config.xml
-Xbootclasspath/a:/home/agmlab/jprofiler4/bin/agent.jar



Reply | Threaded
Open this post in threaded view
|

Re: Jprofiler compile options

Stefan Groschupf-2
Do you use java 1.4 or 1.5 ?
In general have a look to the hadoop code base: TaskRunner.java line:  
145.

Stefan

Am 05.06.2006 um 10:51 schrieb Murat Ali Bayir:

> Hi eveybody, I have problem in running Jprofiler in remote side.  I  
> am using DFS and submitting crawl job.
> I configure LD library path and hadoop-default.xml in machine which  
> I run hadoop tasktracker,
> in hadoop-default.xml, my java child compile option is given as:
>
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> ---------------------------------------------------------------
>
> <property>
>  <name>mapred.child.java.opts</name>
>  <value>-Xmx200m -
> agentlib:jprofilerti=port=8849,nowait,id=121,config=/home/agmlab/
> jprofiler4/config/config.xml -Xbootclasspath/a:/home/agmlab/
> jprofiler4/bin/agent.jar</value>
>  <description>Java opts for the task tracker child processes.  
> Subsumes
>  'mapred.child.heap.size' (If a mapred.child.heap.size value is found
>  in a configuration, its maximum heap size will be used and a warning
>  emitted that heap.size has been deprecated). Also, the following  
> symbols,
>  if present, will be interpolated: @taskid@ is replaced by current  
> TaskID;
>  and @port@ will be replaced by mapred.task.tracker.report.port + 1  
> (A second
>  child will fail with a port-in-use if  
> mapred.tasktracker.tasks.maximum is
>  greater than one). Any other occurrences of '@' will go unchanged.  
> For
>  example, to enable verbose gc logging to a file named for the  
> taskid in
>  /tmp and to set the heap maximum to be a gigabyte, pass a 'value' of:
>
>        -Xmx1024m -verbose:gc -Xloggc:/tmp/@taskid@.gc
>  </description>
> </property>
>
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> ---------------------------------------------------------------
>
> When I submit a job, I run ps aux | grep java from the shell in  
> tasktracker, my child process related to crawl operation is given as:
>
> **********************************************************************
> *******************************************************
>
> agmlab    8775  0.0  1.0 495460 22356 pts/1  Rl+  11:39   0:00 /opt/
> jdk1.5.0_07/jre/bin/java -Xmx200m -classpath /home/agmlab/source/
> Hadoop/conf:/opt/jdk1.5.0_07/lib/tools.jar:/home/agmlab/source/
> Hadoop/build/classes
> :/home/agmlab/source/Hadoop/build:/home/agmlab/source/Hadoop/build/
> test/classes:/home/agmlab/source/Hadoop:
> /home/agmlab/source/Hadoop/hadoop-0.2.0-examples.jar:/home/agmlab/
> source/Hadoop/hadoop-0.2.0-test.jar
> :/home/agmlab/source/Hadoop/hadoop-0.2.1.jar:/home/agmlab/source/
> Hadoop/lib/commons-logging-api-1.0.4.jar:
> /home/agmlab/source/Hadoop/lib/jetty-5.1.4.jar:/home/agmlab/source/
> Hadoop/lib/junit-3.8.1.jar:
> /home/agmlab/source/Hadoop/lib/lucene-core-1.9.1.jar:/home/agmlab/
> source/Hadoop/lib/servlet-api.jar:
> /home/agmlab/source/Hadoop/lib/jetty-ext/ant.jar:/home/agmlab/
> source/Hadoop/lib/jetty-ext/commons-el.jar:
> /home/agmlab/source/Hadoop/lib/jetty-ext/jasper-compiler.jar:/home/
> agmlab/source/Hadoop/lib/jetty-ext/jasper-runtime.jar
> :/home/agmlab/source/Hadoop/lib/jetty-ext/jsp-api.jar:
> :/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/
> jetty-ext:/home/agmlab/temp/local/taskTracker/
> task_0010_m_000001_0/work/lib/pmd-ext:/home/agmlab/temp/local/
> taskTracker/task_0010_m_000001_0/
> work/lib/lucene-core-1.9.1.jar
> :/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/
> xerces-2_6_2-apis.jar
> :/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/
> jetty-5.1.4.jar:/home/agmlab/temp/local/taskTracker
> /task_0010_m_000001_0/work/lib/commons-codec-1.3.jar:
> /home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/
> xerces-2_6_2.jar
> :/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/
> taglibs-i18n.jar:/home/agmlab/temp/local/
> taskTracker/task_0010_m_000001_0/work/lib/commons-logging-
> api-1.0.4.jar:/home/agmlab/temp/local/taskTracker/
> task_0010_m_000001_0/work/lib/je-2.0.83.jar
> :/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/
> lucene-misc-1.9.1.jar:/home/agmlab/temp/local/
> taskTracker/task_0010_m_000001_0/work/lib/xmlrpc-2.0.jar:/home/
> agmlab/temp/local/taskTracker/
> task_0010_m_000001_0/work/lib/servlet.jar:/home/agmlab/temp/local/
> taskTracker/task_0010_m_000001_0/work/lib/
> concurrent-1.3.4.jar:
> /home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/
> jakarta-oro-2.0.7.jar
> :/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/
> commons-lang-2.1.jar
> :/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/
> servlet-api.jar:/home/agmlab/temp/local/
> taskTracker/task_0010_m_000001_0/work/lib/junit-3.8.1.jar:/home/
> agmlab/
> temp/local/taskTracker/task_0010_m_000001_0/work/classes:/home/
> agmlab/temp/local/taskTracker/
> task_0010_m_000001_0/work org.apache.hadoop.mapred.TaskTracker
> $Child 18850 task_0010_m_000001_0
> agmlab    8789  0.0  0.0 51088  664 pts/0    R+   11:39   0:00 grep  
> java
>
> **********************************************************************
> *************************************************************
> The options given below in XML document after -Xmx200m  does not  
> work, what am I doing wrong in this configurations?
> -agentlib:jprofilerti=port=8849,nowait,id=121,config=/home/agmlab/
> jprofiler4/config/config.xml -Xbootclasspath/a:/home/agmlab/
> jprofiler4/bin/agent.jar
>
>
>
>