[jira] Created: (HADOOP-804) Cut down on the "mumbling" in the Task process' stdout/stderr

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (HADOOP-804) Cut down on the "mumbling" in the Task process' stdout/stderr

JIRA jira@apache.org
Cut down on the "mumbling" in the Task process' stdout/stderr
-------------------------------------------------------------

                 Key: HADOOP-804
                 URL: http://issues.apache.org/jira/browse/HADOOP-804
             Project: Hadoop
          Issue Type: Improvement
          Components: mapred
            Reporter: Owen O'Malley
         Assigned To: Sanjay Dahiya
             Fix For: 0.10.0


Currently there is a lot of useless mumbling that Tasks do when running that needs to be removed. For example, here is a list of the output from a map:

06/12/07 16:45:48 INFO mapred.TaskTracker: Child starting
06/12/07 16:45:48 INFO conf.Configuration: parsing jar:file:/local/owen/hadoop-0.8.1-dev/hadoop-0.8.1-dev.jar!/hadoop-default.xml
06/12/07 16:45:48 INFO conf.Configuration: parsing file:/local/owen/conf/mapred-default.xml
06/12/07 16:45:48 INFO ipc.Client: org.apache.hadoop.io.ObjectWritableConnection culler maxidletime= 1000ms
06/12/07 16:45:48 INFO ipc.Client: org.apache.hadoop.io.ObjectWritable Connection Culler: starting
06/12/07 16:45:48 INFO conf.Configuration: parsing jar:file:/local/owen/hadoop-0.8.1-dev/hadoop-0.8.1-dev.jar!/hadoop-default.xml
06/12/07 16:45:48 INFO conf.Configuration: parsing file:/local/owen/conf/mapred-default.xml
06/12/07 16:45:48 INFO conf.Configuration: parsing /local/owen/run/tasktracker/taskTracker/jobcache/job_0002/task_0002_m_000000_0/job.xml
06/12/07 16:45:49 INFO conf.Configuration: parsing jar:file:/local/owen/hadoop-0.8.1-dev/hadoop-0.8.1-dev.jar!/hadoop-default.xml
06/12/07 16:45:49 INFO conf.Configuration: parsing file:/local/owen/conf/mapred-default.xml
06/12/07 16:45:49 INFO mapred.MapTask: opened part-0.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-1.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-2.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-3.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-4.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-5.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-6.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-7.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-8.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-9.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-10.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-11.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-12.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-13.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-14.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-15.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-16.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-17.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-18.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-19.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-20.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-21.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-22.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-23.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-24.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-25.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-26.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-27.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-28.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-29.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-30.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-31.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-32.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-33.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-34.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-35.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-36.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-37.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-38.out
06/12/07 16:45:49 INFO mapred.MapTask: opened part-39.out

Furthermore, it seems like the default logging level for Task subprocesses should be set to WARN.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-804) Cut down on the "mumbling" in the Task process' stdout/stderr

JIRA jira@apache.org
    [ http://issues.apache.org/jira/browse/HADOOP-804?page=comments#action_12458915 ]
           
Doug Cutting commented on HADOOP-804:
-------------------------------------

I don't think we should change the default log level, but rather change most of these to be DEBUG level.  I think the understanding is that INFO and above messages will normally be logged.  WARN and above are candidates for special handling--i.e, also reported and/or counted centrally, but we should not generally quash INFO log messages.

> Cut down on the "mumbling" in the Task process' stdout/stderr
> -------------------------------------------------------------
>
>                 Key: HADOOP-804
>                 URL: http://issues.apache.org/jira/browse/HADOOP-804
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.10.0
>
>
> Currently there is a lot of useless mumbling that Tasks do when running that needs to be removed. For example, here is a list of the output from a map:
> 06/12/07 16:45:48 INFO mapred.TaskTracker: Child starting
> 06/12/07 16:45:48 INFO conf.Configuration: parsing jar:file:/local/owen/hadoop-0.8.1-dev/hadoop-0.8.1-dev.jar!/hadoop-default.xml
> 06/12/07 16:45:48 INFO conf.Configuration: parsing file:/local/owen/conf/mapred-default.xml
> 06/12/07 16:45:48 INFO ipc.Client: org.apache.hadoop.io.ObjectWritableConnection culler maxidletime= 1000ms
> 06/12/07 16:45:48 INFO ipc.Client: org.apache.hadoop.io.ObjectWritable Connection Culler: starting
> 06/12/07 16:45:48 INFO conf.Configuration: parsing jar:file:/local/owen/hadoop-0.8.1-dev/hadoop-0.8.1-dev.jar!/hadoop-default.xml
> 06/12/07 16:45:48 INFO conf.Configuration: parsing file:/local/owen/conf/mapred-default.xml
> 06/12/07 16:45:48 INFO conf.Configuration: parsing /local/owen/run/tasktracker/taskTracker/jobcache/job_0002/task_0002_m_000000_0/job.xml
> 06/12/07 16:45:49 INFO conf.Configuration: parsing jar:file:/local/owen/hadoop-0.8.1-dev/hadoop-0.8.1-dev.jar!/hadoop-default.xml
> 06/12/07 16:45:49 INFO conf.Configuration: parsing file:/local/owen/conf/mapred-default.xml
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-0.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-1.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-2.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-3.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-4.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-5.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-6.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-7.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-8.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-9.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-10.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-11.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-12.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-13.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-14.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-15.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-16.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-17.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-18.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-19.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-20.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-21.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-22.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-23.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-24.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-25.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-26.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-27.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-28.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-29.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-30.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-31.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-32.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-33.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-34.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-35.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-36.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-37.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-38.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-39.out
> Furthermore, it seems like the default logging level for Task subprocesses should be set to WARN.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-804) Cut down on the "mumbling" in the Task process' stdout/stderr

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ http://issues.apache.org/jira/browse/HADOOP-804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sanjay Dahiya updated HADOOP-804:
---------------------------------

    Attachment: Hadoop-804.patch

> Cut down on the "mumbling" in the Task process' stdout/stderr
> -------------------------------------------------------------
>
>                 Key: HADOOP-804
>                 URL: http://issues.apache.org/jira/browse/HADOOP-804
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.10.0
>
>         Attachments: Hadoop-804.patch
>
>
> Currently there is a lot of useless mumbling that Tasks do when running that needs to be removed. For example, here is a list of the output from a map:
> 06/12/07 16:45:48 INFO mapred.TaskTracker: Child starting
> 06/12/07 16:45:48 INFO conf.Configuration: parsing jar:file:/local/owen/hadoop-0.8.1-dev/hadoop-0.8.1-dev.jar!/hadoop-default.xml
> 06/12/07 16:45:48 INFO conf.Configuration: parsing file:/local/owen/conf/mapred-default.xml
> 06/12/07 16:45:48 INFO ipc.Client: org.apache.hadoop.io.ObjectWritableConnection culler maxidletime= 1000ms
> 06/12/07 16:45:48 INFO ipc.Client: org.apache.hadoop.io.ObjectWritable Connection Culler: starting
> 06/12/07 16:45:48 INFO conf.Configuration: parsing jar:file:/local/owen/hadoop-0.8.1-dev/hadoop-0.8.1-dev.jar!/hadoop-default.xml
> 06/12/07 16:45:48 INFO conf.Configuration: parsing file:/local/owen/conf/mapred-default.xml
> 06/12/07 16:45:48 INFO conf.Configuration: parsing /local/owen/run/tasktracker/taskTracker/jobcache/job_0002/task_0002_m_000000_0/job.xml
> 06/12/07 16:45:49 INFO conf.Configuration: parsing jar:file:/local/owen/hadoop-0.8.1-dev/hadoop-0.8.1-dev.jar!/hadoop-default.xml
> 06/12/07 16:45:49 INFO conf.Configuration: parsing file:/local/owen/conf/mapred-default.xml
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-0.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-1.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-2.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-3.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-4.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-5.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-6.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-7.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-8.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-9.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-10.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-11.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-12.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-13.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-14.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-15.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-16.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-17.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-18.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-19.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-20.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-21.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-22.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-23.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-24.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-25.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-26.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-27.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-28.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-29.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-30.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-31.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-32.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-33.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-34.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-35.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-36.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-37.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-38.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-39.out
> Furthermore, it seems like the default logging level for Task subprocesses should be set to WARN.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-804) Cut down on the "mumbling" in the Task process' stdout/stderr

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ http://issues.apache.org/jira/browse/HADOOP-804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12461694 ]

Sanjay Dahiya commented on HADOOP-804:
--------------------------------------

this patch
1. changes some of the log levels to DEBUG in tasks.
2. changes Configuration default to use quiet mode.

> Cut down on the "mumbling" in the Task process' stdout/stderr
> -------------------------------------------------------------
>
>                 Key: HADOOP-804
>                 URL: http://issues.apache.org/jira/browse/HADOOP-804
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.10.0
>
>         Attachments: Hadoop-804.patch
>
>
> Currently there is a lot of useless mumbling that Tasks do when running that needs to be removed. For example, here is a list of the output from a map:
> 06/12/07 16:45:48 INFO mapred.TaskTracker: Child starting
> 06/12/07 16:45:48 INFO conf.Configuration: parsing jar:file:/local/owen/hadoop-0.8.1-dev/hadoop-0.8.1-dev.jar!/hadoop-default.xml
> 06/12/07 16:45:48 INFO conf.Configuration: parsing file:/local/owen/conf/mapred-default.xml
> 06/12/07 16:45:48 INFO ipc.Client: org.apache.hadoop.io.ObjectWritableConnection culler maxidletime= 1000ms
> 06/12/07 16:45:48 INFO ipc.Client: org.apache.hadoop.io.ObjectWritable Connection Culler: starting
> 06/12/07 16:45:48 INFO conf.Configuration: parsing jar:file:/local/owen/hadoop-0.8.1-dev/hadoop-0.8.1-dev.jar!/hadoop-default.xml
> 06/12/07 16:45:48 INFO conf.Configuration: parsing file:/local/owen/conf/mapred-default.xml
> 06/12/07 16:45:48 INFO conf.Configuration: parsing /local/owen/run/tasktracker/taskTracker/jobcache/job_0002/task_0002_m_000000_0/job.xml
> 06/12/07 16:45:49 INFO conf.Configuration: parsing jar:file:/local/owen/hadoop-0.8.1-dev/hadoop-0.8.1-dev.jar!/hadoop-default.xml
> 06/12/07 16:45:49 INFO conf.Configuration: parsing file:/local/owen/conf/mapred-default.xml
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-0.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-1.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-2.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-3.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-4.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-5.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-6.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-7.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-8.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-9.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-10.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-11.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-12.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-13.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-14.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-15.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-16.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-17.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-18.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-19.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-20.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-21.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-22.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-23.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-24.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-25.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-26.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-27.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-28.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-29.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-30.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-31.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-32.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-33.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-34.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-35.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-36.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-37.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-38.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-39.out
> Furthermore, it seems like the default logging level for Task subprocesses should be set to WARN.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-804) Cut down on the "mumbling" in the Task process' stdout/stderr

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/HADOOP-804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12462073 ]

Owen O'Malley commented on HADOOP-804:
--------------------------------------

+1

> Cut down on the "mumbling" in the Task process' stdout/stderr
> -------------------------------------------------------------
>
>                 Key: HADOOP-804
>                 URL: https://issues.apache.org/jira/browse/HADOOP-804
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.10.0
>
>         Attachments: Hadoop-804.patch
>
>
> Currently there is a lot of useless mumbling that Tasks do when running that needs to be removed. For example, here is a list of the output from a map:
> 06/12/07 16:45:48 INFO mapred.TaskTracker: Child starting
> 06/12/07 16:45:48 INFO conf.Configuration: parsing jar:file:/local/owen/hadoop-0.8.1-dev/hadoop-0.8.1-dev.jar!/hadoop-default.xml
> 06/12/07 16:45:48 INFO conf.Configuration: parsing file:/local/owen/conf/mapred-default.xml
> 06/12/07 16:45:48 INFO ipc.Client: org.apache.hadoop.io.ObjectWritableConnection culler maxidletime= 1000ms
> 06/12/07 16:45:48 INFO ipc.Client: org.apache.hadoop.io.ObjectWritable Connection Culler: starting
> 06/12/07 16:45:48 INFO conf.Configuration: parsing jar:file:/local/owen/hadoop-0.8.1-dev/hadoop-0.8.1-dev.jar!/hadoop-default.xml
> 06/12/07 16:45:48 INFO conf.Configuration: parsing file:/local/owen/conf/mapred-default.xml
> 06/12/07 16:45:48 INFO conf.Configuration: parsing /local/owen/run/tasktracker/taskTracker/jobcache/job_0002/task_0002_m_000000_0/job.xml
> 06/12/07 16:45:49 INFO conf.Configuration: parsing jar:file:/local/owen/hadoop-0.8.1-dev/hadoop-0.8.1-dev.jar!/hadoop-default.xml
> 06/12/07 16:45:49 INFO conf.Configuration: parsing file:/local/owen/conf/mapred-default.xml
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-0.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-1.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-2.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-3.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-4.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-5.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-6.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-7.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-8.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-9.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-10.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-11.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-12.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-13.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-14.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-15.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-16.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-17.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-18.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-19.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-20.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-21.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-22.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-23.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-24.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-25.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-26.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-27.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-28.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-29.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-30.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-31.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-32.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-33.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-34.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-35.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-36.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-37.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-38.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-39.out
> Furthermore, it seems like the default logging level for Task subprocesses should be set to WARN.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Resolved: (HADOOP-804) Cut down on the "mumbling" in the Task process' stdout/stderr

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/HADOOP-804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Doug Cutting resolved HADOOP-804.
---------------------------------

    Resolution: Fixed

I just committed this.  Thanks, Sanjay!

> Cut down on the "mumbling" in the Task process' stdout/stderr
> -------------------------------------------------------------
>
>                 Key: HADOOP-804
>                 URL: https://issues.apache.org/jira/browse/HADOOP-804
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.10.0
>
>         Attachments: Hadoop-804.patch
>
>
> Currently there is a lot of useless mumbling that Tasks do when running that needs to be removed. For example, here is a list of the output from a map:
> 06/12/07 16:45:48 INFO mapred.TaskTracker: Child starting
> 06/12/07 16:45:48 INFO conf.Configuration: parsing jar:file:/local/owen/hadoop-0.8.1-dev/hadoop-0.8.1-dev.jar!/hadoop-default.xml
> 06/12/07 16:45:48 INFO conf.Configuration: parsing file:/local/owen/conf/mapred-default.xml
> 06/12/07 16:45:48 INFO ipc.Client: org.apache.hadoop.io.ObjectWritableConnection culler maxidletime= 1000ms
> 06/12/07 16:45:48 INFO ipc.Client: org.apache.hadoop.io.ObjectWritable Connection Culler: starting
> 06/12/07 16:45:48 INFO conf.Configuration: parsing jar:file:/local/owen/hadoop-0.8.1-dev/hadoop-0.8.1-dev.jar!/hadoop-default.xml
> 06/12/07 16:45:48 INFO conf.Configuration: parsing file:/local/owen/conf/mapred-default.xml
> 06/12/07 16:45:48 INFO conf.Configuration: parsing /local/owen/run/tasktracker/taskTracker/jobcache/job_0002/task_0002_m_000000_0/job.xml
> 06/12/07 16:45:49 INFO conf.Configuration: parsing jar:file:/local/owen/hadoop-0.8.1-dev/hadoop-0.8.1-dev.jar!/hadoop-default.xml
> 06/12/07 16:45:49 INFO conf.Configuration: parsing file:/local/owen/conf/mapred-default.xml
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-0.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-1.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-2.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-3.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-4.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-5.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-6.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-7.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-8.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-9.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-10.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-11.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-12.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-13.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-14.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-15.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-16.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-17.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-18.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-19.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-20.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-21.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-22.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-23.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-24.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-25.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-26.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-27.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-28.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-29.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-30.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-31.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-32.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-33.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-34.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-35.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-36.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-37.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-38.out
> 06/12/07 16:45:49 INFO mapred.MapTask: opened part-39.out
> Furthermore, it seems like the default logging level for Task subprocesses should be set to WARN.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira