[jira] Created: (HADOOP-801) job tracker should keep a log of task completion and failure

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
job tracker should keep a log of task completion and failure
------------------------------------------------------------

                 Key: HADOOP-801
                 URL: http://issues.apache.org/jira/browse/HADOOP-801
             Project: Hadoop
          Issue Type: Improvement
          Components: mapred
    Affects Versions: 0.9.1
            Reporter: Owen O'Malley
         Assigned To: Sanjay Dahiya
             Fix For: 0.10.0


The JobTracker should track a list of task completion events in JobInProgress.

So JobClientProtocol & InterTrackerProtocol should get a new method:

  TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;

TaskCompletionEvent should have:
  int getEventId();
  String getTaskTrackerHttp();
  String getTaskId();
  static public enum Status {FAILED, SUCCEEDED};
  Status getTaskStatus();
}

The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.

These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org

     [ http://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sanjay Dahiya updated HADOOP-801:
---------------------------------

    Attachment: Hadoop-801.patch

> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: http://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.10.0
>
>         Attachments: Hadoop-801.patch
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ http://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12461759 ]

Sanjay Dahiya commented on HADOOP-801:
--------------------------------------

This patch includes fixes for HADOOP-801 and HADOOP-805. up for review.

> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: http://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.10.0
>
>         Attachments: Hadoop-801.patch
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ http://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sanjay Dahiya updated HADOOP-801:
---------------------------------

    Attachment:     (was: Hadoop-801.patch)

> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: http://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.10.0
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ http://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sanjay Dahiya updated HADOOP-801:
---------------------------------

    Comment: was deleted

> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: http://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.10.0
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sanjay Dahiya updated HADOOP-801:
---------------------------------

    Attachment: Hadoop-801.patch

> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: https://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.10.0
>
>         Attachments: Hadoop-801.patch
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12461941 ]

Sanjay Dahiya commented on HADOOP-801:
--------------------------------------

This patch includes
- New method in jobtracker, IntertrackerProtocol, JobSubmissionProtocol, JobInProgress, JobClient.RunningJob for taskCompletion events.
- JobClient checks for Task completion events in the status loop and prints out.
- TaskOuputFilter in JobClient to see only SUCCEEDED, FAILED or NONE filtered messages.
- Dummy method in localJobRunner for interface compatibility.
- JobInProgress creates/maintains TaskCompletion events on receiving task status from TaskTrackers.

It doesnt include
- Tasks stdout/stderr channelling back to job tracker. That will be a subsequent patch.

> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: https://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.10.0
>
>         Attachments: Hadoop-801.patch
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sanjay Dahiya updated HADOOP-801:
---------------------------------

    Attachment: Hadoop-801.patch

> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: https://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.10.0
>
>         Attachments: Hadoop-801.patch, Hadoop-801.patch
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12462254 ]

Sanjay Dahiya commented on HADOOP-801:
--------------------------------------

This patch adds functionality to download task logs from respective tasktrackers in jobclient. Also it modifies tasklog.jsp to take an extra argument which produces plain text output for console display. Since the changes are in a few places so I have avoided adding a new jsp file for plain text output and added the checks in tasklog.jsp itself.

> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: https://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.10.0
>
>         Attachments: Hadoop-801.patch, Hadoop-801.patch
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12462336 ]

Owen O'Malley commented on HADOOP-801:
--------------------------------------

1. I don't think it is a good idea to break control constructs between jsp escapes.:

   <% if (flag) { %>
       random text
   <% } %>

seems like bad style. I'd rather see:
<% if (flag) { out.write("random text"); } %>

2. taskCompletionEvents should be initialized to a capacity of maps + reduces + 10 instead of the default 10.

3. JobInProgress line ~294 has duplicated code for adding the new event depending on whether the task failed or succeeded. It should just be:
  taskCompletionEvents.add(..., (state == TaskStatus.Status.SUCCEEDED ?
                                                          TaskCompletionEvent.Status.SUCCEEDED :
                                                          TaskCompletionEvent.Status.FAILED), ... );
4. both  JobInProgress.getTaskCompletionEvents and JobTracker.getTaskCompletionEvents always creates a 0 length array of TaskCompletionEvents. It should either be a static final EMPTY_TASK_COMPLETION_EVENTS or only allocated when there are no events. We could use null to signal no events, but that is less user friendly.

5. the tasklog.jsp should check to make sure that plaintext is actually a boolean and support setting false. (Currently, if it is set at all, it assumes it is true.)

6. the url in the TaskCompletionEvent should include the taskid.

7. printHttpFile depends on content length being set, but the jsp doesn't set the content length.

8. the error message in printHttpFile was copied out of its context from MapOutputLocation.getFile and talks about map output



> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: https://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.10.0
>
>         Attachments: Hadoop-801.patch, Hadoop-801.patch
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12462345 ]

Doug Cutting commented on HADOOP-801:
-------------------------------------

> I don't think it is a good idea to break control constructs between jsp escapes.

You could use JSTL, no?

<c:if test="${flag}">
 random text
</c:if>


> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: https://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.10.0
>
>         Attachments: Hadoop-801.patch, Hadoop-801.patch
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sanjay Dahiya updated HADOOP-801:
---------------------------------

    Attachment: Hadoop-801.patch

> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: https://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.10.0
>
>         Attachments: Hadoop-801.patch, Hadoop-801.patch, Hadoop-801.patch
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12462482 ]

Sanjay Dahiya commented on HADOOP-801:
--------------------------------------

Updated patch with review comments, Thanks Owen for the review.

> 3. JobInProgress line ~294 has duplicated code for adding the new event depending on whether the task failed > or succeeded. It should just be:

TaskStatus.Status has a few more state transitions, we only want to generate an event when Task is done so think that piece of code needs to be wrapped in condition. so i think that should be ok.

> 1. I don't think it is a good idea to break control constructs between jsp escapes.:
Another issue when we use out.print() for more html output is that IDE can't do any syntax validation and all issues will be discovered at runtime. In this case its a small file so i guess its ok to use out.print()

> You could use JSTL, no?
That would be ideal, but we dont have jstl jars as part of hadoop as of now, we can defer it until we successfully move to jetty6 which comes with this stuff pre packaged.



> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: https://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.10.0
>
>         Attachments: Hadoop-801.patch, Hadoop-801.patch, Hadoop-801.patch
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Owen O'Malley updated HADOOP-801:
---------------------------------

    Fix Version/s: 0.11.0

> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: https://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.11.0
>
>         Attachments: Hadoop-801.patch, Hadoop-801.patch, Hadoop-801.patch
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12463773 ]

Owen O'Malley commented on HADOOP-801:
--------------------------------------

+1

> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: https://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.11.0
>
>         Attachments: Hadoop-801.patch, Hadoop-801.patch, Hadoop-801.patch
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sanjay Dahiya updated HADOOP-801:
---------------------------------

    Status: Patch Available  (was: Open)

> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: https://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.11.0
>
>         Attachments: Hadoop-801.patch, Hadoop-801.patch, Hadoop-801.patch
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12465085 ]

Hadoop QA commented on HADOOP-801:
----------------------------------

-1, because the javadoc command appears to have generated warning messages when testing the latest attachment (http://issues.apache.org/jira/secure/attachment/12348359/Hadoop-801.patch) against trunk revision r495045. Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.

> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: https://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.11.0
>
>         Attachments: Hadoop-801.patch, Hadoop-801.patch, Hadoop-801.patch
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12465133 ]

Sanjay Dahiya commented on HADOOP-801:
--------------------------------------

Fixed javadoc warnings.

> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: https://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.11.0
>
>         Attachments: Hadoop-801.patch, Hadoop-801.patch, Hadoop-801.patch, Hadoop-801.patch
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sanjay Dahiya updated HADOOP-801:
---------------------------------

    Attachment: Hadoop-801.patch

> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: https://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.11.0
>
>         Attachments: Hadoop-801.patch, Hadoop-801.patch, Hadoop-801.patch, Hadoop-801.patch
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-801) job tracker should keep a log of task completion and failure

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/HADOOP-801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12465237 ]

Hadoop QA commented on HADOOP-801:
----------------------------------

+1, because http://issues.apache.org/jira/secure/attachment/12349007/Hadoop-801.patch applied and successfully tested against trunk revision r495045.

> job tracker should keep a log of task completion and failure
> ------------------------------------------------------------
>
>                 Key: HADOOP-801
>                 URL: https://issues.apache.org/jira/browse/HADOOP-801
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.9.1
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>             Fix For: 0.11.0
>
>         Attachments: Hadoop-801.patch, Hadoop-801.patch, Hadoop-801.patch, Hadoop-801.patch
>
>
> The JobTracker should track a list of task completion events in JobInProgress.
> So JobClientProtocol & InterTrackerProtocol should get a new method:
>   TaskCompletionEvent[] getTaskCompletionEvents(String jobid, int fromEventId) throws IOException;
> TaskCompletionEvent should have:
>   int getEventId();
>   String getTaskTrackerHttp();
>   String getTaskId();
>   static public enum Status {FAILED, SUCCEEDED};
>   Status getTaskStatus();
> }
> The events will be stored in a List<TaskCompletionEvent> and the eventId is the position in the list.
> These event logs will allow JobClient to display task output to the user as well as provide the start of the fix for HADOOP-248.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
12