[jira] Created: (HADOOP-1518) Add session id to metric data

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (HADOOP-1518) Add session id to metric data

ASF GitHub Bot (Jira)
Add session id to metric data
-----------------------------

                 Key: HADOOP-1518
                 URL: https://issues.apache.org/jira/browse/HADOOP-1518
             Project: Hadoop
          Issue Type: Improvement
            Reporter: David Bowen
            Assignee: David Bowen



In the context of Hadoop-On-Demand (HOD), we want to be able to aggregate metrics across the machines in the virtual cluster that corresponds to a HOD session. To enable this, we'd like to tag all the map/reduce related metrics with a session id.  The session id would be a string created by HOD (it might be a Torque job id) and put into the hadoop-site.xml file for the virtual cluster.

The default value for the session id would be the empty string, so that non-HOD usage of the metrics is not affected accept for the addition of a tag which is always the empty string.





--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1518) Add session id to metric data

ASF GitHub Bot (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

David Bowen updated HADOOP-1518:
--------------------------------

    Status: Patch Available  (was: Open)

> Add session id to metric data
> -----------------------------
>
>                 Key: HADOOP-1518
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1518
>             Project: Hadoop
>          Issue Type: Improvement
>            Reporter: David Bowen
>            Assignee: David Bowen
>         Attachments: patch.txt
>
>
> In the context of Hadoop-On-Demand (HOD), we want to be able to aggregate metrics across the machines in the virtual cluster that corresponds to a HOD session. To enable this, we'd like to tag all the map/reduce related metrics with a session id.  The session id would be a string created by HOD (it might be a Torque job id) and put into the hadoop-site.xml file for the virtual cluster.
> The default value for the session id would be the empty string, so that non-HOD usage of the metrics is not affected accept for the addition of a tag which is always the empty string.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1518) Add session id to metric data

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

David Bowen updated HADOOP-1518:
--------------------------------

    Attachment: patch.txt


This patch adds the session id to all the map/reduce related metrics.



> Add session id to metric data
> -----------------------------
>
>                 Key: HADOOP-1518
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1518
>             Project: Hadoop
>          Issue Type: Improvement
>            Reporter: David Bowen
>            Assignee: David Bowen
>         Attachments: patch.txt
>
>
> In the context of Hadoop-On-Demand (HOD), we want to be able to aggregate metrics across the machines in the virtual cluster that corresponds to a HOD session. To enable this, we'd like to tag all the map/reduce related metrics with a session id.  The session id would be a string created by HOD (it might be a Torque job id) and put into the hadoop-site.xml file for the virtual cluster.
> The default value for the session id would be the empty string, so that non-HOD usage of the metrics is not affected accept for the addition of a tag which is always the empty string.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1518) Add session id to metric data

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12507029 ]

Hadoop QA commented on HADOOP-1518:
-----------------------------------

+1

http://issues.apache.org/jira/secure/attachment/12360319/patch.txt applied and successfully tested against trunk revision r549577.

Test results:   http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/319/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/319/console

> Add session id to metric data
> -----------------------------
>
>                 Key: HADOOP-1518
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1518
>             Project: Hadoop
>          Issue Type: Improvement
>            Reporter: David Bowen
>            Assignee: David Bowen
>         Attachments: patch.txt
>
>
> In the context of Hadoop-On-Demand (HOD), we want to be able to aggregate metrics across the machines in the virtual cluster that corresponds to a HOD session. To enable this, we'd like to tag all the map/reduce related metrics with a session id.  The session id would be a string created by HOD (it might be a Torque job id) and put into the hadoop-site.xml file for the virtual cluster.
> The default value for the session id would be the empty string, so that non-HOD usage of the metrics is not affected accept for the addition of a tag which is always the empty string.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1518) Add session id to metric data

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Doug Cutting updated HADOOP-1518:
---------------------------------

    Status: Open  (was: Patch Available)

The javadoc for set/getSessionId should describe what the session id is used for, as should the description in hadoop-default.xml.  Will this actually be set in site configurations, or only programatically?  If it's only set programatically, then it doesn't need to be in hadoop-default.xml, which primarily exists to document available parameters for site configuration, not necessarily every parameter ever used by programs.

> Add session id to metric data
> -----------------------------
>
>                 Key: HADOOP-1518
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1518
>             Project: Hadoop
>          Issue Type: Improvement
>            Reporter: David Bowen
>            Assignee: David Bowen
>         Attachments: patch.txt
>
>
> In the context of Hadoop-On-Demand (HOD), we want to be able to aggregate metrics across the machines in the virtual cluster that corresponds to a HOD session. To enable this, we'd like to tag all the map/reduce related metrics with a session id.  The session id would be a string created by HOD (it might be a Torque job id) and put into the hadoop-site.xml file for the virtual cluster.
> The default value for the session id would be the empty string, so that non-HOD usage of the metrics is not affected accept for the addition of a tag which is always the empty string.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1518) Add session id to metric data

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

David Bowen updated HADOOP-1518:
--------------------------------

    Attachment: 1518.patch

Improved patch.  Removed the change to hadoop-default.xml, and added comments to JobConf.getSessionId to explain the intended purpose of the session identifier.



> Add session id to metric data
> -----------------------------
>
>                 Key: HADOOP-1518
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1518
>             Project: Hadoop
>          Issue Type: Improvement
>            Reporter: David Bowen
>            Assignee: David Bowen
>         Attachments: 1518.patch, patch.txt
>
>
> In the context of Hadoop-On-Demand (HOD), we want to be able to aggregate metrics across the machines in the virtual cluster that corresponds to a HOD session. To enable this, we'd like to tag all the map/reduce related metrics with a session id.  The session id would be a string created by HOD (it might be a Torque job id) and put into the hadoop-site.xml file for the virtual cluster.
> The default value for the session id would be the empty string, so that non-HOD usage of the metrics is not affected accept for the addition of a tag which is always the empty string.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1518) Add session id to metric data

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

David Bowen updated HADOOP-1518:
--------------------------------

    Status: Patch Available  (was: Open)

> Add session id to metric data
> -----------------------------
>
>                 Key: HADOOP-1518
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1518
>             Project: Hadoop
>          Issue Type: Improvement
>            Reporter: David Bowen
>            Assignee: David Bowen
>         Attachments: 1518.patch, patch.txt
>
>
> In the context of Hadoop-On-Demand (HOD), we want to be able to aggregate metrics across the machines in the virtual cluster that corresponds to a HOD session. To enable this, we'd like to tag all the map/reduce related metrics with a session id.  The session id would be a string created by HOD (it might be a Torque job id) and put into the hadoop-site.xml file for the virtual cluster.
> The default value for the session id would be the empty string, so that non-HOD usage of the metrics is not affected accept for the addition of a tag which is always the empty string.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1518) Add session id to metric data

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12507069 ]

Hadoop QA commented on HADOOP-1518:
-----------------------------------

+1

http://issues.apache.org/jira/secure/attachment/12360329/1518.patch applied and successfully tested against trunk revision r549624.

Test results:   http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/321/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/321/console

> Add session id to metric data
> -----------------------------
>
>                 Key: HADOOP-1518
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1518
>             Project: Hadoop
>          Issue Type: Improvement
>            Reporter: David Bowen
>            Assignee: David Bowen
>         Attachments: 1518.patch, patch.txt
>
>
> In the context of Hadoop-On-Demand (HOD), we want to be able to aggregate metrics across the machines in the virtual cluster that corresponds to a HOD session. To enable this, we'd like to tag all the map/reduce related metrics with a session id.  The session id would be a string created by HOD (it might be a Torque job id) and put into the hadoop-site.xml file for the virtual cluster.
> The default value for the session id would be the empty string, so that non-HOD usage of the metrics is not affected accept for the addition of a tag which is always the empty string.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1518) Add session id to metric data

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Doug Cutting updated HADOOP-1518:
---------------------------------

       Resolution: Fixed
    Fix Version/s: 0.14.0
           Status: Resolved  (was: Patch Available)

I just committed this.  Thanks, David!

> Add session id to metric data
> -----------------------------
>
>                 Key: HADOOP-1518
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1518
>             Project: Hadoop
>          Issue Type: Improvement
>            Reporter: David Bowen
>            Assignee: David Bowen
>             Fix For: 0.14.0
>
>         Attachments: 1518.patch, patch.txt
>
>
> In the context of Hadoop-On-Demand (HOD), we want to be able to aggregate metrics across the machines in the virtual cluster that corresponds to a HOD session. To enable this, we'd like to tag all the map/reduce related metrics with a session id.  The session id would be a string created by HOD (it might be a Torque job id) and put into the hadoop-site.xml file for the virtual cluster.
> The default value for the session id would be the empty string, so that non-HOD usage of the metrics is not affected accept for the addition of a tag which is always the empty string.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1518) Add session id to metric data

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12507595 ]

Hudson commented on HADOOP-1518:
--------------------------------

Integrated in Hadoop-Nightly #133 (See [http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/133/])

> Add session id to metric data
> -----------------------------
>
>                 Key: HADOOP-1518
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1518
>             Project: Hadoop
>          Issue Type: Improvement
>            Reporter: David Bowen
>            Assignee: David Bowen
>             Fix For: 0.14.0
>
>         Attachments: 1518.patch, patch.txt
>
>
> In the context of Hadoop-On-Demand (HOD), we want to be able to aggregate metrics across the machines in the virtual cluster that corresponds to a HOD session. To enable this, we'd like to tag all the map/reduce related metrics with a session id.  The session id would be a string created by HOD (it might be a Torque job id) and put into the hadoop-site.xml file for the virtual cluster.
> The default value for the session id would be the empty string, so that non-HOD usage of the metrics is not affected accept for the addition of a tag which is always the empty string.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.