[jira] Created: (HADOOP-1508) ant Task for FsShell operations

classic Classic list List threaded Threaded
34 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
ant Task for FsShell operations
-------------------------------

                 Key: HADOOP-1508
                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
             Project: Hadoop
          Issue Type: New Feature
          Components: build, fs
            Reporter: Chris Douglas
            Priority: Minor


This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated HADOOP-1508:
----------------------------------

    Status: Patch Available  (was: Open)

This patch provides the first version of an ant Task wrapping FsShell.

In addition to providing a pass-through for FsShell commands, it also implements test and touch in FsShell. The former remains an undocumented command.

Several issues remain to be resolved.

1) It will only work for developer builds. It needs to be repackaged to run from a jar on ant's classpath, taking the location of hadoop jars as a parameter.
2) Since it's wrapping an exec-like interface, it provides special handling for test so if, unless, etc. in ant work as expected. The way it does it is only marginally reusable. It might be useful to refactor FsShell to provide a cleaner interface.
3) Configuration files, etc. must be on the local file system
4) A facility for testing this- that also starts and stops a single namenode/datanode- would be useful

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Priority: Minor
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
In reply to this post by Hudson (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated HADOOP-1508:
----------------------------------

    Attachment: HADOOP-1508.diff

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Priority: Minor
>         Attachments: HADOOP-1508.diff
>
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
In reply to this post by Hudson (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated HADOOP-1508:
----------------------------------

    Attachment: HADOOP-1508.diff

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Priority: Minor
>         Attachments: HADOOP-1508.diff
>
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
In reply to this post by Hudson (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated HADOOP-1508:
----------------------------------

    Attachment:     (was: HADOOP-1508.diff)

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Priority: Minor
>         Attachments: HADOOP-1508.diff
>
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
In reply to this post by Hudson (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated HADOOP-1508:
----------------------------------

    Attachment:     (was: HADOOP-1508.diff)

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Priority: Minor
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
In reply to this post by Hudson (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated HADOOP-1508:
----------------------------------

    Attachment: HADOOP-1508.patch

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Priority: Minor
>         Attachments: HADOOP-1508.patch
>
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
In reply to this post by Hudson (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12506341 ]

Hadoop QA commented on HADOOP-1508:
-----------------------------------

-1, could not apply patch.

The patch command could not apply the latest attachment http://issues.apache.org/jira/secure/attachment/12360159/HADOOP-1508.diff as a patch to trunk revision r548794.

Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/307/console

Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Priority: Minor
>         Attachments: HADOOP-1508.patch
>
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
In reply to this post by Hudson (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12506408 ]

dhruba borthakur commented on HADOOP-1508:
------------------------------------------

1. We need unit tests for the new functionality in FsShell. Please enhance TestDFSShell.java.
2. HADOOP-1377 implements file creation time. The 'touch' command can create an empty file and an application can find the time of creation by looking at the creation time of that file. This approach is similar to what Linux/UNIX supports.

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Priority: Minor
>         Attachments: HADOOP-1508.patch
>
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
In reply to this post by Hudson (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12506411 ]

dhruba borthakur commented on HADOOP-1508:
------------------------------------------

3. The methods test() and touch() in FsShell.java should be package private instead of public.
4. The explanation for touch says that "if the file already exists, then it is overwritten".  But files in DFSland cannot be overwritten. The code seems to thrown an exception if the file already exists.
5. The semantics to touch might need some explanation. For example, the FsShell first sees if the file exists using fs.exists(). If it does not then it tries to create the file. But there is no locking between the exists() and create(). Is this sufficient?
6. The method FsShell.test() is supposed to return an int. However, it is not returning any value. This should have resulted in a compilation warning, unless I am missing something.
7. Why is the 'test' command secret and not documented?

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Priority: Minor
>         Attachments: HADOOP-1508.patch
>
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
In reply to this post by Hudson (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Doug Cutting updated HADOOP-1508:
---------------------------------

    Assignee: Chris Douglas
      Status: Open  (was: Patch Available)

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>            Priority: Minor
>         Attachments: HADOOP-1508.patch
>
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
In reply to this post by Hudson (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated HADOOP-1508:
----------------------------------

    Attachment:     (was: HADOOP-1508.patch)

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>            Priority: Minor
>         Attachments: HADOOP-1508.patch
>
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
In reply to this post by Hudson (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated HADOOP-1508:
----------------------------------

    Attachment: HADOOP-1508.patch

Revised to address Dhruba's comments

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>            Priority: Minor
>         Attachments: HADOOP-1508.patch
>
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
In reply to this post by Hudson (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated HADOOP-1508:
----------------------------------

    Attachment: HADOOP-1508.patch

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>            Priority: Minor
>         Attachments: HADOOP-1508.patch
>
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
In reply to this post by Hudson (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated HADOOP-1508:
----------------------------------

    Status: Patch Available  (was: Open)

Added support for org.apache.tools.ant.condition.Condition and globs for stat

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>            Priority: Minor
>         Attachments: HADOOP-1508.patch
>
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
In reply to this post by Hudson (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated HADOOP-1508:
----------------------------------

    Attachment:     (was: HADOOP-1508.patch)

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>            Priority: Minor
>         Attachments: HADOOP-1508.patch
>
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
In reply to this post by Hudson (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated HADOOP-1508:
----------------------------------

    Status: Patch Available  (was: Open)

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>            Priority: Minor
>         Attachments: HADOOP-1508.patch
>
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
In reply to this post by Hudson (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated HADOOP-1508:
----------------------------------

    Status: Open  (was: Patch Available)

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>            Priority: Minor
>         Attachments: HADOOP-1508.patch
>
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
In reply to this post by Hudson (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12508038 ]

Hadoop QA commented on HADOOP-1508:
-----------------------------------

-1, build or testing failed

2 attempts failed to build and test the latest attachment http://issues.apache.org/jira/secure/attachment/12360528/HADOOP-1508.patch against trunk revision r550635.

Test results:   http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/329/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/329/console

Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>            Priority: Minor
>         Attachments: HADOOP-1508.patch
>
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1508) ant Task for FsShell operations

Hudson (Jira)
In reply to this post by Hudson (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12508046 ]

Hadoop QA commented on HADOOP-1508:
-----------------------------------

-1, build or testing failed

2 attempts failed to build and test the latest attachment http://issues.apache.org/jira/secure/attachment/12360528/HADOOP-1508.patch against trunk revision r550635.

Test results:   http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/330/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/330/console

Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.

> ant Task for FsShell operations
> -------------------------------
>
>                 Key: HADOOP-1508
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1508
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: build, fs
>            Reporter: Chris Douglas
>            Assignee: Chris Douglas
>            Priority: Minor
>         Attachments: HADOOP-1508.patch
>
>
> This issue will document the requirements, design and implementation of an ant Task providing FsShell functionality within that framework.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

12