[jira] Created: (HADOOP-894) dfs client protocol should allow asking for parts of the block map

classic Classic list List threaded Threaded
56 messages Options
123
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-894) dfs client protocol should allow asking for parts of the block map

Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12496236 ]

Hadoop QA commented on HADOOP-894:
----------------------------------

+1

http://issues.apache.org/jira/secure/attachment/12357448/partialBlockList4.patch applied and successfully tested against trunk revision r538318.

Test results:   http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/148/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/148/console

> dfs client protocol should allow asking for parts of the block map
> ------------------------------------------------------------------
>
>                 Key: HADOOP-894
>                 URL: https://issues.apache.org/jira/browse/HADOOP-894
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.12.0
>            Reporter: Owen O'Malley
>         Assigned To: Konstantin Shvachko
>             Fix For: 0.13.0
>
>         Attachments: partialBlockList2.patch, partialBlockList3.patch, partialBlockList4.patch
>
>
> I think that the HDFS client protocol should change like:
> /** The meta-data about a file that was opened. */
> class OpenFileInfo {
>   /** the info for the first block */
>   public LocatedBlockInfo getBlockInfo();
>   public long getBlockSize();
>   public long getLength();
> }
> interface ClientProtocol extends VersionedProtocol {
>   public OpenFileInfo open(String name) throws IOException;
>   /** get block info for any range of blocks */
>   public LocatedBlockInfo[] getBlockInfo(String name, int blockOffset, int blockLength) throws IOException;
> }
> so that the client can decide how much block info to request and when. Currently, when the file is opened or an error occurs, the entire block list is requested and sent.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-894) dfs client protocol should allow asking for parts of the block map

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nigel Daley updated HADOOP-894:
-------------------------------

    Fix Version/s:     (was: 0.13.0)
                   0.14.0

Moving to 0.14 as this is not a bug.

> dfs client protocol should allow asking for parts of the block map
> ------------------------------------------------------------------
>
>                 Key: HADOOP-894
>                 URL: https://issues.apache.org/jira/browse/HADOOP-894
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.12.0
>            Reporter: Owen O'Malley
>         Assigned To: Konstantin Shvachko
>             Fix For: 0.14.0
>
>         Attachments: partialBlockList2.patch, partialBlockList3.patch, partialBlockList4.patch
>
>
> I think that the HDFS client protocol should change like:
> /** The meta-data about a file that was opened. */
> class OpenFileInfo {
>   /** the info for the first block */
>   public LocatedBlockInfo getBlockInfo();
>   public long getBlockSize();
>   public long getLength();
> }
> interface ClientProtocol extends VersionedProtocol {
>   public OpenFileInfo open(String name) throws IOException;
>   /** get block info for any range of blocks */
>   public LocatedBlockInfo[] getBlockInfo(String name, int blockOffset, int blockLength) throws IOException;
> }
> so that the client can decide how much block info to request and when. Currently, when the file is opened or an error occurs, the entire block list is requested and sent.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-894) dfs client protocol should allow asking for parts of the block map

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Doug Cutting updated HADOOP-894:
--------------------------------

    Status: Open  (was: Patch Available)

This patch no longer cleanly applies to trunk.  Sorry!  Can you please update it.  Thanks!

> dfs client protocol should allow asking for parts of the block map
> ------------------------------------------------------------------
>
>                 Key: HADOOP-894
>                 URL: https://issues.apache.org/jira/browse/HADOOP-894
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.12.0
>            Reporter: Owen O'Malley
>         Assigned To: Konstantin Shvachko
>             Fix For: 0.14.0
>
>         Attachments: partialBlockList2.patch, partialBlockList3.patch, partialBlockList4.patch
>
>
> I think that the HDFS client protocol should change like:
> /** The meta-data about a file that was opened. */
> class OpenFileInfo {
>   /** the info for the first block */
>   public LocatedBlockInfo getBlockInfo();
>   public long getBlockSize();
>   public long getLength();
> }
> interface ClientProtocol extends VersionedProtocol {
>   public OpenFileInfo open(String name) throws IOException;
>   /** get block info for any range of blocks */
>   public LocatedBlockInfo[] getBlockInfo(String name, int blockOffset, int blockLength) throws IOException;
> }
> so that the client can decide how much block info to request and when. Currently, when the file is opened or an error occurs, the entire block list is requested and sent.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-894) dfs client protocol should allow asking for parts of the block map

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Shvachko updated HADOOP-894:
---------------------------------------

    Attachment: partialBlockList5.patch

Updated the patch.

> dfs client protocol should allow asking for parts of the block map
> ------------------------------------------------------------------
>
>                 Key: HADOOP-894
>                 URL: https://issues.apache.org/jira/browse/HADOOP-894
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.12.0
>            Reporter: Owen O'Malley
>         Assigned To: Konstantin Shvachko
>             Fix For: 0.14.0
>
>         Attachments: partialBlockList2.patch, partialBlockList3.patch, partialBlockList5.patch
>
>
> I think that the HDFS client protocol should change like:
> /** The meta-data about a file that was opened. */
> class OpenFileInfo {
>   /** the info for the first block */
>   public LocatedBlockInfo getBlockInfo();
>   public long getBlockSize();
>   public long getLength();
> }
> interface ClientProtocol extends VersionedProtocol {
>   public OpenFileInfo open(String name) throws IOException;
>   /** get block info for any range of blocks */
>   public LocatedBlockInfo[] getBlockInfo(String name, int blockOffset, int blockLength) throws IOException;
> }
> so that the client can decide how much block info to request and when. Currently, when the file is opened or an error occurs, the entire block list is requested and sent.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-894) dfs client protocol should allow asking for parts of the block map

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Shvachko updated HADOOP-894:
---------------------------------------

    Attachment:     (was: partialBlockList4.patch)

> dfs client protocol should allow asking for parts of the block map
> ------------------------------------------------------------------
>
>                 Key: HADOOP-894
>                 URL: https://issues.apache.org/jira/browse/HADOOP-894
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.12.0
>            Reporter: Owen O'Malley
>         Assigned To: Konstantin Shvachko
>             Fix For: 0.14.0
>
>         Attachments: partialBlockList2.patch, partialBlockList3.patch, partialBlockList5.patch
>
>
> I think that the HDFS client protocol should change like:
> /** The meta-data about a file that was opened. */
> class OpenFileInfo {
>   /** the info for the first block */
>   public LocatedBlockInfo getBlockInfo();
>   public long getBlockSize();
>   public long getLength();
> }
> interface ClientProtocol extends VersionedProtocol {
>   public OpenFileInfo open(String name) throws IOException;
>   /** get block info for any range of blocks */
>   public LocatedBlockInfo[] getBlockInfo(String name, int blockOffset, int blockLength) throws IOException;
> }
> so that the client can decide how much block info to request and when. Currently, when the file is opened or an error occurs, the entire block list is requested and sent.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-894) dfs client protocol should allow asking for parts of the block map

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Shvachko updated HADOOP-894:
---------------------------------------

    Status: Patch Available  (was: Open)

> dfs client protocol should allow asking for parts of the block map
> ------------------------------------------------------------------
>
>                 Key: HADOOP-894
>                 URL: https://issues.apache.org/jira/browse/HADOOP-894
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.12.0
>            Reporter: Owen O'Malley
>         Assigned To: Konstantin Shvachko
>             Fix For: 0.14.0
>
>         Attachments: partialBlockList2.patch, partialBlockList3.patch, partialBlockList5.patch
>
>
> I think that the HDFS client protocol should change like:
> /** The meta-data about a file that was opened. */
> class OpenFileInfo {
>   /** the info for the first block */
>   public LocatedBlockInfo getBlockInfo();
>   public long getBlockSize();
>   public long getLength();
> }
> interface ClientProtocol extends VersionedProtocol {
>   public OpenFileInfo open(String name) throws IOException;
>   /** get block info for any range of blocks */
>   public LocatedBlockInfo[] getBlockInfo(String name, int blockOffset, int blockLength) throws IOException;
> }
> so that the client can decide how much block info to request and when. Currently, when the file is opened or an error occurs, the entire block list is requested and sent.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-894) dfs client protocol should allow asking for parts of the block map

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12497611 ]

Hadoop QA commented on HADOOP-894:
----------------------------------

+0

http://issues.apache.org/jira/secure/attachment/12357807/partialBlockList5.patch applied and successfully tested against trunk revision r540271,
but there appear to be new Findbugs warnings introduced by this patch.

Findbugs output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/171/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Test results:    http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/171/testReport/
Console output:  http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/171/console

> dfs client protocol should allow asking for parts of the block map
> ------------------------------------------------------------------
>
>                 Key: HADOOP-894
>                 URL: https://issues.apache.org/jira/browse/HADOOP-894
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.12.0
>            Reporter: Owen O'Malley
>         Assigned To: Konstantin Shvachko
>             Fix For: 0.14.0
>
>         Attachments: partialBlockList2.patch, partialBlockList3.patch, partialBlockList5.patch
>
>
> I think that the HDFS client protocol should change like:
> /** The meta-data about a file that was opened. */
> class OpenFileInfo {
>   /** the info for the first block */
>   public LocatedBlockInfo getBlockInfo();
>   public long getBlockSize();
>   public long getLength();
> }
> interface ClientProtocol extends VersionedProtocol {
>   public OpenFileInfo open(String name) throws IOException;
>   /** get block info for any range of blocks */
>   public LocatedBlockInfo[] getBlockInfo(String name, int blockOffset, int blockLength) throws IOException;
> }
> so that the client can decide how much block info to request and when. Currently, when the file is opened or an error occurs, the entire block list is requested and sent.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-894) dfs client protocol should allow asking for parts of the block map

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Shvachko updated HADOOP-894:
---------------------------------------

    Attachment:     (was: partialBlockList5.patch)

> dfs client protocol should allow asking for parts of the block map
> ------------------------------------------------------------------
>
>                 Key: HADOOP-894
>                 URL: https://issues.apache.org/jira/browse/HADOOP-894
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.12.0
>            Reporter: Owen O'Malley
>         Assigned To: Konstantin Shvachko
>             Fix For: 0.14.0
>
>         Attachments: partialBlockList2.patch, partialBlockList3.patch, partialBlockList6.patch
>
>
> I think that the HDFS client protocol should change like:
> /** The meta-data about a file that was opened. */
> class OpenFileInfo {
>   /** the info for the first block */
>   public LocatedBlockInfo getBlockInfo();
>   public long getBlockSize();
>   public long getLength();
> }
> interface ClientProtocol extends VersionedProtocol {
>   public OpenFileInfo open(String name) throws IOException;
>   /** get block info for any range of blocks */
>   public LocatedBlockInfo[] getBlockInfo(String name, int blockOffset, int blockLength) throws IOException;
> }
> so that the client can decide how much block info to request and when. Currently, when the file is opened or an error occurs, the entire block list is requested and sent.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-894) dfs client protocol should allow asking for parts of the block map

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Shvachko updated HADOOP-894:
---------------------------------------

    Attachment: partialBlockList6.patch

Fixed FindBugs warnings.
This was actually useful.

> dfs client protocol should allow asking for parts of the block map
> ------------------------------------------------------------------
>
>                 Key: HADOOP-894
>                 URL: https://issues.apache.org/jira/browse/HADOOP-894
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.12.0
>            Reporter: Owen O'Malley
>         Assigned To: Konstantin Shvachko
>             Fix For: 0.14.0
>
>         Attachments: partialBlockList2.patch, partialBlockList3.patch, partialBlockList6.patch
>
>
> I think that the HDFS client protocol should change like:
> /** The meta-data about a file that was opened. */
> class OpenFileInfo {
>   /** the info for the first block */
>   public LocatedBlockInfo getBlockInfo();
>   public long getBlockSize();
>   public long getLength();
> }
> interface ClientProtocol extends VersionedProtocol {
>   public OpenFileInfo open(String name) throws IOException;
>   /** get block info for any range of blocks */
>   public LocatedBlockInfo[] getBlockInfo(String name, int blockOffset, int blockLength) throws IOException;
> }
> so that the client can decide how much block info to request and when. Currently, when the file is opened or an error occurs, the entire block list is requested and sent.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-894) dfs client protocol should allow asking for parts of the block map

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Shvachko updated HADOOP-894:
---------------------------------------

    Status: Patch Available  (was: Open)

> dfs client protocol should allow asking for parts of the block map
> ------------------------------------------------------------------
>
>                 Key: HADOOP-894
>                 URL: https://issues.apache.org/jira/browse/HADOOP-894
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.12.0
>            Reporter: Owen O'Malley
>         Assigned To: Konstantin Shvachko
>             Fix For: 0.14.0
>
>         Attachments: partialBlockList2.patch, partialBlockList3.patch, partialBlockList6.patch
>
>
> I think that the HDFS client protocol should change like:
> /** The meta-data about a file that was opened. */
> class OpenFileInfo {
>   /** the info for the first block */
>   public LocatedBlockInfo getBlockInfo();
>   public long getBlockSize();
>   public long getLength();
> }
> interface ClientProtocol extends VersionedProtocol {
>   public OpenFileInfo open(String name) throws IOException;
>   /** get block info for any range of blocks */
>   public LocatedBlockInfo[] getBlockInfo(String name, int blockOffset, int blockLength) throws IOException;
> }
> so that the client can decide how much block info to request and when. Currently, when the file is opened or an error occurs, the entire block list is requested and sent.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-894) dfs client protocol should allow asking for parts of the block map

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Konstantin Shvachko updated HADOOP-894:
---------------------------------------

    Status: Open  (was: Patch Available)

> dfs client protocol should allow asking for parts of the block map
> ------------------------------------------------------------------
>
>                 Key: HADOOP-894
>                 URL: https://issues.apache.org/jira/browse/HADOOP-894
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.12.0
>            Reporter: Owen O'Malley
>         Assigned To: Konstantin Shvachko
>             Fix For: 0.14.0
>
>         Attachments: partialBlockList2.patch, partialBlockList3.patch, partialBlockList6.patch
>
>
> I think that the HDFS client protocol should change like:
> /** The meta-data about a file that was opened. */
> class OpenFileInfo {
>   /** the info for the first block */
>   public LocatedBlockInfo getBlockInfo();
>   public long getBlockSize();
>   public long getLength();
> }
> interface ClientProtocol extends VersionedProtocol {
>   public OpenFileInfo open(String name) throws IOException;
>   /** get block info for any range of blocks */
>   public LocatedBlockInfo[] getBlockInfo(String name, int blockOffset, int blockLength) throws IOException;
> }
> so that the client can decide how much block info to request and when. Currently, when the file is opened or an error occurs, the entire block list is requested and sent.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-894) dfs client protocol should allow asking for parts of the block map

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12497678 ]

Hadoop QA commented on HADOOP-894:
----------------------------------

+1

http://issues.apache.org/jira/secure/attachment/12357824/partialBlockList6.patch applied and successfully tested against trunk revision r540359.

Test results:   http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/173/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/173/console

> dfs client protocol should allow asking for parts of the block map
> ------------------------------------------------------------------
>
>                 Key: HADOOP-894
>                 URL: https://issues.apache.org/jira/browse/HADOOP-894
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.12.0
>            Reporter: Owen O'Malley
>         Assigned To: Konstantin Shvachko
>             Fix For: 0.14.0
>
>         Attachments: partialBlockList2.patch, partialBlockList3.patch, partialBlockList6.patch
>
>
> I think that the HDFS client protocol should change like:
> /** The meta-data about a file that was opened. */
> class OpenFileInfo {
>   /** the info for the first block */
>   public LocatedBlockInfo getBlockInfo();
>   public long getBlockSize();
>   public long getLength();
> }
> interface ClientProtocol extends VersionedProtocol {
>   public OpenFileInfo open(String name) throws IOException;
>   /** get block info for any range of blocks */
>   public LocatedBlockInfo[] getBlockInfo(String name, int blockOffset, int blockLength) throws IOException;
> }
> so that the client can decide how much block info to request and when. Currently, when the file is opened or an error occurs, the entire block list is requested and sent.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

Findbugs Jira comment

Nigel Daley
In reply to this post by Sebastian Nagel (Jira)

> Hadoop QA commented on HADOOP-894:
> ----------------------------------
>
> +0
>
> http://issues.apache.org/jira/secure/attachment/12357807/ 
> partialBlockList5.patch applied and successfully tested against  
> trunk revision r540271,
> but there appear to be new Findbugs warnings introduced by this patch.
>
> Findbugs output: http://lucene.zones.apache.org:8080/hudson/job/ 
> Hadoop-Patch/171/artifact/trunk/build/test/findbugs/
> newPatchFindbugsWarnings.html
> Test results:    http://lucene.zones.apache.org:8080/hudson/job/ 
> Hadoop-Patch/171/testReport/
> Console output:  http://lucene.zones.apache.org:8080/hudson/job/ 
> Hadoop-Patch/171/console

I'm inclined to make the Findbugs comment a -1 instead of a +0. Also,  
I'll try to make the message clearer.

Thoughts?

Cheers,
Nige
Reply | Threaded
Open this post in threaded view
|

Re: Findbugs Jira comment

Owen O'Malley-4

On May 21, 2007, at 10:28 PM, Nigel Daley wrote:

> I'm inclined to make the Findbugs comment a -1 instead of a +0.  
> Also, I'll try to make the message clearer.

I think that a +0 is better until we finalize on a set of rules that  
should be enforced.

-- Owen
Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-894) dfs client protocol should allow asking for parts of the block map

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Doug Cutting updated HADOOP-894:
--------------------------------

    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

I just committed this.  Thanks, Konstantin!

> dfs client protocol should allow asking for parts of the block map
> ------------------------------------------------------------------
>
>                 Key: HADOOP-894
>                 URL: https://issues.apache.org/jira/browse/HADOOP-894
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.12.0
>            Reporter: Owen O'Malley
>         Assigned To: Konstantin Shvachko
>             Fix For: 0.14.0
>
>         Attachments: partialBlockList2.patch, partialBlockList3.patch, partialBlockList6.patch
>
>
> I think that the HDFS client protocol should change like:
> /** The meta-data about a file that was opened. */
> class OpenFileInfo {
>   /** the info for the first block */
>   public LocatedBlockInfo getBlockInfo();
>   public long getBlockSize();
>   public long getLength();
> }
> interface ClientProtocol extends VersionedProtocol {
>   public OpenFileInfo open(String name) throws IOException;
>   /** get block info for any range of blocks */
>   public LocatedBlockInfo[] getBlockInfo(String name, int blockOffset, int blockLength) throws IOException;
> }
> so that the client can decide how much block info to request and when. Currently, when the file is opened or an error occurs, the entire block list is requested and sent.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-894) dfs client protocol should allow asking for parts of the block map

Sebastian Nagel (Jira)
In reply to this post by Sebastian Nagel (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12498189 ]

Hadoop QA commented on HADOOP-894:
----------------------------------

Integrated in Hadoop-Nightly #98 (See http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/98/)

> dfs client protocol should allow asking for parts of the block map
> ------------------------------------------------------------------
>
>                 Key: HADOOP-894
>                 URL: https://issues.apache.org/jira/browse/HADOOP-894
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.12.0
>            Reporter: Owen O'Malley
>         Assigned To: Konstantin Shvachko
>             Fix For: 0.14.0
>
>         Attachments: partialBlockList2.patch, partialBlockList3.patch, partialBlockList6.patch
>
>
> I think that the HDFS client protocol should change like:
> /** The meta-data about a file that was opened. */
> class OpenFileInfo {
>   /** the info for the first block */
>   public LocatedBlockInfo getBlockInfo();
>   public long getBlockSize();
>   public long getLength();
> }
> interface ClientProtocol extends VersionedProtocol {
>   public OpenFileInfo open(String name) throws IOException;
>   /** get block info for any range of blocks */
>   public LocatedBlockInfo[] getBlockInfo(String name, int blockOffset, int blockLength) throws IOException;
> }
> so that the client can decide how much block info to request and when. Currently, when the file is opened or an error occurs, the entire block list is requested and sent.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

123