Checksum of blocks

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Checksum of blocks

Gautam Singaraju-2
Hello all,

From HDFS Architecture page
(http://hadoop.apache.org/common/docs/current/hdfs_design.html): "When
a client creates an HDFS file, it computes a checksum of each block of
the file and stores these checksums in a separate hidden file in the
same HDFS namespace."

Is there a way to access this data?

---
Gautam
Reply | Threaded
Open this post in threaded view
|

Re: Checksum of blocks

cvkkumar
Hi,

There is a method getChecksumFile(Path file)  that gives this info.
You could refer the javadoc here:
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/fs/ChecksumFileSystem.html

Hope this helped.

Regards,
Krishnakumar.
On Jul 31, 2010, at 10:10 PM, Gautam Singaraju wrote:

> Hello all,
>
> From HDFS Architecture page
> (http://hadoop.apache.org/common/docs/current/hdfs_design.html): "When
> a client creates an HDFS file, it computes a checksum of each block of
> the file and stores these checksums in a separate hidden file in the
> same HDFS namespace."
>
> Is there a way to access this data?
>
> ---
> Gautam

Reply | Threaded
Open this post in threaded view
|

Re: Checksum of blocks

Gautam Singaraju-2
I had phrased my question incorrectly. I am look to get a list of all
blocks (not files) on HDFS and their checksum. Is there a way?
---
Gautam



On Sun, Aug 1, 2010 at 8:18 PM, cvkkumar <[hidden email]> wrote:

> Hi,
>
> There is a method getChecksumFile(Path file)  that gives this info.
> You could refer the javadoc here:
> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/fs/ChecksumFileSystem.html
>
> Hope this helped.
>
> Regards,
> Krishnakumar.
> On Jul 31, 2010, at 10:10 PM, Gautam Singaraju wrote:
>
>> Hello all,
>>
>> From HDFS Architecture page
>> (http://hadoop.apache.org/common/docs/current/hdfs_design.html): "When
>> a client creates an HDFS file, it computes a checksum of each block of
>> the file and stores these checksums in a separate hidden file in the
>> same HDFS namespace."
>>
>> Is there a way to access this data?
>>
>> ---
>> Gautam
>
>