MapFile usage

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

MapFile usage

alakshman
Would someone be kind enough to share with me any code/sample they have for
using the MapFile class ?

Thanks
A
Reply | Threaded
Open this post in threaded view
|

Re: MapFile usage

Open Study
Hi Phantom

   To operate the Mapfile isn't difficult by itself, simply follow the order
1) create the mapfile writer or reader by specifying the JobConf, file
system, the path name of the mapfile. If creating the MapFile you also need
to specify the key class(shall implement
org.apache.hadoop.io.WritableComparable) and value class (shall implement
org.apache.hadoop.io.Writable).

/* create the MapFile writer and write to it */
MapFile.Writer writer = new MapFile.Writer(conf, fs, dir,
                        MyKey.class, MyValue.class,
                        SequenceFile.CompressionType.BLOCK);
MyKey key = new MyKey();
MyValue value = new MyValue();
writer.append(key, value);

/* create the MapFile reader and read from it */
MapFile.Reader reader = new MapFile.Reader(fs, dir, conf);
MyKey key = new MyKey();
MyValue value = new MyValue();
while (reader.next(key, value))
{
    // do sth with the key and value
}

   You can also turn to the source code of Nutch,
org.apache.nutch.fetcher.FetcherOutputFormat for referrence.

On 7/3/07, Phantom <[hidden email]> wrote:
>
> Would someone be kind enough to share with me any code/sample they have
> for
> using the MapFile class ?
>
> Thanks
> A
>