how to manipulate with MapWritable metaData in CrawlDatum structure

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

how to manipulate with MapWritable metaData in CrawlDatum structure

Feng Ji
hi,

I wonder how to use "MapWritable metaData" in CrawlDatum.java. The API gives
us some function call, but I still don't know how to
input information (String) to metaData and retrieve information; How to
convert MapWritable variable to other types like MetaData type or String
type.

Any good sample in Nutch's java class?

thanks,

Feng
Reply | Threaded
Open this post in threaded view
|

Re: how to manipulate with MapWritable metaData in CrawlDatum structure

Dennis Kubes
The MapWritable acts as a shared memory area or Map that you can put
other writables into and retrieve them from.  To Add metatdata to the
CrawlDataum you would use something like this:

datum.getMetaData().put(key, value)

Where key and value are both Writable implementations such as UTF8 or
IntWritable.  To get metadata out you would do something like this:

Writable writable = datum.getMetaData().get(key)

Dennis

Feng Ji wrote:

> hi,
>
> I wonder how to use "MapWritable metaData" in CrawlDatum.java. The API
> gives
> us some function call, but I still don't know how to
> input information (String) to metaData and retrieve information; How to
> convert MapWritable variable to other types like MetaData type or String
> type.
>
> Any good sample in Nutch's java class?
>
> thanks,
>
> Feng
>
Reply | Threaded
Open this post in threaded view
|

Re: how to manipulate with MapWritable metaData in CrawlDatum structure

Stefan Groschupf-2
In reply to this post by Feng Ji
Hi Feng,

map Writrable is a kind of hashmap.
You can put in any key value pair, but the key and values need to be  
Writables:
http://lucene.apache.org/hadoop/docs/api/org/apache/hadoop/io/ 
Writable.html

You can use UTF8 as StingKey and Value or ByteWritable as key and  
Utf8 as Values.
Etc.
Does this answer your question?
Stefan


Am 12.06.2006 um 04:15 schrieb Feng Ji:

> hi,
>
> I wonder how to use "MapWritable metaData" in CrawlDatum.java. The  
> API gives
> us some function call, but I still don't know how to
> input information (String) to metaData and retrieve information;  
> How to
> convert MapWritable variable to other types like MetaData type or  
> String
> type.
>
> Any good sample in Nutch's java class?
>
> thanks,
>
> Feng