write out fetch results without map-reduce

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

write out fetch results without map-reduce

AJ Chen-2
I'm trying to use latest nutch code base to write a fetcher that does not
use map-reduce operations. After fetching a list of urls, CrawlDatum and
Content are available to output to file. What's the most direct way to write
the fetch results to file without going through map-reduce operations? Use
SequenceFile class directly? It writes in binary. Any existing class for
writing results in plain text?
thanks,

AJ