Hadoop compression

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Hadoop compression

Christopher Laux
Hi all,

I have a fundamental question about compressing the segments nutch
produces: is it switched on by default or how can I switch it on if
needed? I can only find general documentation about turning
compression on for Hadoop, but there is no hadoop-site.xml in
nutch/conf ?

Thanks,
Chris
Reply | Threaded
Open this post in threaded view
|

Re: Hadoop compression

Julien Nioche-4
Hi Chris,

You can create your own hadoop site files in conf/ to specify that, Nutch
will use them.

Julien

--
*
*Open Source Solutions for Text Engineering

http://digitalpebble.blogspot.com/
http://www.digitalpebble.com

On 4 October 2010 16:48, Christopher Laux <[hidden email]> wrote:

> Hi all,
>
> I have a fundamental question about compressing the segments nutch
> produces: is it switched on by default or how can I switch it on if
> needed? I can only find general documentation about turning
> compression on for Hadoop, but there is no hadoop-site.xml in
> nutch/conf ?
>
> Thanks,
> Chris
>