upgrade to hadoop-0.13?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

upgrade to hadoop-0.13?

Doğacan Güney-3
Hi all,

As you know, hadoop-0.13 was recently released and it brings some
impressive improvements over hadoop-0.12.x series. So the obvious
question is: should we switch to hadoop-0.13?

I have tested nutch with hadoop-0.13 with all basic jobs (inject,
generate, fetch, parse, updatedb, invertlinks, index, dedup) and they
work fine.

--
Doğacan Güney
Reply | Threaded
Open this post in threaded view
|

Re: upgrade to hadoop-0.13?

Andrzej Białecki-2
Doğacan Güney wrote:

> Hi all,
>
> As you know, hadoop-0.13 was recently released and it brings some
> impressive improvements over hadoop-0.12.x series. So the obvious
> question is: should we switch to hadoop-0.13?
>
> I have tested nutch with hadoop-0.13 with all basic jobs (inject,
> generate, fetch, parse, updatedb, invertlinks, index, dedup) and they
> work fine.
>

We need to start implementing a different caching mechanism for objects
that we thus far cached in a Configuration instance. Respective methods
in Configuration are now deprecated, and will be removed in Hadoop 0.14.
  See HADOOP-1343 for more details.

This change will affect a lot of places in our code, so it would be best
to do it long before the next Nutch release.


--
Best regards,
Andrzej Bialecki     <><
  ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com

Reply | Threaded
Open this post in threaded view
|

Re: upgrade to hadoop-0.13?

Doğacan Güney-3
On 6/18/07, Andrzej Bialecki <[hidden email]> wrote:

> Doğacan Güney wrote:
> > Hi all,
> >
> > As you know, hadoop-0.13 was recently released and it brings some
> > impressive improvements over hadoop-0.12.x series. So the obvious
> > question is: should we switch to hadoop-0.13?
> >
> > I have tested nutch with hadoop-0.13 with all basic jobs (inject,
> > generate, fetch, parse, updatedb, invertlinks, index, dedup) and they
> > work fine.
> >
>
> We need to start implementing a different caching mechanism for objects
> that we thus far cached in a Configuration instance. Respective methods
> in Configuration are now deprecated, and will be removed in Hadoop 0.14.
>   See HADOOP-1343 for more details.
>
> This change will affect a lot of places in our code, so it would be best
> to do it long before the next Nutch release.

Opened NUTCH-501 for this. I also attached a (draft) patch there.

>
>
> --
> Best regards,
> Andrzej Bialecki     <><
>   ___. ___ ___ ___ _ _   __________________________________
> [__ || __|__/|__||\/|  Information Retrieval, Semantic Web
> ___|||__||  \|  ||  |  Embedded Unix, System Integration
> http://www.sigram.com  Contact: info at sigram dot com
>
>


--
Doğacan Güney