Upgrading Nutch to Hadoop 0.14 or 0.15

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Upgrading Nutch to Hadoop 0.14 or 0.15

Dennis Kubes-2
What does everyone think about upgrading the Nutch trunk to 0.14 or 0.15?

I know there is an open Jira NUTCH-552 about upgrading that says it is
dependent upon a new caching mechanism.  Is that really necessary before
upgrading.

We have been running nutch with 0.15 for some time now and we have
patches available.  If everyone thinks it is time, I can submit the
patches for review.

Dennis Kubes
Reply | Threaded
Open this post in threaded view
|

Re: Upgrading Nutch to Hadoop 0.14 or 0.15

Andrzej Białecki-2
Dennis Kubes wrote:
> What does everyone think about upgrading the Nutch trunk to 0.14 or 0.15?
>
> I know there is an open Jira NUTCH-552 about upgrading that says it is
> dependent upon a new caching mechanism.  Is that really necessary before
> upgrading.
>
> We have been running nutch with 0.15 for some time now and we have
> patches available.  If everyone thinks it is time, I can submit the
> patches for review.

Hadoop 0.14 and 0.15 deprecate Configuration.setObject(), which means
that 0.16 will remove this method completely. IMO we should first commit
the caching impl. (NUTCH-501) - I'm not sure what's holding us there ...
and then proceed to upgrade to 0.15. I propose to skip 0.14 altogether,
because by the time we make our release that version of Hadoop will be
already old (and poorly supported).


--
Best regards,
Andrzej Bialecki     <><
  ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com

Reply | Threaded
Open this post in threaded view
|

Re: Upgrading Nutch to Hadoop 0.14 or 0.15

Dennis Kubes-2
OK, will change the NUTCH-552 Jira to be Hadoop-0.15 and will attach
patch for that.  Then will test the caching implementation for unit
tests.  If that passes I suggest we upgrade.

Dennis Kubes

Andrzej Bialecki wrote:

> Dennis Kubes wrote:
>> What does everyone think about upgrading the Nutch trunk to 0.14 or 0.15?
>>
>> I know there is an open Jira NUTCH-552 about upgrading that says it is
>> dependent upon a new caching mechanism.  Is that really necessary
>> before upgrading.
>>
>> We have been running nutch with 0.15 for some time now and we have
>> patches available.  If everyone thinks it is time, I can submit the
>> patches for review.
>
> Hadoop 0.14 and 0.15 deprecate Configuration.setObject(), which means
> that 0.16 will remove this method completely. IMO we should first commit
> the caching impl. (NUTCH-501) - I'm not sure what's holding us there ...
> and then proceed to upgrade to 0.15. I propose to skip 0.14 altogether,
> because by the time we make our release that version of Hadoop will be
> already old (and poorly supported).
>
>