Moving ahead to Hadoop 0.22

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Moving ahead to Hadoop 0.22

Sean Owen
In my own client, I'm forging ahead to Hadoop 0.22 to see if it works
for me. If it's not too much change to update what we've got to 0.22
(and the change is nonzero) and it works better for me, maybe we can
jump ahead to depend on it.
Reply | Threaded
Open this post in threaded view
|

Re: Moving ahead to Hadoop 0.22

Ted Dunning
I hate it when people on the commons list start whining about dropping
support for java version -17 (joke, exaggeration warning), but here I am
about to do it.

I still run 0.19.  A fair number of production clusters run 0.18.3.  Many
run 0.20

Hopefully when 1.0 comes out there will be a large move to that, but is it
important yet to gauge where our largest audience is or even will be in,
say, 6-12 months?

I don't see a large benefit to moving to the latest just because it is the
latest.  I do see a benefit in moving to a future-proof API sooner rather
than later, but is 0.22 realistic yet?

On Mon, Nov 23, 2009 at 5:22 PM, Sean Owen <[hidden email]> wrote:

> In my own client, I'm forging ahead to Hadoop 0.22 to see if it works
> for me. If it's not too much change to update what we've got to 0.22
> (and the change is nonzero) and it works better for me, maybe we can
> jump ahead to depend on it.
>



--
Ted Dunning, CTO
DeepDyve
Reply | Threaded
Open this post in threaded view
|

Re: Moving ahead to Hadoop 0.22

Robin Anil
0.22 is supposed to stabilize the new mapreduce package and remove
support for the old mapred package. So I am guessing the reason for
moving to 0.22 would go side by side with conversion of all our
existing mapred programs to mapreduce ones. And I believe I read
somewhere  that this is the api that is going to used as they move
closer to hadoop 1.0

Robin



On Tue, Nov 24, 2009 at 1:10 PM, Ted Dunning <[hidden email]> wrote:

> I hate it when people on the commons list start whining about dropping
> support for java version -17 (joke, exaggeration warning), but here I am
> about to do it.
>
> I still run 0.19.  A fair number of production clusters run 0.18.3.  Many
> run 0.20
>
> Hopefully when 1.0 comes out there will be a large move to that, but is it
> important yet to gauge where our largest audience is or even will be in,
> say, 6-12 months?
>
> I don't see a large benefit to moving to the latest just because it is the
> latest.  I do see a benefit in moving to a future-proof API sooner rather
> than later, but is 0.22 realistic yet?
>
> On Mon, Nov 23, 2009 at 5:22 PM, Sean Owen <[hidden email]> wrote:
>
>> In my own client, I'm forging ahead to Hadoop 0.22 to see if it works
>> for me. If it's not too much change to update what we've got to 0.22
>> (and the change is nonzero) and it works better for me, maybe we can
>> jump ahead to depend on it.
>>
>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>
Reply | Threaded
Open this post in threaded view
|

Re: Moving ahead to Hadoop 0.22

Sean Owen
Early report from my testing is it's going to break a lot of our code,
so, perhaps a bridge too far now.

There's one reason I'm keen to move forward and it's not merely
wanting to be on the bleeding edge, far from it. It's that 0.20.x does
not work at all for my jobs. It runs into bugs that 0.20.1 does not
appear to fix. So I'm kind of between a rock and hard place. My
alternative is to go back to 0.19, which isn't the end of the world,
but feels wrong.

On Tue, Nov 24, 2009 at 8:25 AM, Robin Anil <[hidden email]> wrote:

> 0.22 is supposed to stabilize the new mapreduce package and remove
> support for the old mapred package. So I am guessing the reason for
> moving to 0.22 would go side by side with conversion of all our
> existing mapred programs to mapreduce ones. And I believe I read
> somewhere  that this is the api that is going to used as they move
> closer to hadoop 1.0
>
> Robin
>
>
>
> On Tue, Nov 24, 2009 at 1:10 PM, Ted Dunning <[hidden email]> wrote:
>> I hate it when people on the commons list start whining about dropping
>> support for java version -17 (joke, exaggeration warning), but here I am
>> about to do it.
>>
>> I still run 0.19.  A fair number of production clusters run 0.18.3.  Many
>> run 0.20
>>
>> Hopefully when 1.0 comes out there will be a large move to that, but is it
>> important yet to gauge where our largest audience is or even will be in,
>> say, 6-12 months?
>>
>> I don't see a large benefit to moving to the latest just because it is the
>> latest.  I do see a benefit in moving to a future-proof API sooner rather
>> than later, but is 0.22 realistic yet?
>>
>> On Mon, Nov 23, 2009 at 5:22 PM, Sean Owen <[hidden email]> wrote:
>>
>>> In my own client, I'm forging ahead to Hadoop 0.22 to see if it works
>>> for me. If it's not too much change to update what we've got to 0.22
>>> (and the change is nonzero) and it works better for me, maybe we can
>>> jump ahead to depend on it.
>>>
>>
>>
>>
>> --
>> Ted Dunning, CTO
>> DeepDyve
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Moving ahead to Hadoop 0.22

Ted Dunning
Well, it is wrong at some level and it will become more and more wrong.

I have heard from Chris Wenzel that the cost of moving post 19 was pretty
high.  It would be good to do that when we can do it whole-heartedly.

(what is the situation with 21?)

On Tue, Nov 24, 2009 at 1:23 AM, Sean Owen <[hidden email]> wrote:

> My alternative is to go back to 0.19, which isn't the end of the world,
> but feels wrong.
>



--
Ted Dunning, CTO
DeepDyve