0.12.1 release plan

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

0.12.1 release plan

Doug Cutting
I've pushed out the release date for 0.12.1 to Monday.  With recent
patches, stability is looking a lot better, but there are still a few
blocker issues.   My hope is 0.12.1 will be a stable release that, e.g.,
Nutch can confidently integrate into its upcoming 0.9 release.

Trunk currently contains only changes for 0.12.1.  No 0.13.0 changes
have yet been committed.  Let's hold off committing those until 0.12.1
is out the door, okay?

Doug
Reply | Threaded
Open this post in threaded view
|

Re: 0.12.1 release plan

Nigel Daley

On Mar 9, 2007, at 9:49 AM, Doug Cutting wrote:

> I've pushed out the release date for 0.12.1 to Monday.  With recent  
> patches, stability is looking a lot better, but there are still a  
> few blocker issues.   My hope is 0.12.1 will be a stable release  
> that, e.g., Nutch can confidently integrate into its upcoming 0.9  
> release.
>
> Trunk currently contains only changes for 0.12.1.  No 0.13.0  
> changes have yet been committed.  Let's hold off committing those  
> until 0.12.1 is out the door, okay?

+1
Reply | Threaded
Open this post in threaded view
|

Re: 0.12.1 release plan

Andrzej Białecki-2
In reply to this post by Doug Cutting
Doug Cutting wrote:
> I've pushed out the release date for 0.12.1 to Monday.  With recent
> patches, stability is looking a lot better, but there are still a few
> blocker issues.   My hope is 0.12.1 will be a stable release that,
> e.g., Nutch can confidently integrate into its upcoming 0.9 release.

Actually we discussed this on the Nutch mailing list, and the overall
consensus was that for this release we should stick to 0.11.2, since
0.12 still looks too fresh. Do you think we should revisit this issue?

--
Best regards,
Andrzej Bialecki     <><
 ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com


Reply | Threaded
Open this post in threaded view
|

Re: 0.12.1 release plan

Doug Cutting
Andrzej Bialecki wrote:
> Actually we discussed this on the Nutch mailing list, and the overall
> consensus was that for this release we should stick to 0.11.2, since
> 0.12 still looks too fresh. Do you think we should revisit this issue?

I think the experience on big clusters at Yahoo! is that 0.12.1 should
be more stable than 0.11.2, but others can confirm that.

Doug
Reply | Threaded
Open this post in threaded view
|

Re: 0.12.1 release plan

Andrzej Białecki-2
Doug Cutting wrote:
> Andrzej Bialecki wrote:
>> Actually we discussed this on the Nutch mailing list, and the overall
>> consensus was that for this release we should stick to 0.11.2, since
>> 0.12 still looks too fresh. Do you think we should revisit this issue?
>
> I think the experience on big clusters at Yahoo! is that 0.12.1 should
> be more stable than 0.11.2, but others can confirm that.

Hm.. That's not the impression I have from JIRA and the mailing list. My
impression is that even though 0.12.1 is more robust in some situations,
the significant changes (checksum filesystem, speculative execution, in
memory sorting, improved map output handling, etc, etc) made between
these releases introduced many subtle bugs which only now start coming
into light.

At the same time, I've had relatively positive experience with 0.11.2 on
moderately sized clusters (dozen machines). That's why I proposed that
Nutch should use 0.11.2 for the upcoming release.

--
Best regards,
Andrzej Bialecki     <><
 ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com


Reply | Threaded
Open this post in threaded view
|

Re: 0.12.1 release plan

Owen O'Malley-5

On Mar 10, 2007, at 12:32 AM, Andrzej Bialecki wrote:

>> I think the experience on big clusters at Yahoo! is that 0.12.1  
>> should be more stable than 0.11.2, but others can confirm that.
>
> Hm.. That's not the impression I have from JIRA and the mailing  
> list. My impression is that even though 0.12.1 is more robust in  
> some situations, the significant changes (checksum filesystem,  
> speculative execution, in memory sorting, improved map output  
> handling, etc, etc) made between these releases introduced many  
> subtle bugs which only now start coming into light.

We never upgraded our main clusters to 11.2 because it never  
stabilized to our satisfaction, which is why I was proposing an 11.3.  
However, 12.1 is looking pretty good with the exception  of a couple  
of bugs and we decided to hold out for 12.1. At this point, if I was  
going to 11, I'd want a lot of the fixes that have been done in between.

-- Owen
Reply | Threaded
Open this post in threaded view
|

Re: 0.12.1 release plan

Nigel Daley
In reply to this post by Doug Cutting
A fix for HADOOP-1093 is still pending.  The fix for HADOOP-1091  
failed and a new fix is pending.  In light of these, I've rescheduled  
Hadoop 0.12.1 to tomorrow, Tuesday, March 13.

Cheers,
Nige

On Mar 9, 2007, at 9:49 AM, Doug Cutting wrote:

> I've pushed out the release date for 0.12.1 to Monday.  With recent  
> patches, stability is looking a lot better, but there are still a  
> few blocker issues.   My hope is 0.12.1 will be a stable release  
> that, e.g., Nutch can confidently integrate into its upcoming 0.9  
> release.
>
> Trunk currently contains only changes for 0.12.1.  No 0.13.0  
> changes have yet been committed.  Let's hold off committing those  
> until 0.12.1 is out the door, okay?
>
> Doug

Reply | Threaded
Open this post in threaded view
|

Re: 0.12.1 release plan

Nigel Daley
A number of additional 0.12.1 blockers have come up and have patches  
available for them.  However, a fix for HADOOP-1093 is still  
pending.  In light of this, I'm pushing Hadoop 0.12.1 release out 1  
more day to tomorrow, Wednesday, March 14.

Cheers,
Nige

On Mar 12, 2007, at 2:42 PM, Nigel Daley wrote:

> A fix for HADOOP-1093 is still pending.  The fix for HADOOP-1091  
> failed and a new fix is pending.  In light of these, I've  
> rescheduled Hadoop 0.12.1 to tomorrow, Tuesday, March 13.
>
> Cheers,
> Nige
>
> On Mar 9, 2007, at 9:49 AM, Doug Cutting wrote:
>
>> I've pushed out the release date for 0.12.1 to Monday.  With  
>> recent patches, stability is looking a lot better, but there are  
>> still a few blocker issues.   My hope is 0.12.1 will be a stable  
>> release that, e.g., Nutch can confidently integrate into its  
>> upcoming 0.9 release.
>>
>> Trunk currently contains only changes for 0.12.1.  No 0.13.0  
>> changes have yet been committed.  Let's hold off committing those  
>> until 0.12.1 is out the door, okay?
>>
>> Doug
>

Reply | Threaded
Open this post in threaded view
|

Re: 0.12.1 release plan

Dennis Kubes
Can we get the new HADOOP-1115 included in this release?  This would
really help with allowing Nutch to use 0.12.1 as opposed to the older
0.11.x release

Dennis Kubes

Nigel Daley wrote:

> A number of additional 0.12.1 blockers have come up and have patches
> available for them.  However, a fix for HADOOP-1093 is still pending.  
> In light of this, I'm pushing Hadoop 0.12.1 release out 1 more day to
> tomorrow, Wednesday, March 14.
>
> Cheers,
> Nige
>
> On Mar 12, 2007, at 2:42 PM, Nigel Daley wrote:
>
>> A fix for HADOOP-1093 is still pending.  The fix for HADOOP-1091
>> failed and a new fix is pending.  In light of these, I've rescheduled
>> Hadoop 0.12.1 to tomorrow, Tuesday, March 13.
>>
>> Cheers,
>> Nige
>>
>> On Mar 9, 2007, at 9:49 AM, Doug Cutting wrote:
>>
>>> I've pushed out the release date for 0.12.1 to Monday.  With recent
>>> patches, stability is looking a lot better, but there are still a few
>>> blocker issues.   My hope is 0.12.1 will be a stable release that,
>>> e.g., Nutch can confidently integrate into its upcoming 0.9 release.
>>>
>>> Trunk currently contains only changes for 0.12.1.  No 0.13.0 changes
>>> have yet been committed.  Let's hold off committing those until
>>> 0.12.1 is out the door, okay?
>>>
>>> Doug
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: 0.12.1 release plan

Andrzej Białecki-2
Dennis Kubes wrote:
> Can we get the new HADOOP-1115 included in this release?  This would
> really help with allowing Nutch to use 0.12.1 as opposed to the older
> 0.11.x release

Why are you (the Hadoop guys) pushing for the release so quickly? Are
you under a deadline? IMHO things should be allowed to settle a bit - I
keep having a distinct impression that the code is unstable now, and
needs to be tested more thoroughly before proclaiming its
release-worthiness - and IMO delaying the release by 24 hours won't help
much.

Having that many critical issues 24 hours before the release doesn't
give me too much confidence in the quality of this release - I'm more
and more convinced that Nutch should use 0.11.2 for now. I understand
the principle of releasing early and often, but there has to be some
point in making a release, i.e. the code should be mostly usable - as I
see it now there is a constant stream of new critical issues being
uncovered with the current code.

May I suggest that you should postpone the Hadoop release by 2 weeks or so?

--
Best regards,
Andrzej Bialecki     <><
 ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com


Reply | Threaded
Open this post in threaded view
|

Re: 0.12.1 release plan

Nigel Daley
In reply to this post by Dennis Kubes
Hi Dennis,

If you go to http://issues.apache.org/jira/browse/HADOOP and click on  
0.12.1 you'll see that HADOOP-1115 is already assigned to 0.12.1  
release.  You can also see the other uncommitted defects that are  
assigned to this release.

hth,
Nige

On Mar 13, 2007, at 12:35 PM, Dennis Kubes wrote:

> Can we get the new HADOOP-1115 included in this release?  This  
> would really help with allowing Nutch to use 0.12.1 as opposed to  
> the older 0.11.x release
>
> Dennis Kubes
>
> Nigel Daley wrote:
>> A number of additional 0.12.1 blockers have come up and have  
>> patches available for them.  However, a fix for HADOOP-1093 is  
>> still pending.  In light of this, I'm pushing Hadoop 0.12.1  
>> release out 1 more day to tomorrow, Wednesday, March 14.
>> Cheers,
>> Nige
>> On Mar 12, 2007, at 2:42 PM, Nigel Daley wrote:
>>> A fix for HADOOP-1093 is still pending.  The fix for HADOOP-1091  
>>> failed and a new fix is pending.  In light of these, I've  
>>> rescheduled Hadoop 0.12.1 to tomorrow, Tuesday, March 13.
>>>
>>> Cheers,
>>> Nige
>>>
>>> On Mar 9, 2007, at 9:49 AM, Doug Cutting wrote:
>>>
>>>> I've pushed out the release date for 0.12.1 to Monday.  With  
>>>> recent patches, stability is looking a lot better, but there are  
>>>> still a few blocker issues.   My hope is 0.12.1 will be a stable  
>>>> release that, e.g., Nutch can confidently integrate into its  
>>>> upcoming 0.9 release.
>>>>
>>>> Trunk currently contains only changes for 0.12.1.  No 0.13.0  
>>>> changes have yet been committed.  Let's hold off committing  
>>>> those until 0.12.1 is out the door, okay?
>>>>
>>>> Doug
>>>

Reply | Threaded
Open this post in threaded view
|

Re: 0.12.1 release plan

Dennis Kubes


Nigel Daley wrote:
> Hi Dennis,
>
> If you go to http://issues.apache.org/jira/browse/HADOOP and click on
> 0.12.1 you'll see that HADOOP-1115 is already assigned to 0.12.1
> release.  You can also see the other uncommitted defects that are
> assigned to this release.
>
> hth,
> Nige

Excellent.  Thanks.

>
> On Mar 13, 2007, at 12:35 PM, Dennis Kubes wrote:
>
>> Can we get the new HADOOP-1115 included in this release?  This would
>> really help with allowing Nutch to use 0.12.1 as opposed to the older
>> 0.11.x release
>>
>> Dennis Kubes
>>
>> Nigel Daley wrote:
>>> A number of additional 0.12.1 blockers have come up and have patches
>>> available for them.  However, a fix for HADOOP-1093 is still
>>> pending.  In light of this, I'm pushing Hadoop 0.12.1 release out 1
>>> more day to tomorrow, Wednesday, March 14.
>>> Cheers,
>>> Nige
>>> On Mar 12, 2007, at 2:42 PM, Nigel Daley wrote:
>>>> A fix for HADOOP-1093 is still pending.  The fix for HADOOP-1091
>>>> failed and a new fix is pending.  In light of these, I've
>>>> rescheduled Hadoop 0.12.1 to tomorrow, Tuesday, March 13.
>>>>
>>>> Cheers,
>>>> Nige
>>>>
>>>> On Mar 9, 2007, at 9:49 AM, Doug Cutting wrote:
>>>>
>>>>> I've pushed out the release date for 0.12.1 to Monday.  With recent
>>>>> patches, stability is looking a lot better, but there are still a
>>>>> few blocker issues.   My hope is 0.12.1 will be a stable release
>>>>> that, e.g., Nutch can confidently integrate into its upcoming 0.9
>>>>> release.
>>>>>
>>>>> Trunk currently contains only changes for 0.12.1.  No 0.13.0
>>>>> changes have yet been committed.  Let's hold off committing those
>>>>> until 0.12.1 is out the door, okay?
>>>>>
>>>>> Doug
>>>>
>
Reply | Threaded
Open this post in threaded view
|

Re: 0.12.1 release plan

Nigel Daley
In reply to this post by Andrzej Białecki-2
Hi Andrzej,

The current process seems to be that we release often and then wait  
some time before declaring a particular release as stable.  So I see  
no problem in releasing something sooner than later that contains  
fixes for major defects that we currently know about.  It will then  
'soak' in the community for some time before we declare it stable or  
discover we need a follow-on bug fix release.

Make sense?

Cheers,
Nige


On Mar 13, 2007, at 12:50 PM, Andrzej Bialecki wrote:

> Dennis Kubes wrote:
>> Can we get the new HADOOP-1115 included in this release?  This  
>> would really help with allowing Nutch to use 0.12.1 as opposed to  
>> the older 0.11.x release
>
> Why are you (the Hadoop guys) pushing for the release so quickly?  
> Are you under a deadline? IMHO things should be allowed to settle a  
> bit - I keep having a distinct impression that the code is unstable  
> now, and needs to be tested more thoroughly before proclaiming its  
> release-worthiness - and IMO delaying the release by 24 hours won't  
> help much.
>
> Having that many critical issues 24 hours before the release  
> doesn't give me too much confidence in the quality of this release  
> - I'm more and more convinced that Nutch should use 0.11.2 for now.  
> I understand the principle of releasing early and often, but there  
> has to be some point in making a release, i.e. the code should be  
> mostly usable - as I see it now there is a constant stream of new  
> critical issues being uncovered with the current code.
>
> May I suggest that you should postpone the Hadoop release by 2  
> weeks or so?
>
> --
> Best regards,
> Andrzej Bialecki     <><
> ___. ___ ___ ___ _ _   __________________________________
> [__ || __|__/|__||\/|  Information Retrieval, Semantic Web
> ___|||__||  \|  ||  |  Embedded Unix, System Integration
> http://www.sigram.com  Contact: info at sigram dot com
>
>

Reply | Threaded
Open this post in threaded view
|

Re: 0.12.1 release plan

Andrzej Białecki-2
Nigel Daley wrote:

> Hi Andrzej,
>
> The current process seems to be that we release often and then wait
> some time before declaring a particular release as stable.  So I see
> no problem in releasing something sooner than later that contains
> fixes for major defects that we currently know about.  It will then
> 'soak' in the community for some time before we declare it stable or
> discover we need a follow-on bug fix release.
>
> Make sense?

Sure, it does. All I'm saying is that this release is going to be more
shaky than the previous ones, and perhaps giving it another two weeks
would save you the effort of making a release that is known to be
unstable ... I guess I was hoping too much that we can use 0.12.1 in
this Nutch release ... ;)

--
Best regards,
Andrzej Bialecki     <><
 ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com


Reply | Threaded
Open this post in threaded view
|

Re: 0.12.1 release plan

Nigel Daley
In reply to this post by Nigel Daley
[cross posting to nutch-dev since they're waiting for 0.12.1 release]

Still no progress on HADOOP-1093.  Postponing Hadoop 0.12.1 release  
to Monday.

Also, HADOOP-1118 and HADOOP-1119 are being investigated.

Cheers,
Nige

On Mar 13, 2007, at 10:49 AM, Nigel Daley wrote:

> A number of additional 0.12.1 blockers have come up and have  
> patches available for them.  However, a fix for HADOOP-1093 is  
> still pending.  In light of this, I'm pushing Hadoop 0.12.1 release  
> out 1 more day to tomorrow, Wednesday, March 14.
>
> Cheers,
> Nige
>
> On Mar 12, 2007, at 2:42 PM, Nigel Daley wrote:
>
>> A fix for HADOOP-1093 is still pending.  The fix for HADOOP-1091  
>> failed and a new fix is pending.  In light of these, I've  
>> rescheduled Hadoop 0.12.1 to tomorrow, Tuesday, March 13.
>>
>> Cheers,
>> Nige
>>
>> On Mar 9, 2007, at 9:49 AM, Doug Cutting wrote:
>>
>>> I've pushed out the release date for 0.12.1 to Monday.  With  
>>> recent patches, stability is looking a lot better, but there are  
>>> still a few blocker issues.   My hope is 0.12.1 will be a stable  
>>> release that, e.g., Nutch can confidently integrate into its  
>>> upcoming 0.9 release.
>>>
>>> Trunk currently contains only changes for 0.12.1.  No 0.13.0  
>>> changes have yet been committed.  Let's hold off committing those  
>>> until 0.12.1 is out the door, okay?
>>>
>>> Doug
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: 0.12.1 release plan

tomwhite
Hadoop 0.12.1 is now available
(http://www.apache.org/dyn/closer.cgi/lucene/hadoop/). Release notes
are here: http://tinyurl.com/2kynuc.

Cheers,

Tom

On 14/03/07, Nigel Daley <[hidden email]> wrote:

> [cross posting to nutch-dev since they're waiting for 0.12.1 release]
>
> Still no progress on HADOOP-1093.  Postponing Hadoop 0.12.1 release
> to Monday.
>
> Also, HADOOP-1118 and HADOOP-1119 are being investigated.
>
> Cheers,
> Nige
>
> On Mar 13, 2007, at 10:49 AM, Nigel Daley wrote:
>
> > A number of additional 0.12.1 blockers have come up and have
> > patches available for them.  However, a fix for HADOOP-1093 is
> > still pending.  In light of this, I'm pushing Hadoop 0.12.1 release
> > out 1 more day to tomorrow, Wednesday, March 14.
> >
> > Cheers,
> > Nige
> >
> > On Mar 12, 2007, at 2:42 PM, Nigel Daley wrote:
> >
> >> A fix for HADOOP-1093 is still pending.  The fix for HADOOP-1091
> >> failed and a new fix is pending.  In light of these, I've
> >> rescheduled Hadoop 0.12.1 to tomorrow, Tuesday, March 13.
> >>
> >> Cheers,
> >> Nige
> >>
> >> On Mar 9, 2007, at 9:49 AM, Doug Cutting wrote:
> >>
> >>> I've pushed out the release date for 0.12.1 to Monday.  With
> >>> recent patches, stability is looking a lot better, but there are
> >>> still a few blocker issues.   My hope is 0.12.1 will be a stable
> >>> release that, e.g., Nutch can confidently integrate into its
> >>> upcoming 0.9 release.
> >>>
> >>> Trunk currently contains only changes for 0.12.1.  No 0.13.0
> >>> changes have yet been committed.  Let's hold off committing those
> >>> until 0.12.1 is out the door, okay?
> >>>
> >>> Doug
> >>
> >
>
>