Re: Branch merges and 3.0.0-beta1 scope

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Branch merges and 3.0.0-beta1 scope

Wangda Tan
Andrew,

Thanks for your help to pushing this release.

Echoing what Vinod said, all contributors in these branches are putting
months to years of time working on these features, we don't have to decide
excluded features now since we have 25 days till 3.0-beta1 planned release
time.

The best approach to stabilize feature is to let people try that, instead
of waiting for feature becomes perfect. For features which can be turned
off, I think we should consider to bring it in if it is end-to-end ready. I
will try best to help merge efforts of YARN-3926 branch to trunk before Sep
15, and I'm OK with moving to the next release train if we fail to merge
the feature before release date.

Thanks,
Wangda


On Mon, Aug 21, 2017 at 2:22 PM, Vinod Kumar Vavilapalli <[hidden email]
> wrote:

> Steve,
>
> You can be strict & ruthless about the timelines. Anything that doesn’t
> get in by mid-September, as was originally planned, can move to the next
> release - whether it is feature work on branches or feature work on trunk.
>
> The problem I see here is that code & branches being worked on for a year
> are now (apparently) close to being done and we are telling them to hold
> for 7 more months - this is not a reasonable ask..
>
> If you are advocating for a 3.1 plan, I’m sure one of these branch
> ‘owners’ can volunteer. But this is how you get competing releases and
> split bandwidth.
>
> As for compatibility / testing etc, it seems like there is a belief that
> the current ‘scoped’ features are all tested well in these areas and so
> adding more is going to hurt the release. There is no way this is the
> reality, trunk has so many features that have been landing for years, the
> only way we can collectively attempt towards making this stable is by
> getting as many parties together as possible, each verifying stuff that
> they need. Not by excluding specific features.
>
> +Vinod
>
> > This is one of those curse-of-cadence things: The higher your release
> cadence, the less pressure to get "everything in". With a slower cadence,
> more pressure to get stuff in, more pressure to hold up the release, slows
> the cadence, gets even more stuff in, etc. etc.
> >
> > - Andrew has been working on the release for months, we all need to
> appreciate how much hard work that is and has been, especially for what is
> going to be a major release.
> >
> > - We know that things will be unstable in 3.0; Andrew's concern is about
> making sure that the newest, unstablest (?) features can at least be
> bypassed if there are problems. I we should also call out in the release
> notes what we think are the unstable bits where people need to use caution
> (example: S3Guard in "authoritative" mode)
> >
> > - Anything related to wire compatibility has been problematic in the
> past; I think it's essential that whatever packets get sent around are
> going to be stable, so changes there need to be in, or at least the
> payloads set up ready for the features. Same for new public APIs.
> >
> > - As fpr the rest, I don't know. I think being strict about it and
> ruthless in assessing the feature's stability & consequences of postponing
> the feature until a Hadoop 3.1 release in Jan/Feb, with a plan to ship then
> and follow up with a 3.2 in the summer.
> >
> > Then: start planning that 3.1 release. Maybe I should put my hand up as
> release manager for that one. Then everyone would realise how amenable
> Andrew is being today.
> >
> >
> > One other thing: alongside the big branches, there's the eternal backlog
> of small patches. We should organise spending a few days updating,
> reviewing & merging them in
> >
> > -Steve
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email] <mailto:
> [hidden email]>
> > For additional commands, e-mail: [hidden email]
> <mailto:[hidden email]>
>
Reply | Threaded
Open this post in threaded view
|

Re: Branch merges and 3.0.0-beta1 scope

Ray Chiang-2
On 8/22/17 3:20 AM, Steve Loughran wrote:

>> On 21 Aug 2017, at 22:22, Vinod Kumar Vavilapalli <[hidden email]> wrote:
>>
>> Steve,
>>
>> You can be strict & ruthless about the timelines. Anything that doesn’t get in by mid-September, as was originally planned, can move to the next release - whether it is feature work on branches or feature work on trunk.
>>
>> The problem I see here is that code & branches being worked on for a year are now (apparently) close to being done and we are telling them to hold for 7 more months - this is not a reasonable ask..
>>
>> If you are advocating for a 3.1 plan, I’m sure one of these branch ‘owners’ can volunteer. But this is how you get competing releases and split bandwidth.
>>
>> As for compatibility / testing etc, it seems like there is a belief that the current ‘scoped’ features are all tested well in these areas and so adding more is going to hurt the release. There is no way this is the reality, trunk has so many features that have been landing for years, the only way we can collectively attempt towards making this stable is by getting as many parties together as possible, each verifying stuff that they need. Not by excluding specific features.
>>
> If everyone is confident & its coming together, it does make sense. I think those of us (myself included) who are merging stuff in do have to recognise that we really need to follow it through by being responsive to any problem -and with the release manager having the right to pull things out if its felt to be significantly threatening the stability of the final 3.0 release.
>
> I think we should also consider making the 3.0 beta the feature freeze; after that fixes on the features go in, but nothing else of significance, otherwise the value of the beta "test this code more broadly" is diminoshed
At this point, there have been three planned alphas from September 2016
until July 2017 to "get in features".  While a couple of upcoming
features are "a few weeks" away, I think all of us are aware how
predictable software development schedules can be.  I think we can also
all agree that rushing just to meet a release deadline isn't the best
practice when it comes to software development either.

Andrew has been very clear about his goals at each step and I think
Wangda's willingness to not rush in resource types was an appropriate
response.  I'm sympathetic to the goals of getting in a feature for 3.0,
but it might be a good idea for each project that is a "few weeks away"
to seriously look at the readiness compared to the features which have
been testing for 6+ months already.

-Ray


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Branch merges and 3.0.0-beta1 scope

Vinod Kumar Vavilapalli-2
Agreed. I was very clearly not advocating for rushing in features. If you have followed my past emails, I have only strongly advocated features be worked in branches and get merged when they are in a reasonable state.

Each branch contributor group should look at their readiness and merge stuff in assuming that the branch reached a satisfactory state. That’s it.

From release management perspective, blocking features just because we are a month close to the deadline is not reasonable. Let the branch contributors rationalize, make this decision and the rest of us can support them in making the decision.

+Vinod

> At this point, there have been three planned alphas from September 2016 until July 2017 to "get in features".  While a couple of upcoming features are "a few weeks" away, I think all of us are aware how predictable software development schedules can be.  I think we can also all agree that rushing just to meet a release deadline isn't the best practice when it comes to software development either.
>
> Andrew has been very clear about his goals at each step and I think Wangda's willingness to not rush in resource types was an appropriate response.  I'm sympathetic to the goals of getting in a feature for 3.0, but it might be a good idea for each project that is a "few weeks away" to seriously look at the readiness compared to the features which have been testing for 6+ months already.
>
> -Ray