[PROPOSAL] Hadoop branch-1.2

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[PROPOSAL] Hadoop branch-1.2

Matt Foley-2
Hi,
this weekend I propose to create a branch-1.2 from current branch-1, and
create a first release candidate.  There's a ton of new stuff since 1.1,
and it will probably need stabilization, but that is best done in the
context of a release candidate.

Any objections?  Any must-have items for 1.2 that can't be in by then?
 (There's always 1.3... :-)

Thanks,
--Matt
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Hadoop branch-1.2

Steve Loughran-3
I've been working on some stricter contract tests for filesystems, and have
some changes for S3 and S3N that we've been working through. While this is
mostly minor, rename(dir,dir/subdir) can lose all data in the source
directory

https://issues.apache.org/jira/browse/HADOOP-9261

I think that should be one to pick up -it and the extended test are still
be reviewed

On 4 February 2013 12:39, Matt Foley <[hidden email]> wrote:

> Hi,
> this weekend I propose to create a branch-1.2 from current branch-1, and
> create a first release candidate.  There's a ton of new stuff since 1.1,
> and it will probably need stabilization, but that is best done in the
> context of a release candidate.
>
> Any objections?  Any must-have items for 1.2 that can't be in by then?
>  (There's always 1.3... :-)
>
> Thanks,
> --Matt
>
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Hadoop branch-1.2

Matt Foley-2
Hi, I got stuck in other work and did not make the Hadoop 1.2 branch in
February.
Now that release 1.1.2 is out, I'm ready to make the 1.2 branch.

I intend to branch for 1.2 in the next night or two, and at that point will
make the formal proposal
to schedule an rc a week after that.  If any issues, please let me know.

Regards,
--Matt


On Mon, Feb 4, 2013 at 7:02 PM, Steve Loughran <[hidden email]>wrote:

> I've been working on some stricter contract tests for filesystems, and have
> some changes for S3 and S3N that we've been working through. While this is
> mostly minor, rename(dir,dir/subdir) can lose all data in the source
> directory
>
> https://issues.apache.org/jira/browse/HADOOP-9261
>
> I think that should be one to pick up -it and the extended test are still
> be reviewed
>
> On 4 February 2013 12:39, Matt Foley <[hidden email]> wrote:
>
> > Hi,
> > this weekend I propose to create a branch-1.2 from current branch-1, and
> > create a first release candidate.  There's a ton of new stuff since 1.1,
> > and it will probably need stabilization, but that is best done in the
> > context of a release candidate.
> >
> > Any objections?  Any must-have items for 1.2 that can't be in by then?
> >  (There's always 1.3... :-)
> >
> > Thanks,
> > --Matt
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Hadoop branch-1.2

Steve Loughran-3
On 6 March 2013 23:17, Matt Foley <[hidden email]> wrote:

> Hi, I got stuck in other work and did not make the Hadoop 1.2 branch in
> February.
> Now that release 1.1.2 is out, I'm ready to make the 1.2 branch.
>
> I intend to branch for 1.2 in the next night or two, and at that point will
> make the formal proposal
> to schedule an rc a week after that.  If any issues, please let me know.
>
> Regards,
> --Matt
>

I'd like to get my FS contract tests in there with the fixes for s3n and s3
to avoid data loss if you try to move a directory under itself.

https://issues.apache.org/jira/browse/HADOOP-9258
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Hadoop branch-1.2

Matt Foley
Steve, I'm okay with this, but
HADOOP-9258<https://issues.apache.org/jira/browse/HADOOP-9258> is
dependent on  HADOOP-9265<https://issues.apache.org/jira/browse/HADOOP-9265>,
which itself seems ready to commit.  As I read the comments, both are ready
to commit.  Please let me know if this is correct.

In meantime, I'm going to make the branch.  We'll just have to commit to
both branch-1 and branch-1.2.

Thanks,
--Matt


On Thu, Mar 7, 2013 at 1:29 AM, Steve Loughran <[hidden email]>wrote:

> On 6 March 2013 23:17, Matt Foley <[hidden email]> wrote:
>
> > Hi, I got stuck in other work and did not make the Hadoop 1.2 branch in
> > February.
> > Now that release 1.1.2 is out, I'm ready to make the 1.2 branch.
> >
> > I intend to branch for 1.2 in the next night or two, and at that point
> will
> > make the formal proposal
> > to schedule an rc a week after that.  If any issues, please let me know.
> >
> > Regards,
> > --Matt
> >
>
> I'd like to get my FS contract tests in there with the fixes for s3n and s3
> to avoid data loss if you try to move a directory under itself.
>
> https://issues.apache.org/jira/browse/HADOOP-9258
>
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Hadoop branch-1.2

Suresh Srinivas-2
Steve, is there a timeline all these changes will be ready? If it is not
going to be
ready soon, perhaps these changes could be consider for 1.3?


On Sat, Mar 9, 2013 at 2:38 PM, Matt Foley <[hidden email]> wrote:

> Steve, I'm okay with this, but
> HADOOP-9258<https://issues.apache.org/jira/browse/HADOOP-9258> is
> dependent on  HADOOP-9265<
> https://issues.apache.org/jira/browse/HADOOP-9265>,
> which itself seems ready to commit.  As I read the comments, both are ready
> to commit.  Please let me know if this is correct.
>
> In meantime, I'm going to make the branch.  We'll just have to commit to
> both branch-1 and branch-1.2.
>
> Thanks,
> --Matt
>
>
> On Thu, Mar 7, 2013 at 1:29 AM, Steve Loughran <[hidden email]
> >wrote:
>
> > On 6 March 2013 23:17, Matt Foley <[hidden email]> wrote:
> >
> > > Hi, I got stuck in other work and did not make the Hadoop 1.2 branch in
> > > February.
> > > Now that release 1.1.2 is out, I'm ready to make the 1.2 branch.
> > >
> > > I intend to branch for 1.2 in the next night or two, and at that point
> > will
> > > make the formal proposal
> > > to schedule an rc a week after that.  If any issues, please let me
> know.
> > >
> > > Regards,
> > > --Matt
> > >
> >
> > I'd like to get my FS contract tests in there with the fixes for s3n and
> s3
> > to avoid data loss if you try to move a directory under itself.
> >
> > https://issues.apache.org/jira/browse/HADOOP-9258
> >
>



--
http://hortonworks.com/download/
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Hadoop branch-1.2

Steve Loughran-3
let me do one quick merge of the current S3 one, if there's a problem pull
tomW in and try to get this closed off. As it risks data I care about it

On 10 March 2013 18:32, Suresh Srinivas <[hidden email]> wrote:

> Steve, is there a timeline all these changes will be ready? If it is not
> going to be
> ready soon, perhaps these changes could be consider for 1.3?
>
>
> On Sat, Mar 9, 2013 at 2:38 PM, Matt Foley <[hidden email]> wrote:
>
> > Steve, I'm okay with this, but
> > HADOOP-9258<https://issues.apache.org/jira/browse/HADOOP-9258> is
> > dependent on  HADOOP-9265<
> > https://issues.apache.org/jira/browse/HADOOP-9265>,
> > which itself seems ready to commit.  As I read the comments, both are
> ready
> > to commit.  Please let me know if this is correct.
> >
> > In meantime, I'm going to make the branch.  We'll just have to commit to
> > both branch-1 and branch-1.2.
> >
> > Thanks,
> > --Matt
> >
> >
> > On Thu, Mar 7, 2013 at 1:29 AM, Steve Loughran <[hidden email]
> > >wrote:
> >
> > > On 6 March 2013 23:17, Matt Foley <[hidden email]> wrote:
> > >
> > > > Hi, I got stuck in other work and did not make the Hadoop 1.2 branch
> in
> > > > February.
> > > > Now that release 1.1.2 is out, I'm ready to make the 1.2 branch.
> > > >
> > > > I intend to branch for 1.2 in the next night or two, and at that
> point
> > > will
> > > > make the formal proposal
> > > > to schedule an rc a week after that.  If any issues, please let me
> > know.
> > > >
> > > > Regards,
> > > > --Matt
> > > >
> > >
> > > I'd like to get my FS contract tests in there with the fixes for s3n
> and
> > s3
> > > to avoid data loss if you try to move a directory under itself.
> > >
> > > https://issues.apache.org/jira/browse/HADOOP-9258
> > >
> >
>
>
>
> --
> http://hortonworks.com/download/
>
Reply | Threaded
Open this post in threaded view
|

Re: [PROPOSAL] Hadoop branch-1.2

Steve Loughran-3
In reply to this post by Suresh Srinivas-2
On 10 March 2013 18:32, Suresh Srinivas <[hidden email]> wrote:

> Steve, is there a timeline all these changes will be ready? If it is not
> going to be
> ready soon, perhaps these changes could be consider for 1.3?
>
> Suresh, I've put everything into
https://issues.apache.org/jira/browse/HADOOP-9258 - the tests and the
patches for S3 and S3N.

Data loss is only going to happen if you go rename(srcDir, srcDir) or
rename(srcDir,srcDir/child); nobody has ever reported it. We have to decide
whether this is critical enough to get in, or postpone to 1.3. Given this
is a last minute change and the changes to the S3FileSystem's rename()
operation are significant, delaying to 1.3 may actually be safer.

your choice

-steve