[REVIEW] Hadoop-1.1.0-RC4

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[REVIEW] Hadoop-1.1.0-RC4

Matt Foley
Hi,
Please review this release candidate for Hadoop-1.1.0.  As suggested
before, I am posting it for review, and will start a vote on it if it
passes the next week with no serious issues being found.

Tarballs, rpms, and debs are available at
    http://people.apache.org/~mattf/hadoop-1.1.0-rc4/
The release notes are also at the top level of that directory.

Nexus seems a bit wedged currently.  I will try again to push to Nexus
tomorrow morning.

Thanks,
--Matt
Reply | Threaded
Open this post in threaded view
|

Re: [REVIEW] Hadoop-1.1.0-RC4

Matt Foley-2
The RC has been uploaded to Nexus / maven.
--Matt

On Mon, Sep 17, 2012 at 6:46 PM, Matt Foley <[hidden email]> wrote:

> Hi,
> Please review this release candidate for Hadoop-1.1.0.  As suggested
> before, I am posting it for review, and will start a vote on it if it
> passes the next week with no serious issues being found.
>
> Tarballs, rpms, and debs are available at
>     http://people.apache.org/~mattf/hadoop-1.1.0-rc4/
> The release notes are also at the top level of that directory.
>
> Nexus seems a bit wedged currently.  I will try again to push to Nexus
> tomorrow morning.
>
> Thanks,
> --Matt
>
Reply | Threaded
Open this post in threaded view
|

Re: [REVIEW] Hadoop-1.1.0-RC4

Vinod Vavilapalli

Matt,

Brandon has a backport for MAPREDUCE-461, which we can pull in if that is okay. Plz let me know.

Thanks,
+Vinod

On Sep 19, 2012, at 1:09 PM, Matt Foley wrote:

> The RC has been uploaded to Nexus / maven.
> --Matt
>
> On Mon, Sep 17, 2012 at 6:46 PM, Matt Foley <[hidden email]> wrote:
>
>> Hi,
>> Please review this release candidate for Hadoop-1.1.0.  As suggested
>> before, I am posting it for review, and will start a vote on it if it
>> passes the next week with no serious issues being found.
>>
>> Tarballs, rpms, and debs are available at
>>    http://people.apache.org/~mattf/hadoop-1.1.0-rc4/
>> The release notes are also at the top level of that directory.
>>
>> Nexus seems a bit wedged currently.  I will try again to push to Nexus
>> tomorrow morning.
>>
>> Thanks,
>> --Matt
>>

Reply | Threaded
Open this post in threaded view
|

Re: [REVIEW] Hadoop-1.1.0-RC4

Matt Foley-2
In reply to this post by Matt Foley-2
Hi all,
Since the Review RC went out, I've been asked to include the following
fixes:

Three significant bugs:
HDFS-3461 hftp uses http to the Namenode's https port, which doesn't work
HDFS-3846 NN deadlock
HDFS-3596 decrease the incidence of corrupted logs after disk full
conditions
and a related enhancement:
HDFS-3521 deal with edit log corruption if it occurs

Four unit test fixes (obviously not critical, but good to have):
HDFS-3698
HDFS-3966
MAPREDUCE-4673
MAPREDUCE-4675

and four performance enhancements:
HADOOP-8617, HDFS-496, MAPREDUCE-782 performance of CRCs calculations
HDFS-2751 Datanode drops OS cache behind reads even for short reads
MAPREDUCE-1906 default minimum heartbeat
MAPREDUCE-3289 use fadvise

I will spin a new RC with these fixes, and hope to offer it for vote by
Wednesday.
Thanks,
--Matt


On Wed, Sep 19, 2012 at 1:09 PM, Matt Foley <[hidden email]> wrote:

> The RC has been uploaded to Nexus / maven.
> --Matt
>
>
> On Mon, Sep 17, 2012 at 6:46 PM, Matt Foley <[hidden email]> wrote:
>
>> Hi,
>> Please review this release candidate for Hadoop-1.1.0.  As suggested
>> before, I am posting it for review, and will start a vote on it if it
>> passes the next week with no serious issues being found.
>>
>> Tarballs, rpms, and debs are available at
>>     http://people.apache.org/~mattf/hadoop-1.1.0-rc4/
>> The release notes are also at the top level of that directory.
>>
>> Nexus seems a bit wedged currently.  I will try again to push to Nexus
>> tomorrow morning.
>>
>> Thanks,
>> --Matt
>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [REVIEW] Hadoop-1.1.0-RC4

Uma Maheswara Rao G-2
Hi Matt,

Good to include HDFS-3701 also. I set the fixed versions as 1.1.0 as well.

Regards,
Uma

On Tue, Sep 25, 2012 at 7:29 AM, Matt Foley <[hidden email]> wrote:

> Hi all,
> Since the Review RC went out, I've been asked to include the following
> fixes:
>
> Three significant bugs:
> HDFS-3461 hftp uses http to the Namenode's https port, which doesn't work
> HDFS-3846 NN deadlock
> HDFS-3596 decrease the incidence of corrupted logs after disk full
> conditions
> and a related enhancement:
> HDFS-3521 deal with edit log corruption if it occurs
>
> Four unit test fixes (obviously not critical, but good to have):
> HDFS-3698
> HDFS-3966
> MAPREDUCE-4673
> MAPREDUCE-4675
>
> and four performance enhancements:
> HADOOP-8617, HDFS-496, MAPREDUCE-782 performance of CRCs calculations
> HDFS-2751 Datanode drops OS cache behind reads even for short reads
> MAPREDUCE-1906 default minimum heartbeat
> MAPREDUCE-3289 use fadvise
>
> I will spin a new RC with these fixes, and hope to offer it for vote by
> Wednesday.
> Thanks,
> --Matt
>
>
> On Wed, Sep 19, 2012 at 1:09 PM, Matt Foley <[hidden email]> wrote:
>
>> The RC has been uploaded to Nexus / maven.
>> --Matt
>>
>>
>> On Mon, Sep 17, 2012 at 6:46 PM, Matt Foley <[hidden email]> wrote:
>>
>>> Hi,
>>> Please review this release candidate for Hadoop-1.1.0.  As suggested
>>> before, I am posting it for review, and will start a vote on it if it
>>> passes the next week with no serious issues being found.
>>>
>>> Tarballs, rpms, and debs are available at
>>>     http://people.apache.org/~mattf/hadoop-1.1.0-rc4/
>>> The release notes are also at the top level of that directory.
>>>
>>> Nexus seems a bit wedged currently.  I will try again to push to Nexus
>>> tomorrow morning.
>>>
>>> Thanks,
>>> --Matt
>>>
>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: [REVIEW] Hadoop-1.1.0-RC4

Matt Foley
Okay.

On Mon, Sep 24, 2012 at 7:05 PM, Uma Maheswara Rao G
<[hidden email]>wrote:

> Hi Matt,
>
> Good to include HDFS-3701 also. I set the fixed versions as 1.1.0 as well.
>
> Regards,
> Uma
>
> On Tue, Sep 25, 2012 at 7:29 AM, Matt Foley <[hidden email]> wrote:
> > Hi all,
> > Since the Review RC went out, I've been asked to include the following
> > fixes:
> >
> > Three significant bugs:
> > HDFS-3461 hftp uses http to the Namenode's https port, which doesn't work
> > HDFS-3846 NN deadlock
> > HDFS-3596 decrease the incidence of corrupted logs after disk full
> > conditions
> > and a related enhancement:
> > HDFS-3521 deal with edit log corruption if it occurs
> >
> > Four unit test fixes (obviously not critical, but good to have):
> > HDFS-3698
> > HDFS-3966
> > MAPREDUCE-4673
> > MAPREDUCE-4675
> >
> > and four performance enhancements:
> > HADOOP-8617, HDFS-496, MAPREDUCE-782 performance of CRCs calculations
> > HDFS-2751 Datanode drops OS cache behind reads even for short reads
> > MAPREDUCE-1906 default minimum heartbeat
> > MAPREDUCE-3289 use fadvise
> >
> > I will spin a new RC with these fixes, and hope to offer it for vote by
> > Wednesday.
> > Thanks,
> > --Matt
> >
> >
> > On Wed, Sep 19, 2012 at 1:09 PM, Matt Foley <[hidden email]> wrote:
> >
> >> The RC has been uploaded to Nexus / maven.
> >> --Matt
> >>
> >>
> >> On Mon, Sep 17, 2012 at 6:46 PM, Matt Foley <[hidden email]> wrote:
> >>
> >>> Hi,
> >>> Please review this release candidate for Hadoop-1.1.0.  As suggested
> >>> before, I am posting it for review, and will start a vote on it if it
> >>> passes the next week with no serious issues being found.
> >>>
> >>> Tarballs, rpms, and debs are available at
> >>>     http://people.apache.org/~mattf/hadoop-1.1.0-rc4/
> >>> The release notes are also at the top level of that directory.
> >>>
> >>> Nexus seems a bit wedged currently.  I will try again to push to Nexus
> >>> tomorrow morning.
> >>>
> >>> Thanks,
> >>> --Matt
> >>>
> >>
> >>
>
Reply | Threaded
Open this post in threaded view
|

Re: [REVIEW] Hadoop-1.1.0-RC4

Arun Murthy
I'd also recommend MR-4558, MR-4328 & H-8748+friends. Thoughts?

thanks,
Arun

On Sep 24, 2012, at 9:58 PM, Matt Foley wrote:

> Okay.
>
> On Mon, Sep 24, 2012 at 7:05 PM, Uma Maheswara Rao G
> <[hidden email]>wrote:
>
>> Hi Matt,
>>
>> Good to include HDFS-3701 also. I set the fixed versions as 1.1.0 as well.
>>
>> Regards,
>> Uma
>>
>> On Tue, Sep 25, 2012 at 7:29 AM, Matt Foley <[hidden email]> wrote:
>>> Hi all,
>>> Since the Review RC went out, I've been asked to include the following
>>> fixes:
>>>
>>> Three significant bugs:
>>> HDFS-3461 hftp uses http to the Namenode's https port, which doesn't work
>>> HDFS-3846 NN deadlock
>>> HDFS-3596 decrease the incidence of corrupted logs after disk full
>>> conditions
>>> and a related enhancement:
>>> HDFS-3521 deal with edit log corruption if it occurs
>>>
>>> Four unit test fixes (obviously not critical, but good to have):
>>> HDFS-3698
>>> HDFS-3966
>>> MAPREDUCE-4673
>>> MAPREDUCE-4675
>>>
>>> and four performance enhancements:
>>> HADOOP-8617, HDFS-496, MAPREDUCE-782 performance of CRCs calculations
>>> HDFS-2751 Datanode drops OS cache behind reads even for short reads
>>> MAPREDUCE-1906 default minimum heartbeat
>>> MAPREDUCE-3289 use fadvise
>>>
>>> I will spin a new RC with these fixes, and hope to offer it for vote by
>>> Wednesday.
>>> Thanks,
>>> --Matt
>>>
>>>
>>> On Wed, Sep 19, 2012 at 1:09 PM, Matt Foley <[hidden email]> wrote:
>>>
>>>> The RC has been uploaded to Nexus / maven.
>>>> --Matt
>>>>
>>>>
>>>> On Mon, Sep 17, 2012 at 6:46 PM, Matt Foley <[hidden email]> wrote:
>>>>
>>>>> Hi,
>>>>> Please review this release candidate for Hadoop-1.1.0.  As suggested
>>>>> before, I am posting it for review, and will start a vote on it if it
>>>>> passes the next week with no serious issues being found.
>>>>>
>>>>> Tarballs, rpms, and debs are available at
>>>>>    http://people.apache.org/~mattf/hadoop-1.1.0-rc4/
>>>>> The release notes are also at the top level of that directory.
>>>>>
>>>>> Nexus seems a bit wedged currently.  I will try again to push to Nexus
>>>>> tomorrow morning.
>>>>>
>>>>> Thanks,
>>>>> --Matt
>>>>>
>>>>
>>>>
>>

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/


Reply | Threaded
Open this post in threaded view
|

Re: [REVIEW] Hadoop-1.1.0-RC4

Arun Murthy

On Sep 25, 2012, at 8:40 AM, Arun C Murthy wrote:

> I'd also recommend MR-4558, MR-4328 & H-8748+friends. Thoughts?

Also, MR-3837.

Initially, I didn't recommend them since I thought 1.1 was too close, but since these (MR-3837 & MR-4328 etc.) are well-baked at this point (deployed at multiple places) I'd like to include them in.

thanks,
Arun

>
> thanks,
> Arun
>
> On Sep 24, 2012, at 9:58 PM, Matt Foley wrote:
>
>> Okay.
>>
>> On Mon, Sep 24, 2012 at 7:05 PM, Uma Maheswara Rao G
>> <[hidden email]>wrote:
>>
>>> Hi Matt,
>>>
>>> Good to include HDFS-3701 also. I set the fixed versions as 1.1.0 as well.
>>>
>>> Regards,
>>> Uma
>>>
>>> On Tue, Sep 25, 2012 at 7:29 AM, Matt Foley <[hidden email]> wrote:
>>>> Hi all,
>>>> Since the Review RC went out, I've been asked to include the following
>>>> fixes:
>>>>
>>>> Three significant bugs:
>>>> HDFS-3461 hftp uses http to the Namenode's https port, which doesn't work
>>>> HDFS-3846 NN deadlock
>>>> HDFS-3596 decrease the incidence of corrupted logs after disk full
>>>> conditions
>>>> and a related enhancement:
>>>> HDFS-3521 deal with edit log corruption if it occurs
>>>>
>>>> Four unit test fixes (obviously not critical, but good to have):
>>>> HDFS-3698
>>>> HDFS-3966
>>>> MAPREDUCE-4673
>>>> MAPREDUCE-4675
>>>>
>>>> and four performance enhancements:
>>>> HADOOP-8617, HDFS-496, MAPREDUCE-782 performance of CRCs calculations
>>>> HDFS-2751 Datanode drops OS cache behind reads even for short reads
>>>> MAPREDUCE-1906 default minimum heartbeat
>>>> MAPREDUCE-3289 use fadvise
>>>>
>>>> I will spin a new RC with these fixes, and hope to offer it for vote by
>>>> Wednesday.
>>>> Thanks,
>>>> --Matt
>>>>
>>>>
>>>> On Wed, Sep 19, 2012 at 1:09 PM, Matt Foley <[hidden email]> wrote:
>>>>
>>>>> The RC has been uploaded to Nexus / maven.
>>>>> --Matt
>>>>>
>>>>>
>>>>> On Mon, Sep 17, 2012 at 6:46 PM, Matt Foley <[hidden email]> wrote:
>>>>>
>>>>>> Hi,
>>>>>> Please review this release candidate for Hadoop-1.1.0.  As suggested
>>>>>> before, I am posting it for review, and will start a vote on it if it
>>>>>> passes the next week with no serious issues being found.
>>>>>>
>>>>>> Tarballs, rpms, and debs are available at
>>>>>>    http://people.apache.org/~mattf/hadoop-1.1.0-rc4/
>>>>>> The release notes are also at the top level of that directory.
>>>>>>
>>>>>> Nexus seems a bit wedged currently.  I will try again to push to Nexus
>>>>>> tomorrow morning.
>>>>>>
>>>>>> Thanks,
>>>>>> --Matt
>>>>>>
>>>>>
>>>>>
>>>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/


Reply | Threaded
Open this post in threaded view
|

Re: [REVIEW] Hadoop-1.1.0-RC4

Matt Foley
Hi Arun,
can you get them committed to branch-1 and branch-1.1 today?
--Matt

On Tue, Sep 25, 2012 at 8:44 AM, Arun C Murthy <[hidden email]> wrote:

>
> On Sep 25, 2012, at 8:40 AM, Arun C Murthy wrote:
>
> > I'd also recommend MR-4558, MR-4328 & H-8748+friends. Thoughts?
>
> Also, MR-3837.
>
> Initially, I didn't recommend them since I thought 1.1 was too close, but
> since these (MR-3837 & MR-4328 etc.) are well-baked at this point (deployed
> at multiple places) I'd like to include them in.
>
> thanks,
> Arun
>
> >
> > thanks,
> > Arun
> >
> > On Sep 24, 2012, at 9:58 PM, Matt Foley wrote:
> >
> >> Okay.
> >>
> >> On Mon, Sep 24, 2012 at 7:05 PM, Uma Maheswara Rao G
> >> <[hidden email]>wrote:
> >>
> >>> Hi Matt,
> >>>
> >>> Good to include HDFS-3701 also. I set the fixed versions as 1.1.0 as
> well.
> >>>
> >>> Regards,
> >>> Uma
> >>>
> >>> On Tue, Sep 25, 2012 at 7:29 AM, Matt Foley <[hidden email]> wrote:
> >>>> Hi all,
> >>>> Since the Review RC went out, I've been asked to include the following
> >>>> fixes:
> >>>>
> >>>> Three significant bugs:
> >>>> HDFS-3461 hftp uses http to the Namenode's https port, which doesn't
> work
> >>>> HDFS-3846 NN deadlock
> >>>> HDFS-3596 decrease the incidence of corrupted logs after disk full
> >>>> conditions
> >>>> and a related enhancement:
> >>>> HDFS-3521 deal with edit log corruption if it occurs
> >>>>
> >>>> Four unit test fixes (obviously not critical, but good to have):
> >>>> HDFS-3698
> >>>> HDFS-3966
> >>>> MAPREDUCE-4673
> >>>> MAPREDUCE-4675
> >>>>
> >>>> and four performance enhancements:
> >>>> HADOOP-8617, HDFS-496, MAPREDUCE-782 performance of CRCs calculations
> >>>> HDFS-2751 Datanode drops OS cache behind reads even for short reads
> >>>> MAPREDUCE-1906 default minimum heartbeat
> >>>> MAPREDUCE-3289 use fadvise
> >>>>
> >>>> I will spin a new RC with these fixes, and hope to offer it for vote
> by
> >>>> Wednesday.
> >>>> Thanks,
> >>>> --Matt
> >>>>
> >>>>
> >>>> On Wed, Sep 19, 2012 at 1:09 PM, Matt Foley <[hidden email]> wrote:
> >>>>
> >>>>> The RC has been uploaded to Nexus / maven.
> >>>>> --Matt
> >>>>>
> >>>>>
> >>>>> On Mon, Sep 17, 2012 at 6:46 PM, Matt Foley <[hidden email]>
> wrote:
> >>>>>
> >>>>>> Hi,
> >>>>>> Please review this release candidate for Hadoop-1.1.0.  As suggested
> >>>>>> before, I am posting it for review, and will start a vote on it if
> it
> >>>>>> passes the next week with no serious issues being found.
> >>>>>>
> >>>>>> Tarballs, rpms, and debs are available at
> >>>>>>    http://people.apache.org/~mattf/hadoop-1.1.0-rc4/
> >>>>>> The release notes are also at the top level of that directory.
> >>>>>>
> >>>>>> Nexus seems a bit wedged currently.  I will try again to push to
> Nexus
> >>>>>> tomorrow morning.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> --Matt
> >>>>>>
> >>>>>
> >>>>>
> >>>
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> >
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [REVIEW] Hadoop-1.1.0-RC4

Arun Murthy
Yes, on it. Thanks Matt.

On Sep 25, 2012, at 9:46 AM, Matt Foley wrote:

> Hi Arun,
> can you get them committed to branch-1 and branch-1.1 today?
> --Matt
>
> On Tue, Sep 25, 2012 at 8:44 AM, Arun C Murthy <[hidden email]> wrote:
>
>>
>> On Sep 25, 2012, at 8:40 AM, Arun C Murthy wrote:
>>
>>> I'd also recommend MR-4558, MR-4328 & H-8748+friends. Thoughts?
>>
>> Also, MR-3837.
>>
>> Initially, I didn't recommend them since I thought 1.1 was too close, but
>> since these (MR-3837 & MR-4328 etc.) are well-baked at this point (deployed
>> at multiple places) I'd like to include them in.
>>
>> thanks,
>> Arun
>>
>>>
>>> thanks,
>>> Arun
>>>
>>> On Sep 24, 2012, at 9:58 PM, Matt Foley wrote:
>>>
>>>> Okay.
>>>>
>>>> On Mon, Sep 24, 2012 at 7:05 PM, Uma Maheswara Rao G
>>>> <[hidden email]>wrote:
>>>>
>>>>> Hi Matt,
>>>>>
>>>>> Good to include HDFS-3701 also. I set the fixed versions as 1.1.0 as
>> well.
>>>>>
>>>>> Regards,
>>>>> Uma
>>>>>
>>>>> On Tue, Sep 25, 2012 at 7:29 AM, Matt Foley <[hidden email]> wrote:
>>>>>> Hi all,
>>>>>> Since the Review RC went out, I've been asked to include the following
>>>>>> fixes:
>>>>>>
>>>>>> Three significant bugs:
>>>>>> HDFS-3461 hftp uses http to the Namenode's https port, which doesn't
>> work
>>>>>> HDFS-3846 NN deadlock
>>>>>> HDFS-3596 decrease the incidence of corrupted logs after disk full
>>>>>> conditions
>>>>>> and a related enhancement:
>>>>>> HDFS-3521 deal with edit log corruption if it occurs
>>>>>>
>>>>>> Four unit test fixes (obviously not critical, but good to have):
>>>>>> HDFS-3698
>>>>>> HDFS-3966
>>>>>> MAPREDUCE-4673
>>>>>> MAPREDUCE-4675
>>>>>>
>>>>>> and four performance enhancements:
>>>>>> HADOOP-8617, HDFS-496, MAPREDUCE-782 performance of CRCs calculations
>>>>>> HDFS-2751 Datanode drops OS cache behind reads even for short reads
>>>>>> MAPREDUCE-1906 default minimum heartbeat
>>>>>> MAPREDUCE-3289 use fadvise
>>>>>>
>>>>>> I will spin a new RC with these fixes, and hope to offer it for vote
>> by
>>>>>> Wednesday.
>>>>>> Thanks,
>>>>>> --Matt
>>>>>>
>>>>>>
>>>>>> On Wed, Sep 19, 2012 at 1:09 PM, Matt Foley <[hidden email]> wrote:
>>>>>>
>>>>>>> The RC has been uploaded to Nexus / maven.
>>>>>>> --Matt
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Sep 17, 2012 at 6:46 PM, Matt Foley <[hidden email]>
>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>> Please review this release candidate for Hadoop-1.1.0.  As suggested
>>>>>>>> before, I am posting it for review, and will start a vote on it if
>> it
>>>>>>>> passes the next week with no serious issues being found.
>>>>>>>>
>>>>>>>> Tarballs, rpms, and debs are available at
>>>>>>>>   http://people.apache.org/~mattf/hadoop-1.1.0-rc4/
>>>>>>>> The release notes are also at the top level of that directory.
>>>>>>>>
>>>>>>>> Nexus seems a bit wedged currently.  I will try again to push to
>> Nexus
>>>>>>>> tomorrow morning.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> --Matt
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>>
>>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>>

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/