Re: [VOTE] Release Apache Hadoop 3.1.0 (RC0)

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC0)

Bharat Viswanadham
Hi Wangda,
Maven Artifact repositories is not having all Hadoop jars. (It is missing many like hadoop-hdfs, hadoop-client etc.,)
https://repository.apache.org/content/repositories/orgapachehadoop-1086/


Thanks,
Bharat


On 3/21/18, 11:44 PM, "Wangda Tan" <[hidden email]> wrote:

    Hi folks,
   
    Thanks to the many who helped with this release since Dec 2017 [1]. We've
    created RC0 for Apache Hadoop 3.1.0. The artifacts are available here:
   
    http://people.apache.org/~wangda/hadoop-3.1.0-RC0/
   
    The RC tag in git is release-3.1.0-RC0.
   
    The maven artifacts are available via repository.apache.org at
    https://repository.apache.org/content/repositories/orgapachehadoop-1086/
   
    This vote will run 7 days (5 weekdays), ending on Mar 28 at 11:59 pm
    Pacific.
   
    3.1.0 contains 727 [2] fixed JIRA issues since 3.0.0. Notable additions
    include the first class GPU/FPGA support on YARN, Native services, Support
    rich placement constraints in YARN, S3-related enhancements, allow HDFS
    block replicas to be provided by an external storage system, etc.
   
    We’d like to use this as a starting release for 3.1.x [1], depending on how
    it goes, get it stabilized and potentially use a 3.1.1 in several weeks as
    the stable release.
   
    We have done testing with a pseudo cluster and distributed shell job. My +1
    to start.
   
    Best,
    Wangda/Vinod
   
    [1]
    https://lists.apache.org/thread.html/b3fb3b6da8b6357a68513a6dfd104bc9e19e559aedc5ebedb4ca08c8@%3Cyarn-dev.hadoop.apache.org%3E
    [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.1.0)
    AND fixVersion not in (3.0.0, 3.0.0-beta1) AND status = Resolved ORDER BY
    fixVersion ASC
   


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC0)

Wangda Tan
Thanks @Bharat for the quick check, the previously staged repository has
some issues. I re-deployed jars to nexus.

Here's the new repo (1087)

https://repository.apache.org/content/repositories/orgapachehadoop-1087/

Other artifacts remain same, no additional code changes.

On Wed, Mar 21, 2018 at 11:54 PM, Bharat Viswanadham <
[hidden email]> wrote:

> Hi Wangda,
> Maven Artifact repositories is not having all Hadoop jars. (It is missing
> many like hadoop-hdfs, hadoop-client etc.,)
> https://repository.apache.org/content/repositories/orgapachehadoop-1086/
>
>
> Thanks,
> Bharat
>
>
> On 3/21/18, 11:44 PM, "Wangda Tan" <[hidden email]> wrote:
>
>     Hi folks,
>
>     Thanks to the many who helped with this release since Dec 2017 [1].
> We've
>     created RC0 for Apache Hadoop 3.1.0. The artifacts are available here:
>
>     http://people.apache.org/~wangda/hadoop-3.1.0-RC0/
>
>     The RC tag in git is release-3.1.0-RC0.
>
>     The maven artifacts are available via repository.apache.org at
>     https://repository.apache.org/content/repositories/
> orgapachehadoop-1086/
>
>     This vote will run 7 days (5 weekdays), ending on Mar 28 at 11:59 pm
>     Pacific.
>
>     3.1.0 contains 727 [2] fixed JIRA issues since 3.0.0. Notable additions
>     include the first class GPU/FPGA support on YARN, Native services,
> Support
>     rich placement constraints in YARN, S3-related enhancements, allow HDFS
>     block replicas to be provided by an external storage system, etc.
>
>     We’d like to use this as a starting release for 3.1.x [1], depending
> on how
>     it goes, get it stabilized and potentially use a 3.1.1 in several
> weeks as
>     the stable release.
>
>     We have done testing with a pseudo cluster and distributed shell job.
> My +1
>     to start.
>
>     Best,
>     Wangda/Vinod
>
>     [1]
>     https://lists.apache.org/thread.html/b3fb3b6da8b6357a68513a6dfd104b
> c9e19e559aedc5ebedb4ca08c8@%3Cyarn-dev.hadoop.apache.org%3E
>     [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
> (3.1.0)
>     AND fixVersion not in (3.0.0, 3.0.0-beta1) AND status = Resolved ORDER
> BY
>     fixVersion ASC
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC0)

Weiwei Yang
Hi Wangda

While testing the build, we found a bug https://issues.apache.org/jira/browse/YARN-8085, it might cause NPE during RM fail-over. I think that needs to be included in 3.1.0.

We have run some other tests, and they look good so far. This includes
- Some basic teragen/terasort/DS jobs
- Various of metrics, UI displays, compatible tests
- Creating queues, different queue configurations
- Verified RM fail-over, app-recovery
- Enabled async scheduling
- Enabled placement constraints
- SLS tests

Hope it helps.
Thanks

--
Weiwei

On 29 Mar 2018, 12:34 PM +0800, Ajay Kumar <[hidden email]>, wrote:
+1 (non-binding)

- verified binary checksum
- built from source and setup 4 node cluster
- run basic hdfs command
- run wordcount, pi & TestDFSIO (read/write)

Ajay

On 3/28/18, 5:45 PM, "Jonathan Hung" <[hidden email]> wrote:

Hi Wangda, thanks for handling this release.

+1 (non-binding)

- verified binary checksum
- launched single node RM
- verified refreshQueues functionality
- verified capacity scheduler conf mutation disabled in this case
- verified capacity scheduler conf mutation with leveldb storage
- verified refreshQueues mutation is disabled in this case


Jonathan Hung

On Thu, Mar 22, 2018 at 9:10 AM, Wangda Tan <[hidden email]> wrote:

Thanks @Bharat for the quick check, the previously staged repository has
some issues. I re-deployed jars to nexus.

Here's the new repo (1087)

https://repository.apache.org/content/repositories/orgapachehadoop-1087/

Other artifacts remain same, no additional code changes.

On Wed, Mar 21, 2018 at 11:54 PM, Bharat Viswanadham <
[hidden email]> wrote:

Hi Wangda,
Maven Artifact repositories is not having all Hadoop jars. (It is missing
many like hadoop-hdfs, hadoop-client etc.,)
https://repository.apache.org/content/repositories/orgapachehadoop-1086/


Thanks,
Bharat


On 3/21/18, 11:44 PM, "Wangda Tan" <[hidden email]> wrote:

Hi folks,

Thanks to the many who helped with this release since Dec 2017 [1].
We've
created RC0 for Apache Hadoop 3.1.0. The artifacts are available
here:

http://people.apache.org/~wangda/hadoop-3.1.0-RC0/

The RC tag in git is release-3.1.0-RC0.

The maven artifacts are available via repository.apache.org at
https://repository.apache.org/content/repositories/
orgapachehadoop-1086/

This vote will run 7 days (5 weekdays), ending on Mar 28 at 11:59 pm
Pacific.

3.1.0 contains 727 [2] fixed JIRA issues since 3.0.0. Notable
additions
include the first class GPU/FPGA support on YARN, Native services,
Support
rich placement constraints in YARN, S3-related enhancements, allow
HDFS
block replicas to be provided by an external storage system, etc.

We’d like to use this as a starting release for 3.1.x [1], depending
on how
it goes, get it stabilized and potentially use a 3.1.1 in several
weeks as
the stable release.

We have done testing with a pseudo cluster and distributed shell job.
My +1
to start.

Best,
Wangda/Vinod

[1]
https://lists.apache.org/thread.html/b3fb3b6da8b6357a68513a6dfd104b
c9e19e559aedc5ebedb4ca08c8@%3Cyarn-dev.hadoop.apache.org%3E
[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
(3.1.0)
AND fixVersion not in (3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER
BY
fixVersion ASC






B�KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKCB��[��X��ܚX�KK[XZ[���Y]�][��X��ܚX�PY���\X�K�ܙ�B��܈Y][ۘ[��[X[��K[XZ[���Y]�Z[Y���\X�K�ܙ�B�
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC0)

Wangda Tan
Thanks all for doing testing and reporting issues.

YARN-8085 discovered by Weiwei/Tao is a blocker, we will roll an RC1 today.

- Wangda

On Thu, Mar 29, 2018 at 7:26 AM, Weiwei Yang <[hidden email]> wrote:

> Hi Wangda
>
> While testing the build, we found a bug https://issues.apache.org/
> jira/browse/YARN-8085, it might cause NPE during RM fail-over. I think
> that needs to be included in 3.1.0.
>
> We have run some other tests, and they look good so far. This includes
> - Some basic teragen/terasort/DS jobs
> - Various of metrics, UI displays, compatible tests
> - Creating queues, different queue configurations
> - Verified RM fail-over, app-recovery
> - Enabled async scheduling
> - Enabled placement constraints
> - SLS tests
>
> Hope it helps.
> Thanks
>
> --
> Weiwei
>
> On 29 Mar 2018, 12:34 PM +0800, Ajay Kumar <[hidden email]>,
> wrote:
>
> +1 (non-binding)
>
> - verified binary checksum
> - built from source and setup 4 node cluster
> - run basic hdfs command
> - run wordcount, pi & TestDFSIO (read/write)
>
> Ajay
>
> On 3/28/18, 5:45 PM, "Jonathan Hung" <[hidden email]> wrote:
>
> Hi Wangda, thanks for handling this release.
>
> +1 (non-binding)
>
> - verified binary checksum
> - launched single node RM
> - verified refreshQueues functionality
> - verified capacity scheduler conf mutation disabled in this case
> - verified capacity scheduler conf mutation with leveldb storage
> - verified refreshQueues mutation is disabled in this case
>
>
> Jonathan Hung
>
> On Thu, Mar 22, 2018 at 9:10 AM, Wangda Tan <[hidden email]> wrote:
>
> Thanks @Bharat for the quick check, the previously staged repository has
> some issues. I re-deployed jars to nexus.
>
> Here's the new repo (1087)
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1087/
>
> Other artifacts remain same, no additional code changes.
>
> On Wed, Mar 21, 2018 at 11:54 PM, Bharat Viswanadham <
> [hidden email]> wrote:
>
> Hi Wangda,
> Maven Artifact repositories is not having all Hadoop jars. (It is missing
> many like hadoop-hdfs, hadoop-client etc.,)
> https://repository.apache.org/content/repositories/orgapachehadoop-1086/
>
>
> Thanks,
> Bharat
>
>
> On 3/21/18, 11:44 PM, "Wangda Tan" <[hidden email]> wrote:
>
> Hi folks,
>
> Thanks to the many who helped with this release since Dec 2017 [1].
> We've
> created RC0 for Apache Hadoop 3.1.0. The artifacts are available
>
> here:
>
>
> http://people.apache.org/~wangda/hadoop-3.1.0-RC0/
>
> The RC tag in git is release-3.1.0-RC0.
>
> The maven artifacts are available via repository.apache.org at
> https://repository.apache.org/content/repositories/
> orgapachehadoop-1086/
>
> This vote will run 7 days (5 weekdays), ending on Mar 28 at 11:59 pm
> Pacific.
>
> 3.1.0 contains 727 [2] fixed JIRA issues since 3.0.0. Notable
>
> additions
>
> include the first class GPU/FPGA support on YARN, Native services,
> Support
> rich placement constraints in YARN, S3-related enhancements, allow
>
> HDFS
>
> block replicas to be provided by an external storage system, etc.
>
> We’d like to use this as a starting release for 3.1.x [1], depending
> on how
> it goes, get it stabilized and potentially use a 3.1.1 in several
> weeks as
> the stable release.
>
> We have done testing with a pseudo cluster and distributed shell job.
> My +1
> to start.
>
> Best,
> Wangda/Vinod
>
> [1]
> https://lists.apache.org/thread.html/b3fb3b6da8b6357a68513a6dfd104b
> c9e19e559aedc5ebedb4ca08c8@%3Cyarn-dev.hadoop.apache.org%3E
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in
> (3.1.0)
> AND fixVersion not in (3.0.0, 3.0.0-beta1) AND status = Resolved
>
> ORDER
>
> BY
> fixVersion ASC
>
>
>
>
>
>
> B�KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKCB�
> �[��X��ܚX�KK[XZ[���Y]�][��X��ܚX�PY ���\X�K�ܙ�B��܈Y][ۘ[��[X[��K[XZ[���Y]�Z[Y
> ���\X�K�ܙ�B�
>
>