[VOTE] Release Apache Hadoop 2.8.0 (RC2)

classic Classic list List threaded Threaded
58 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Zhihai Xu-3
Thanks Junping for creating release 2.8.0 (RC3).

+1 (non-binding)

* Downloaded and built from source
* Verified md5 checksums and signature
* Deployed a pseudo cluster
* verified basic HDFS operations and Pi job.
* Did a sanity check for RM and NM UI.

Thanks

zhihai

On Tue, Mar 21, 2017 at 12:02 PM, Steve Loughran <[hidden email]>
wrote:

>
> I'm an =0 I'm afraid
>
> I think those extra files need to be pulled which triggers a repackage, at
> which point there's a couple of
>
> HADOOP-14205 No FileSystem for scheme: adl
> HADOOP-14204 S3A multipart commit failing,
>
> of the two, the ADL FS one is the most significant, though if there was
> going to be a re-roll, I'd like them in.
>
> I also worry about windows test coverage, as irrespective of whether
> people are planning to run it on a windows host in production, the hadoop
> libs get used client side in job submit, and more now in Spark .
> HADOOP-14201 showed up that (a) my new VM is a mess, so complicating
> testing and (b) gzip was failing. I Worry about gzip, as that could be a
> sign of native lib problems.. someone needs to look at it a bit more.
> However, I don't think it should really be a blocker, given the ASF doesn't
> officially redistribute windows binaries.
>
> so: some issues with packaging and a couple of patches which need to go
> in. Test coverage on windows is potentially longer, and I don't want it to
> hold things up.
>
> Anyhow: =0. If everyone else is happy to ship, then let's get it out the
> door, as that's the only way that we'll get all the in-the-field bugreps
> coming back in. We will need to release some new versions in the coming
> weeks to handle those
>
> -Steve
>
>
>
>
> > On 21 Mar 2017, at 18:43, Haibo Chen <[hidden email]> wrote:
> >
> > Thanks Junping for working on the new release!
> >
> > +1 non-binding
> >
> > 1) Downloaded the source, verified the checksum
> > 2) Built natively from source, and deployed it to a pseudo-distributed
> > cluster
> > 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> > 4) Played with yarn + mapreduce command lines
> >
> > Best,
> > Haibo Chen
> >
> > On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <[hidden email]>
> wrote:
> >
> >> ?Thanks for update, John. Then we should be OK with fixing this issue in
> >> 2.8.1.
> >>
> >> Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> >> up to blocker in case we could miss this in releasing 2.8.1. :)
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <[hidden email]>
> >> Sent: Monday, March 20, 2017 10:31 AM
> >> To: Junping Du
> >> Cc: [hidden email]; [hidden email];
> >> [hidden email]; [hidden email]
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Yes, it only affects ADL. There is a workaround of adding these 2
> >> properties to core-site.xml:
> >>
> >>  <property>
> >>    <name>fs.adl.impl</name>
> >>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >>  </property>
> >>
> >>  <property>
> >>    <name>fs.AbstractFileSystem.adl.impl</name>
> >>    <value>org.apache.hadoop.fs.adl.Adl</value>
> >>  </property>
> >>
> >> I have the initial patch ready but hitting these live unit test
> failures:
> >>
> >> Failed tests:
> >>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> >> expected:<1> but was:<10>
> >>
> >> Tests in error:
> >>  TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >>  TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >> testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >>
> >>
> >> Stay tuned...
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <[hidden email]
> <mailto:
> >> [hidden email]>> wrote:
> >>
> >> Thank you for reporting the issue, John! Does this issue only affect ADL
> >> (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> >> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> >> regression and just a new feature get broken.?
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Junping
> >>
> >> ________________________________
> >> From: John Zhuge <[hidden email]<mailto:[hidden email]>>
> >> Sent: Monday, March 20, 2017 9:07 AM
> >> To: Junping Du
> >> Cc: [hidden email]<mailto:[hidden email]>;
> >> [hidden email]<mailto:[hidden email]>;
> >> [hidden email]<mailto:[hidden email]>;
> >> [hidden email]<mailto:[hidden email]>
> >> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>
> >> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >> FileSystem for scheme: adl".
> >>
> >> The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> >> HADOOP-12666 should not be backported, but some changes are needed:
> >> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >>
> >> I am working on a patch.
> >>
> >>
> >> John Zhuge
> >> Software Engineer, Cloudera
> >>
> >> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <[hidden email]
> <mailto:jd
> >> [hidden email]>> wrote:
> >> Hi all,
> >>     With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>     This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~
> >> junping_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org
> <http://
> >> repository.apache.org> at: https://repository.apache.org/
> >> content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

John Zhuge-2
In reply to this post by Junping Du-2
+1. Thanks for the great effort, Junping!


   - Verified checksums and signatures of the tarballs
   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
   - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
   - Deployed a pseudo cluster, passed the following sanity tests in
   both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
   - HDFS-11132. Allow AccessControlException in contract tests when
   getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
   Dusane
   - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
   runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <[hidden email]> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>   <property>
>     <name>fs.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>   </property>
>
>   <property>
>     <name>fs.AbstractFileSystem.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.Adl</value>
>   </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <[hidden email]> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.​
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <[hidden email]>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* [hidden email]; [hidden email];
> > [hidden email]; [hidden email]
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <[hidden email]> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>       More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>       New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>       The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>       Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.
> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



--
John
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Allen Wittenauer-6
In reply to this post by Andrew Wang

> On Mar 21, 2017, at 10:12 AM, Andrew Wang <[hidden email]> wrote:
>
> I poked around a bit. The 3.0.0-alpha2 binary tarball is only 246M and has
> more changes than 2.8.0.


Not to disclaim any other potential issues, but it's worth noting 3.x de-dupes jar files as part of the packaging process.  So it's not exactly an apples-to-apples comparison. (Although I think the new yarn-ui made the significant loss in excess jars moot.  Without that, I'd expect 3.x to be about half the size.)
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Eric Badger
In reply to this post by John Zhuge-2
+1 (non-binding)

- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs

Thanks,

Eric

On Tuesday, March 21, 2017 2:56 PM, John Zhuge <[hidden email]> wrote:



+1. Thanks for the great effort, Junping!


   - Verified checksums and signatures of the tarballs
   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
   - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
   - Deployed a pseudo cluster, passed the following sanity tests in
   both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
   - HDFS-11132. Allow AccessControlException in contract tests when
   getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
   Dusane
   - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
   runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <[hidden email]> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>   <property>
>     <name>fs.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>   </property>
>
>   <property>
>     <name>fs.AbstractFileSystem.adl.impl</name>
>     <value>org.apache.hadoop.fs.adl.Adl</value>
>   </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <[hidden email]> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.​
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <[hidden email]>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* [hidden email]; [hidden email];
> > [hidden email]; [hidden email]
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <[hidden email]> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>       More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>       New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>       The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>       Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.

> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



--
John

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Kuhu Shukla

+1 (non-binding)

- Verified signatures.
- Downloaded and built from source tar.gz.
- Deployed a pseudo-distributed cluster on Mac Sierra.
- Ran example Sleep job successfully.
- Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.

Thank you Junping and everyone else who worked on getting this release out.

Warm Regards,
Kuhu
On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <[hidden email]> wrote:
+1 (non-binding)

- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs

Thanks,

Eric

On Tuesday, March 21, 2017 2:56 PM, John Zhuge <[hidden email]> wrote:



+1. Thanks for the great effort, Junping!


  - Verified checksums and signatures of the tarballs
  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
  - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
  - Deployed a pseudo cluster, passed the following sanity tests in
  both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
  - HDFS-11132. Allow AccessControlException in contract tests when
  getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
  Dusane
  - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
  runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <[hidden email]> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>  <property>
>    <name>fs.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>  </property>
>
>  <property>
>    <name>fs.AbstractFileSystem.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.Adl</value>
>  </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <[hidden email]> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.​
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <[hidden email]>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* [hidden email]; [hidden email];
> > [hidden email]; [hidden email]
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <[hidden email]> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.

> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



--
John


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

zhenke luo
Hello:

I found an issue with HADOOP-10027 that :

SnappyCompressor can’t load native lib properly.

Test TestHadoopnative is failing  in hadoop-apache2 version 0.10

stacktrace:
Caused by: java.lang.RuntimeException: native snappy library not available: SnappyCompressor has not been loaded.
        at org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:72)
        at org.apache.hadoop.io.compress.SnappyCodec.getDecompressorType(SnappyCodec.java:195)
        at com.facebook.presto.hadoop.HadoopNative.loadAllCodecs(HadoopNative.java:71)
        at com.facebook.presto.hadoop.HadoopNative.requireHadoopNative(HadoopNative.java:52)
        ... 24 more

After look into the code found that in SnappyCompressor.java
static {
    if (NativeCodeLoader.isNativeCodeLoaded() &&
        NativeCodeLoader.buildSupportsSnappy()) {
      try {
        initIDs();
        nativeSnappyLoaded = true;
      } catch (Throwable t) {
        LOG.error("failed to load SnappyCompressor", t);
      }
    }
  }
the native method initIDs will complain: no such field clazz.


> On Mar 21, 2017, at 4:32 PM, Poepping, Thomas <[hidden email]> wrote:
>
>
>
> On 3/21/17, 3:17 PM, "Kuhu Shukla" <[hidden email]> wrote:
>
>
>    +1 (non-binding)
>
>    - Verified signatures.
>    - Downloaded and built from source tar.gz.
>    - Deployed a pseudo-distributed cluster on Mac Sierra.
>    - Ran example Sleep job successfully.
>    - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.
>
>    Thank you Junping and everyone else who worked on getting this release out.
>
>    Warm Regards,
>    Kuhu
>    On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <[hidden email]> wrote:
>    +1 (non-binding)
>
>    - Verified checksums and signatures of all files
>    - Built from source on MacOS Sierra via JDK 1.8.0 u65
>    - Deployed single-node cluster
>    - Successfully ran a few sample jobs
>
>    Thanks,
>
>    Eric
>
>    On Tuesday, March 21, 2017 2:56 PM, John Zhuge <[hidden email]> wrote:
>
>
>
>    +1. Thanks for the great effort, Junping!
>
>
>      - Verified checksums and signatures of the tarballs
>      - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
>      - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
>      - Cloud connectors:
>          - s3a: integration tests, basic fs commands
>          - adl: live unit tests, basic fs commands. See notes below.
>      - Deployed a pseudo cluster, passed the following sanity tests in
>      both insecure and SSL mode:
>          - HDFS: basic dfs, distcp, ACL commands
>          - KMS and HttpFS: basic tests
>          - MapReduce wordcount
>          - balancer start/stop
>
>
>    Needs the following JIRAs to pass all ADL tests:
>
>      - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
>      - HDFS-11132. Allow AccessControlException in contract tests when
>      getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
>      Dusane
>      - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
>      runtime error. (John Zhuge via lei)
>
>
>    On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <[hidden email]> wrote:
>
>> Yes, it only affects ADL. There is a workaround of adding these 2
>> properties to core-site.xml:
>>
>> <property>
>>   <name>fs.adl.impl</name>
>>   <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>> </property>
>>
>> <property>
>>   <name>fs.AbstractFileSystem.adl.impl</name>
>>   <value>org.apache.hadoop.fs.adl.Adl</value>
>> </property>
>>
>> I have the initial patch ready but hitting these live unit test failures:
>>
>> Failed tests:
>>
>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>> testListStatus:257
>> expected:<1> but was:<10>
>>
>> Tests in error:
>>
>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:254
>> » AccessControl
>>
>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>> testMkdirsFailsForSubdirectoryOfExistingFile:190
>> » AccessControl
>>
>>
>> Stay tuned...
>>
>> John Zhuge
>> Software Engineer, Cloudera
>>
>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <[hidden email]> wrote:
>>
>>> Thank you for reporting the issue, John! Does this issue only affect ADL
>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
>> existing
>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
>> not a
>>> regression and just a new feature get broken.​
>>>
>>>
>>> Thanks,
>>>
>>>
>>> Junping
>>> ------------------------------
>>> *From:* John Zhuge <[hidden email]>
>>> *Sent:* Monday, March 20, 2017 9:07 AM
>>> *To:* Junping Du
>>> *Cc:* [hidden email]; [hidden email];
>>> [hidden email]; [hidden email]
>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>>
>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>>> FileSystem for scheme: adl".
>>>
>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
>> earlier.
>>> HADOOP-12666 should not be backported, but some changes are needed:
>>> property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
>>>
>>> I am working on a patch.
>>>
>>>
>>> John Zhuge
>>> Software Engineer, Cloudera
>>>
>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <[hidden email]> wrote:
>>>
>>>> Hi all,
>>>>     With fix of HDFS-11431 get in, I've created a new release candidate
>>>> (RC3) for Apache Hadoop 2.8.0.
>>>>
>>>>     This is the next minor release to follow up 2.7.0 which has been
>>>> released for more than 1 year. It comprises 2,900+ fixes, improvements,
>> and
>>>> new features. Most of these commits are released for the first time in
>>>> branch-2.
>>>>
>>>>     More information about the 2.8.0 release plan can be found here:
>>>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>>>
>>>>     New RC is available at: http://home.apache.org/~junpin
>>>> g_du/hadoop-2.8.0-RC3
>>>>
>>>>     The RC tag in git is: release-2.8.0-RC3, and the latest commit id
>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>>>
>>>>     The maven artifacts are available via repository.apache.org at:
>>>> https://repository.apache.org/content/repositories/orgapachehadoop-1057
>>>>
>>>>     Please try the release and vote; the vote will run for the usual 5
>>>> days, ending on 03/22/2017 PDT time.
>
>>>>
>>>> Thanks,
>>>>
>>>> Junping
>>>>
>>>
>>>
>>
>
>
>
>    --
>    John
>
>
>    ---------------------------------------------------------------------
>    To unsubscribe, e-mail: [hidden email]
>    For additional commands, e-mail: [hidden email]
>
>

Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Junping Du-2
In reply to this post by Kuhu Shukla
Thanks all for response with verification work and vote!


Sounds like we are hitting several issues here, although none seems to be blockers so far. Given the large commit set - 2000+ commits first landed in branch-2 release, we may should follow 2.7.0 practice that to claim this release is not for production cluster, just like Vinod's suggestion in previous email. We should quickly come up with 2.8.1 release in next 1 or 2 month for production deployment.


We will close the vote in next 24 hours. For people who haven't vote, please keep on verification work and report any issues if founded - I will check if another round of RC is needed based on your findings. Thanks!


Thanks,


Junping


________________________________
From: Kuhu Shukla <[hidden email]>
Sent: Tuesday, March 21, 2017 3:17 PM
Cc: Junping Du; [hidden email]; [hidden email]; [hidden email]; [hidden email]
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)


+1 (non-binding)

- Verified signatures.
- Downloaded and built from source tar.gz.
- Deployed a pseudo-distributed cluster on Mac Sierra.
- Ran example Sleep job successfully.
- Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount successfully.

Thank you Junping and everyone else who worked on getting this release out.

Warm Regards,
Kuhu
On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger <[hidden email]> wrote:
+1 (non-binding)

- Verified checksums and signatures of all files
- Built from source on MacOS Sierra via JDK 1.8.0 u65
- Deployed single-node cluster
- Successfully ran a few sample jobs

Thanks,

Eric

On Tuesday, March 21, 2017 2:56 PM, John Zhuge <[hidden email]> wrote:



+1. Thanks for the great effort, Junping!


  - Verified checksums and signatures of the tarballs
  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
  - Cloud connectors:
      - s3a: integration tests, basic fs commands
      - adl: live unit tests, basic fs commands. See notes below.
  - Deployed a pseudo cluster, passed the following sanity tests in
  both insecure and SSL mode:
      - HDFS: basic dfs, distcp, ACL commands
      - KMS and HttpFS: basic tests
      - MapReduce wordcount
      - balancer start/stop


Needs the following JIRAs to pass all ADL tests:

  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
  - HDFS-11132. Allow AccessControlException in contract tests when
  getFileStatus on subdirectory of existing files. Contributed by Vishwajeet
  Dusane
  - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
  runtime error. (John Zhuge via lei)


On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <[hidden email]> wrote:

> Yes, it only affects ADL. There is a workaround of adding these 2
> properties to core-site.xml:
>
>  <property>
>    <name>fs.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>  </property>
>
>  <property>
>    <name>fs.AbstractFileSystem.adl.impl</name>
>    <value>org.apache.hadoop.fs.adl.Adl</value>
>  </property>
>
> I have the initial patch ready but hitting these live unit test failures:
>
> Failed tests:
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testListStatus:257
> expected:<1> but was:<10>
>
> Tests in error:
>
> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:254
> » AccessControl
>
> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> testMkdirsFailsForSubdirectoryOfExistingFile:190
> » AccessControl
>
>
> Stay tuned...
>
> John Zhuge
> Software Engineer, Cloudera
>
> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <[hidden email]> wrote:
>
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> > ------------------------------
> > *From:* John Zhuge <[hidden email]>
> > *Sent:* Monday, March 20, 2017 9:07 AM
> > *To:* Junping Du
> > *Cc:* [hidden email]; [hidden email];
> > [hidden email]; [hidden email]
> > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <[hidden email]> wrote:
> >
> >> Hi all,
> >>      With fix of HDFS-11431 get in, I've created a new release candidate
> >> (RC3) for Apache Hadoop 2.8.0.
> >>
> >>      This is the next minor release to follow up 2.7.0 which has been
> >> released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> >> new features. Most of these commits are released for the first time in
> >> branch-2.
> >>
> >>      More information about the 2.8.0 release plan can be found here:
> >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >>
> >>      New RC is available at: http://home.apache.org/~junpin
> >> g_du/hadoop-2.8.0-RC3
> >>
> >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>
> >>      The maven artifacts are available via repository.apache.org at:
> >> https://repository.apache.org/content/repositories/orgapachehadoop-1057
> >>
> >>      Please try the release and vote; the vote will run for the usual 5
> >> days, ending on 03/22/2017 PDT time.

> >>
> >> Thanks,
> >>
> >> Junping
> >>
> >
> >
>



--
John


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Ravi Prakash-3
Thanks for all the effort Junping!

+1 (binding)
+ Verified signature and MD5, SHA1, SHA256 checksum of tarball
+ Verified SHA ID in git corresponds to RC3 tag
+ Verified wordcount for one small text file produces same output as
hadoop-2.7.3.
+ HDFS Namenode UI looks good.

I agree none of the issues reported so far are blockers. Looking forward to
another great release.

Thanks
Ravi

On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <[hidden email]> wrote:

> Thanks all for response with verification work and vote!
>
>
> Sounds like we are hitting several issues here, although none seems to be
> blockers so far. Given the large commit set - 2000+ commits first landed in
> branch-2 release, we may should follow 2.7.0 practice that to claim this
> release is not for production cluster, just like Vinod's suggestion in
> previous email. We should quickly come up with 2.8.1 release in next 1 or 2
> month for production deployment.
>
>
> We will close the vote in next 24 hours. For people who haven't vote,
> please keep on verification work and report any issues if founded - I will
> check if another round of RC is needed based on your findings. Thanks!
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Kuhu Shukla <[hidden email]>
> Sent: Tuesday, March 21, 2017 3:17 PM
> Cc: Junping Du; [hidden email]; [hidden email];
> [hidden email]; [hidden email]
> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>
>
> +1 (non-binding)
>
> - Verified signatures.
> - Downloaded and built from source tar.gz.
> - Deployed a pseudo-distributed cluster on Mac Sierra.
> - Ran example Sleep job successfully.
> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> successfully.
>
> Thank you Junping and everyone else who worked on getting this release out.
>
> Warm Regards,
> Kuhu
> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> <[hidden email]> wrote:
> +1 (non-binding)
>
> - Verified checksums and signatures of all files
> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> - Deployed single-node cluster
> - Successfully ran a few sample jobs
>
> Thanks,
>
> Eric
>
> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <[hidden email]>
> wrote:
>
>
>
> +1. Thanks for the great effort, Junping!
>
>
>   - Verified checksums and signatures of the tarballs
>   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
>   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
>   - Cloud connectors:
>       - s3a: integration tests, basic fs commands
>       - adl: live unit tests, basic fs commands. See notes below.
>   - Deployed a pseudo cluster, passed the following sanity tests in
>   both insecure and SSL mode:
>       - HDFS: basic dfs, distcp, ACL commands
>       - KMS and HttpFS: basic tests
>       - MapReduce wordcount
>       - balancer start/stop
>
>
> Needs the following JIRAs to pass all ADL tests:
>
>   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge.
>   - HDFS-11132. Allow AccessControlException in contract tests when
>   getFileStatus on subdirectory of existing files. Contributed by
> Vishwajeet
>   Dusane
>   - HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1
>   runtime error. (John Zhuge via lei)
>
>
> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <[hidden email]> wrote:
>
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> >  <property>
> >    <name>fs.adl.impl</name>
> >    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >  </property>
> >
> >  <property>
> >    <name>fs.AbstractFileSystem.adl.impl</name>
> >    <value>org.apache.hadoop.fs.adl.Adl</value>
> >  </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >
> > TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254
> > » AccessControl
> >
> > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190
> > » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <[hidden email]>
> wrote:
> >
> > > Thank you for reporting the issue, John! Does this issue only affect
> ADL
> > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > existing
> > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > not a
> > > regression and just a new feature get broken.?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > > ------------------------------
> > > *From:* John Zhuge <[hidden email]>
> > > *Sent:* Monday, March 20, 2017 9:07 AM
> > > *To:* Junping Du
> > > *Cc:* [hidden email]; [hidden email];
> > > [hidden email]; [hidden email]
> > > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > FileSystem for scheme: adl".
> > >
> > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > earlier.
> > > HADOOP-12666 should not be backported, but some changes are needed:
> > > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> > >
> > > I am working on a patch.
> > >
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <[hidden email]>
> wrote:
> > >
> > >> Hi all,
> > >>      With fix of HDFS-11431 get in, I've created a new release
> candidate
> > >> (RC3) for Apache Hadoop 2.8.0.
> > >>
> > >>      This is the next minor release to follow up 2.7.0 which has been
> > >> released for more than 1 year. It comprises 2,900+ fixes,
> improvements,
> > and
> > >> new features. Most of these commits are released for the first time in
> > >> branch-2.
> > >>
> > >>      More information about the 2.8.0 release plan can be found here:
> > >> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> > >>
> > >>      New RC is available at: http://home.apache.org/~junpin
> > >> g_du/hadoop-2.8.0-RC3
> > >>
> > >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>
> > >>      The maven artifacts are available via repository.apache.org at:
> > >> https://repository.apache.org/content/repositories/
> orgapachehadoop-1057
> > >>
> > >>      Please try the release and vote; the vote will run for the usual
> 5
> > >> days, ending on 03/22/2017 PDT time.
>
> > >>
> > >> Thanks,
> > >>
> > >> Junping
> > >>
> > >
> > >
> >
>
>
>
> --
> John
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

larry mccay-3
+1 (non-binding)

- verified signatures
- built from source and ran tests
- deployed pseudo cluster
- ran basic tests for hdfs, wordcount, credential provider API and related
commands
- tested webhdfs with knox


On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <[hidden email]> wrote:

> Thanks for all the effort Junping!
>
> +1 (binding)
> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> + Verified SHA ID in git corresponds to RC3 tag
> + Verified wordcount for one small text file produces same output as
> hadoop-2.7.3.
> + HDFS Namenode UI looks good.
>
> I agree none of the issues reported so far are blockers. Looking forward to
> another great release.
>
> Thanks
> Ravi
>
> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <[hidden email]> wrote:
>
> > Thanks all for response with verification work and vote!
> >
> >
> > Sounds like we are hitting several issues here, although none seems to be
> > blockers so far. Given the large commit set - 2000+ commits first landed
> in
> > branch-2 release, we may should follow 2.7.0 practice that to claim this
> > release is not for production cluster, just like Vinod's suggestion in
> > previous email. We should quickly come up with 2.8.1 release in next 1
> or 2
> > month for production deployment.
> >
> >
> > We will close the vote in next 24 hours. For people who haven't vote,
> > please keep on verification work and report any issues if founded - I
> will
> > check if another round of RC is needed based on your findings. Thanks!
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: Kuhu Shukla <[hidden email]>
> > Sent: Tuesday, March 21, 2017 3:17 PM
> > Cc: Junping Du; [hidden email]; [hidden email]
> ;
> > [hidden email]; [hidden email]
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> >
> > +1 (non-binding)
> >
> > - Verified signatures.
> > - Downloaded and built from source tar.gz.
> > - Deployed a pseudo-distributed cluster on Mac Sierra.
> > - Ran example Sleep job successfully.
> > - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> > successfully.
> >
> > Thank you Junping and everyone else who worked on getting this release
> out.
> >
> > Warm Regards,
> > Kuhu
> > On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> > <[hidden email]> wrote:
> > +1 (non-binding)
> >
> > - Verified checksums and signatures of all files
> > - Built from source on MacOS Sierra via JDK 1.8.0 u65
> > - Deployed single-node cluster
> > - Successfully ran a few sample jobs
> >
> > Thanks,
> >
> > Eric
> >
> > On Tuesday, March 21, 2017 2:56 PM, John Zhuge <[hidden email]>
> > wrote:
> >
> >
> >
> > +1. Thanks for the great effort, Junping!
> >
> >
> >   - Verified checksums and signatures of the tarballs
> >   - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> >   - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> >   - Cloud connectors:
> >       - s3a: integration tests, basic fs commands
> >       - adl: live unit tests, basic fs commands. See notes below.
> >   - Deployed a pseudo cluster, passed the following sanity tests in
> >   both insecure and SSL mode:
> >       - HDFS: basic dfs, distcp, ACL commands
> >       - KMS and HttpFS: basic tests
> >       - MapReduce wordcount
> >       - balancer start/stop
> >
> >
> > Needs the following JIRAs to pass all ADL tests:
> >
> >   - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> Zhuge.
> >   - HDFS-11132. Allow AccessControlException in contract tests when
> >   getFileStatus on subdirectory of existing files. Contributed by
> > Vishwajeet
> >   Dusane
> >   - HADOOP-13928. TestAdlFileContextMainOperatio
> nsLive.testGetFileContext1
> >   runtime error. (John Zhuge via lei)
> >
> >
> > On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <[hidden email]>
> wrote:
> >
> > > Yes, it only affects ADL. There is a workaround of adding these 2
> > > properties to core-site.xml:
> > >
> > >  <property>
> > >    <name>fs.adl.impl</name>
> > >    <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > >  </property>
> > >
> > >  <property>
> > >    <name>fs.AbstractFileSystem.adl.impl</name>
> > >    <value>org.apache.hadoop.fs.adl.Adl</value>
> > >  </property>
> > >
> > > I have the initial patch ready but hitting these live unit test
> failures:
> > >
> > > Failed tests:
> > >
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > > testListStatus:257
> > > expected:<1> but was:<10>
> > >
> > > Tests in error:
> > >
> > > TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> est.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:254
> > > » AccessControl
> > >
> > > TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:190
> > > » AccessControl
> > >
> > >
> > > Stay tuned...
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <[hidden email]>
> > wrote:
> > >
> > > > Thank you for reporting the issue, John! Does this issue only affect
> > ADL
> > > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > > existing
> > > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > > not a
> > > > regression and just a new feature get broken.?
> > > >
> > > >
> > > > Thanks,
> > > >
> > > >
> > > > Junping
> > > > ------------------------------
> > > > *From:* John Zhuge <[hidden email]>
> > > > *Sent:* Monday, March 20, 2017 9:07 AM
> > > > *To:* Junping Du
> > > > *Cc:* [hidden email]; [hidden email];
> > > > [hidden email]; [hidden email]
> > > > *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > > >
> > > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > > FileSystem for scheme: adl".
> > > >
> > > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > > earlier.
> > > > HADOOP-12666 should not be backported, but some changes are needed:
> > > > property fs.adl.impl in core-default.xml and
> hadoop-tools-dist/pom.xml.
> > > >
> > > > I am working on a patch.
> > > >
> > > >
> > > > John Zhuge
> > > > Software Engineer, Cloudera
> > > >
> > > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <[hidden email]>
> > wrote:
> > > >
> > > >> Hi all,
> > > >>      With fix of HDFS-11431 get in, I've created a new release
> > candidate
> > > >> (RC3) for Apache Hadoop 2.8.0.
> > > >>
> > > >>      This is the next minor release to follow up 2.7.0 which has
> been
> > > >> released for more than 1 year. It comprises 2,900+ fixes,
> > improvements,
> > > and
> > > >> new features. Most of these commits are released for the first time
> in
> > > >> branch-2.
> > > >>
> > > >>      More information about the 2.8.0 release plan can be found
> here:
> > > >> https://cwiki.apache.org/confluence/display/HADOOP/
> Hadoop+2.8+Release
> > > >>
> > > >>      New RC is available at: http://home.apache.org/~junpin
> > > >> g_du/hadoop-2.8.0-RC3
> > > >>
> > > >>      The RC tag in git is: release-2.8.0-RC3, and the latest commit
> id
> > > >> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > > >>
> > > >>      The maven artifacts are available via repository.apache.org
> at:
> > > >> https://repository.apache.org/content/repositories/
> > orgapachehadoop-1057
> > > >>
> > > >>      Please try the release and vote; the vote will run for the
> usual
> > 5
> > > >> days, ending on 03/22/2017 PDT time.
> >
> > > >>
> > > >> Thanks,
> > > >>
> > > >> Junping
> > > >>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > John
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Jian He
+1 (binding)

- built from source
- deployed a pseudo cluster
- ran basic example tests.
- Navigate the UI a bit, looks good.

Jian

> On Mar 22, 2017, at 9:03 PM, larry mccay <[hidden email]> wrote:
>
> +1 (non-binding)
>
> - verified signatures
> - built from source and ran tests
> - deployed pseudo cluster
> - ran basic tests for hdfs, wordcount, credential provider API and related
> commands
> - tested webhdfs with knox
>
>
> On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <[hidden email]> wrote:
>
>> Thanks for all the effort Junping!
>>
>> +1 (binding)
>> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
>> + Verified SHA ID in git corresponds to RC3 tag
>> + Verified wordcount for one small text file produces same output as
>> hadoop-2.7.3.
>> + HDFS Namenode UI looks good.
>>
>> I agree none of the issues reported so far are blockers. Looking forward to
>> another great release.
>>
>> Thanks
>> Ravi
>>
>> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <[hidden email]> wrote:
>>
>>> Thanks all for response with verification work and vote!
>>>
>>>
>>> Sounds like we are hitting several issues here, although none seems to be
>>> blockers so far. Given the large commit set - 2000+ commits first landed
>> in
>>> branch-2 release, we may should follow 2.7.0 practice that to claim this
>>> release is not for production cluster, just like Vinod's suggestion in
>>> previous email. We should quickly come up with 2.8.1 release in next 1
>> or 2
>>> month for production deployment.
>>>
>>>
>>> We will close the vote in next 24 hours. For people who haven't vote,
>>> please keep on verification work and report any issues if founded - I
>> will
>>> check if another round of RC is needed based on your findings. Thanks!
>>>
>>>
>>> Thanks,
>>>
>>>
>>> Junping
>>>
>>>
>>> ________________________________
>>> From: Kuhu Shukla <[hidden email]>
>>> Sent: Tuesday, March 21, 2017 3:17 PM
>>> Cc: Junping Du; [hidden email]; [hidden email]
>> ;
>>> [hidden email]; [hidden email]
>>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>>
>>>
>>> +1 (non-binding)
>>>
>>> - Verified signatures.
>>> - Downloaded and built from source tar.gz.
>>> - Deployed a pseudo-distributed cluster on Mac Sierra.
>>> - Ran example Sleep job successfully.
>>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
>>> successfully.
>>>
>>> Thank you Junping and everyone else who worked on getting this release
>> out.
>>>
>>> Warm Regards,
>>> Kuhu
>>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
>>> <[hidden email]> wrote:
>>> +1 (non-binding)
>>>
>>> - Verified checksums and signatures of all files
>>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
>>> - Deployed single-node cluster
>>> - Successfully ran a few sample jobs
>>>
>>> Thanks,
>>>
>>> Eric
>>>
>>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <[hidden email]>
>>> wrote:
>>>
>>>
>>>
>>> +1. Thanks for the great effort, Junping!
>>>
>>>
>>>  - Verified checksums and signatures of the tarballs
>>>  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
>>>  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
>>>  - Cloud connectors:
>>>      - s3a: integration tests, basic fs commands
>>>      - adl: live unit tests, basic fs commands. See notes below.
>>>  - Deployed a pseudo cluster, passed the following sanity tests in
>>>  both insecure and SSL mode:
>>>      - HDFS: basic dfs, distcp, ACL commands
>>>      - KMS and HttpFS: basic tests
>>>      - MapReduce wordcount
>>>      - balancer start/stop
>>>
>>>
>>> Needs the following JIRAs to pass all ADL tests:
>>>
>>>  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
>> Zhuge.
>>>  - HDFS-11132. Allow AccessControlException in contract tests when
>>>  getFileStatus on subdirectory of existing files. Contributed by
>>> Vishwajeet
>>>  Dusane
>>>  - HADOOP-13928. TestAdlFileContextMainOperatio
>> nsLive.testGetFileContext1
>>>  runtime error. (John Zhuge via lei)
>>>
>>>
>>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <[hidden email]>
>> wrote:
>>>
>>>> Yes, it only affects ADL. There is a workaround of adding these 2
>>>> properties to core-site.xml:
>>>>
>>>> <property>
>>>>   <name>fs.adl.impl</name>
>>>>   <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
>>>> </property>
>>>>
>>>> <property>
>>>>   <name>fs.AbstractFileSystem.adl.impl</name>
>>>>   <value>org.apache.hadoop.fs.adl.Adl</value>
>>>> </property>
>>>>
>>>> I have the initial patch ready but hitting these live unit test
>> failures:
>>>>
>>>> Failed tests:
>>>>
>>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>>>> testListStatus:257
>>>> expected:<1> but was:<10>
>>>>
>>>> Tests in error:
>>>>
>>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
>> est.
>>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
>>>> » AccessControl
>>>>
>>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
>>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
>>>> » AccessControl
>>>>
>>>>
>>>> Stay tuned...
>>>>
>>>> John Zhuge
>>>> Software Engineer, Cloudera
>>>>
>>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <[hidden email]>
>>> wrote:
>>>>
>>>>> Thank you for reporting the issue, John! Does this issue only affect
>>> ADL
>>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
>>>> existing
>>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
>>>> not a
>>>>> regression and just a new feature get broken.?
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>>
>>>>> Junping
>>>>> ------------------------------
>>>>> *From:* John Zhuge <[hidden email]>
>>>>> *Sent:* Monday, March 20, 2017 9:07 AM
>>>>> *To:* Junping Du
>>>>> *Cc:* [hidden email]; [hidden email];
>>>>> [hidden email]; [hidden email]
>>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
>>>>>
>>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
>>>>> FileSystem for scheme: adl".
>>>>>
>>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
>>>> earlier.
>>>>> HADOOP-12666 should not be backported, but some changes are needed:
>>>>> property fs.adl.impl in core-default.xml and
>> hadoop-tools-dist/pom.xml.
>>>>>
>>>>> I am working on a patch.
>>>>>
>>>>>
>>>>> John Zhuge
>>>>> Software Engineer, Cloudera
>>>>>
>>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <[hidden email]>
>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>     With fix of HDFS-11431 get in, I've created a new release
>>> candidate
>>>>>> (RC3) for Apache Hadoop 2.8.0.
>>>>>>
>>>>>>     This is the next minor release to follow up 2.7.0 which has
>> been
>>>>>> released for more than 1 year. It comprises 2,900+ fixes,
>>> improvements,
>>>> and
>>>>>> new features. Most of these commits are released for the first time
>> in
>>>>>> branch-2.
>>>>>>
>>>>>>     More information about the 2.8.0 release plan can be found
>> here:
>>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
>> Hadoop+2.8+Release
>>>>>>
>>>>>>     New RC is available at: http://home.apache.org/~junpin
>>>>>> g_du/hadoop-2.8.0-RC3
>>>>>>
>>>>>>     The RC tag in git is: release-2.8.0-RC3, and the latest commit
>> id
>>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
>>>>>>
>>>>>>     The maven artifacts are available via repository.apache.org
>> at:
>>>>>> https://repository.apache.org/content/repositories/
>>> orgapachehadoop-1057
>>>>>>
>>>>>>     Please try the release and vote; the vote will run for the
>> usual
>>> 5
>>>>>> days, ending on 03/22/2017 PDT time.
>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Junping
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> John
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Rakesh Radhakrishnan-2
Thanks Junping for getting this out.

+1 (non-binding)

* downloaded and built from source with jdk1.8.0_45
* deployed HDFS-HA cluster
* ran some sample jobs
* run balancer
* executed basic dfs cmds


Rakesh

On Wed, Mar 22, 2017 at 8:30 PM, Jian He <[hidden email]> wrote:

> +1 (binding)
>
> - built from source
> - deployed a pseudo cluster
> - ran basic example tests.
> - Navigate the UI a bit, looks good.
>
> Jian
>
> > On Mar 22, 2017, at 9:03 PM, larry mccay <[hidden email]> wrote:
> >
> > +1 (non-binding)
> >
> > - verified signatures
> > - built from source and ran tests
> > - deployed pseudo cluster
> > - ran basic tests for hdfs, wordcount, credential provider API and
> related
> > commands
> > - tested webhdfs with knox
> >
> >
> > On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <[hidden email]>
> wrote:
> >
> >> Thanks for all the effort Junping!
> >>
> >> +1 (binding)
> >> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> >> + Verified SHA ID in git corresponds to RC3 tag
> >> + Verified wordcount for one small text file produces same output as
> >> hadoop-2.7.3.
> >> + HDFS Namenode UI looks good.
> >>
> >> I agree none of the issues reported so far are blockers. Looking
> forward to
> >> another great release.
> >>
> >> Thanks
> >> Ravi
> >>
> >> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <[hidden email]>
> wrote:
> >>
> >>> Thanks all for response with verification work and vote!
> >>>
> >>>
> >>> Sounds like we are hitting several issues here, although none seems to
> be
> >>> blockers so far. Given the large commit set - 2000+ commits first
> landed
> >> in
> >>> branch-2 release, we may should follow 2.7.0 practice that to claim
> this
> >>> release is not for production cluster, just like Vinod's suggestion in
> >>> previous email. We should quickly come up with 2.8.1 release in next 1
> >> or 2
> >>> month for production deployment.
> >>>
> >>>
> >>> We will close the vote in next 24 hours. For people who haven't vote,
> >>> please keep on verification work and report any issues if founded - I
> >> will
> >>> check if another round of RC is needed based on your findings. Thanks!
> >>>
> >>>
> >>> Thanks,
> >>>
> >>>
> >>> Junping
> >>>
> >>>
> >>> ________________________________
> >>> From: Kuhu Shukla <[hidden email]>
> >>> Sent: Tuesday, March 21, 2017 3:17 PM
> >>> Cc: Junping Du; [hidden email];
> [hidden email]
> >> ;
> >>> [hidden email]; [hidden email]
> >>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>>
> >>>
> >>> +1 (non-binding)
> >>>
> >>> - Verified signatures.
> >>> - Downloaded and built from source tar.gz.
> >>> - Deployed a pseudo-distributed cluster on Mac Sierra.
> >>> - Ran example Sleep job successfully.
> >>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> >>> successfully.
> >>>
> >>> Thank you Junping and everyone else who worked on getting this release
> >> out.
> >>>
> >>> Warm Regards,
> >>> Kuhu
> >>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> >>> <[hidden email]> wrote:
> >>> +1 (non-binding)
> >>>
> >>> - Verified checksums and signatures of all files
> >>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> >>> - Deployed single-node cluster
> >>> - Successfully ran a few sample jobs
> >>>
> >>> Thanks,
> >>>
> >>> Eric
> >>>
> >>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <[hidden email]>
> >>> wrote:
> >>>
> >>>
> >>>
> >>> +1. Thanks for the great effort, Junping!
> >>>
> >>>
> >>>  - Verified checksums and signatures of the tarballs
> >>>  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> >>>  - Built source and native code with Java 1.8.0_111 on Centos 7.2.1511
> >>>  - Cloud connectors:
> >>>      - s3a: integration tests, basic fs commands
> >>>      - adl: live unit tests, basic fs commands. See notes below.
> >>>  - Deployed a pseudo cluster, passed the following sanity tests in
> >>>  both insecure and SSL mode:
> >>>      - HDFS: basic dfs, distcp, ACL commands
> >>>      - KMS and HttpFS: basic tests
> >>>      - MapReduce wordcount
> >>>      - balancer start/stop
> >>>
> >>>
> >>> Needs the following JIRAs to pass all ADL tests:
> >>>
> >>>  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> >> Zhuge.
> >>>  - HDFS-11132. Allow AccessControlException in contract tests when
> >>>  getFileStatus on subdirectory of existing files. Contributed by
> >>> Vishwajeet
> >>>  Dusane
> >>>  - HADOOP-13928. TestAdlFileContextMainOperatio
> >> nsLive.testGetFileContext1
> >>>  runtime error. (John Zhuge via lei)
> >>>
> >>>
> >>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <[hidden email]>
> >> wrote:
> >>>
> >>>> Yes, it only affects ADL. There is a workaround of adding these 2
> >>>> properties to core-site.xml:
> >>>>
> >>>> <property>
> >>>>   <name>fs.adl.impl</name>
> >>>>   <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >>>> </property>
> >>>>
> >>>> <property>
> >>>>   <name>fs.AbstractFileSystem.adl.impl</name>
> >>>>   <value>org.apache.hadoop.fs.adl.Adl</value>
> >>>> </property>
> >>>>
> >>>> I have the initial patch ready but hitting these live unit test
> >> failures:
> >>>>
> >>>> Failed tests:
> >>>>
> >>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >>>> testListStatus:257
> >>>> expected:<1> but was:<10>
> >>>>
> >>>> Tests in error:
> >>>>
> >>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> >> est.
> >>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
> >>>> » AccessControl
> >>>>
> >>>> TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> >>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
> >>>> » AccessControl
> >>>>
> >>>>
> >>>> Stay tuned...
> >>>>
> >>>> John Zhuge
> >>>> Software Engineer, Cloudera
> >>>>
> >>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <[hidden email]>
> >>> wrote:
> >>>>
> >>>>> Thank you for reporting the issue, John! Does this issue only affect
> >>> ADL
> >>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
> >>>> existing
> >>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> >>>> not a
> >>>>> regression and just a new feature get broken.?
> >>>>>
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>>
> >>>>> Junping
> >>>>> ------------------------------
> >>>>> *From:* John Zhuge <[hidden email]>
> >>>>> *Sent:* Monday, March 20, 2017 9:07 AM
> >>>>> *To:* Junping Du
> >>>>> *Cc:* [hidden email]; [hidden email];
> >>>>> [hidden email]; [hidden email]
> >>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >>>>>
> >>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> >>>>> FileSystem for scheme: adl".
> >>>>>
> >>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
> >>>> earlier.
> >>>>> HADOOP-12666 should not be backported, but some changes are needed:
> >>>>> property fs.adl.impl in core-default.xml and
> >> hadoop-tools-dist/pom.xml.
> >>>>>
> >>>>> I am working on a patch.
> >>>>>
> >>>>>
> >>>>> John Zhuge
> >>>>> Software Engineer, Cloudera
> >>>>>
> >>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <[hidden email]>
> >>> wrote:
> >>>>>
> >>>>>> Hi all,
> >>>>>>     With fix of HDFS-11431 get in, I've created a new release
> >>> candidate
> >>>>>> (RC3) for Apache Hadoop 2.8.0.
> >>>>>>
> >>>>>>     This is the next minor release to follow up 2.7.0 which has
> >> been
> >>>>>> released for more than 1 year. It comprises 2,900+ fixes,
> >>> improvements,
> >>>> and
> >>>>>> new features. Most of these commits are released for the first time
> >> in
> >>>>>> branch-2.
> >>>>>>
> >>>>>>     More information about the 2.8.0 release plan can be found
> >> here:
> >>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> >> Hadoop+2.8+Release
> >>>>>>
> >>>>>>     New RC is available at: http://home.apache.org/~junpin
> >>>>>> g_du/hadoop-2.8.0-RC3
> >>>>>>
> >>>>>>     The RC tag in git is: release-2.8.0-RC3, and the latest commit
> >> id
> >>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >>>>>>
> >>>>>>     The maven artifacts are available via repository.apache.org
> >> at:
> >>>>>> https://repository.apache.org/content/repositories/
> >>> orgapachehadoop-1057
> >>>>>>
> >>>>>>     Please try the release and vote; the vote will run for the
> >> usual
> >>> 5
> >>>>>> days, ending on 03/22/2017 PDT time.
> >>>
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Junping
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> John
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: [hidden email]
> >>> For additional commands, e-mail: [hidden email]
> >>>
> >>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Naganarasimha Garla-2
Thanks Junping for putting this up,.

+1 (non-binding)

* downloaded verified signature and md5.
* Built from source & deployed  pseudo cluster with labels enabled
* ran some sample jobs with partitions specified.
* Navigate through YARN & MR UI's
* executed basic dfs cmds.

Regards,
+ Naga



On Wed, Mar 22, 2017 at 11:23 PM, Rakesh Radhakrishnan <[hidden email]>
wrote:

> Thanks Junping for getting this out.
>
> +1 (non-binding)
>
> * downloaded and built from source with jdk1.8.0_45
> * deployed HDFS-HA cluster
> * ran some sample jobs
> * run balancer
> * executed basic dfs cmds
>
>
> Rakesh
>
> On Wed, Mar 22, 2017 at 8:30 PM, Jian He <[hidden email]> wrote:
>
> > +1 (binding)
> >
> > - built from source
> > - deployed a pseudo cluster
> > - ran basic example tests.
> > - Navigate the UI a bit, looks good.
> >
> > Jian
> >
> > > On Mar 22, 2017, at 9:03 PM, larry mccay <[hidden email]>
> wrote:
> > >
> > > +1 (non-binding)
> > >
> > > - verified signatures
> > > - built from source and ran tests
> > > - deployed pseudo cluster
> > > - ran basic tests for hdfs, wordcount, credential provider API and
> > related
> > > commands
> > > - tested webhdfs with knox
> > >
> > >
> > > On Wed, Mar 22, 2017 at 7:21 AM, Ravi Prakash <[hidden email]>
> > wrote:
> > >
> > >> Thanks for all the effort Junping!
> > >>
> > >> +1 (binding)
> > >> + Verified signature and MD5, SHA1, SHA256 checksum of tarball
> > >> + Verified SHA ID in git corresponds to RC3 tag
> > >> + Verified wordcount for one small text file produces same output as
> > >> hadoop-2.7.3.
> > >> + HDFS Namenode UI looks good.
> > >>
> > >> I agree none of the issues reported so far are blockers. Looking
> > forward to
> > >> another great release.
> > >>
> > >> Thanks
> > >> Ravi
> > >>
> > >> On Tue, Mar 21, 2017 at 8:10 PM, Junping Du <[hidden email]>
> > wrote:
> > >>
> > >>> Thanks all for response with verification work and vote!
> > >>>
> > >>>
> > >>> Sounds like we are hitting several issues here, although none seems
> to
> > be
> > >>> blockers so far. Given the large commit set - 2000+ commits first
> > landed
> > >> in
> > >>> branch-2 release, we may should follow 2.7.0 practice that to claim
> > this
> > >>> release is not for production cluster, just like Vinod's suggestion
> in
> > >>> previous email. We should quickly come up with 2.8.1 release in next
> 1
> > >> or 2
> > >>> month for production deployment.
> > >>>
> > >>>
> > >>> We will close the vote in next 24 hours. For people who haven't vote,
> > >>> please keep on verification work and report any issues if founded - I
> > >> will
> > >>> check if another round of RC is needed based on your findings.
> Thanks!
> > >>>
> > >>>
> > >>> Thanks,
> > >>>
> > >>>
> > >>> Junping
> > >>>
> > >>>
> > >>> ________________________________
> > >>> From: Kuhu Shukla <[hidden email]>
> > >>> Sent: Tuesday, March 21, 2017 3:17 PM
> > >>> Cc: Junping Du; [hidden email];
> > [hidden email]
> > >> ;
> > >>> [hidden email]; [hidden email]
> > >>> Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >>>
> > >>>
> > >>> +1 (non-binding)
> > >>>
> > >>> - Verified signatures.
> > >>> - Downloaded and built from source tar.gz.
> > >>> - Deployed a pseudo-distributed cluster on Mac Sierra.
> > >>> - Ran example Sleep job successfully.
> > >>> - Deployed latest Apache Tez 0.9 and ran sample Tez orderedwordcount
> > >>> successfully.
> > >>>
> > >>> Thank you Junping and everyone else who worked on getting this
> release
> > >> out.
> > >>>
> > >>> Warm Regards,
> > >>> Kuhu
> > >>> On Tuesday, March 21, 2017, 3:42:46 PM CDT, Eric Badger
> > >>> <[hidden email]> wrote:
> > >>> +1 (non-binding)
> > >>>
> > >>> - Verified checksums and signatures of all files
> > >>> - Built from source on MacOS Sierra via JDK 1.8.0 u65
> > >>> - Deployed single-node cluster
> > >>> - Successfully ran a few sample jobs
> > >>>
> > >>> Thanks,
> > >>>
> > >>> Eric
> > >>>
> > >>> On Tuesday, March 21, 2017 2:56 PM, John Zhuge <[hidden email]
> >
> > >>> wrote:
> > >>>
> > >>>
> > >>>
> > >>> +1. Thanks for the great effort, Junping!
> > >>>
> > >>>
> > >>>  - Verified checksums and signatures of the tarballs
> > >>>  - Built source code with Java 1.8.0_66-b17 on Mac OS X 10.12.3
> > >>>  - Built source and native code with Java 1.8.0_111 on Centos
> 7.2.1511
> > >>>  - Cloud connectors:
> > >>>      - s3a: integration tests, basic fs commands
> > >>>      - adl: live unit tests, basic fs commands. See notes below.
> > >>>  - Deployed a pseudo cluster, passed the following sanity tests in
> > >>>  both insecure and SSL mode:
> > >>>      - HDFS: basic dfs, distcp, ACL commands
> > >>>      - KMS and HttpFS: basic tests
> > >>>      - MapReduce wordcount
> > >>>      - balancer start/stop
> > >>>
> > >>>
> > >>> Needs the following JIRAs to pass all ADL tests:
> > >>>
> > >>>  - HADOOP-14205. No FileSystem for scheme: adl. Contributed by John
> > >> Zhuge.
> > >>>  - HDFS-11132. Allow AccessControlException in contract tests when
> > >>>  getFileStatus on subdirectory of existing files. Contributed by
> > >>> Vishwajeet
> > >>>  Dusane
> > >>>  - HADOOP-13928. TestAdlFileContextMainOperatio
> > >> nsLive.testGetFileContext1
> > >>>  runtime error. (John Zhuge via lei)
> > >>>
> > >>>
> > >>> On Mon, Mar 20, 2017 at 10:31 AM, John Zhuge <[hidden email]>
> > >> wrote:
> > >>>
> > >>>> Yes, it only affects ADL. There is a workaround of adding these 2
> > >>>> properties to core-site.xml:
> > >>>>
> > >>>> <property>
> > >>>>   <name>fs.adl.impl</name>
> > >>>>   <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > >>>> </property>
> > >>>>
> > >>>> <property>
> > >>>>   <name>fs.AbstractFileSystem.adl.impl</name>
> > >>>>   <value>org.apache.hadoop.fs.adl.Adl</value>
> > >>>> </property>
> > >>>>
> > >>>> I have the initial patch ready but hitting these live unit test
> > >> failures:
> > >>>>
> > >>>> Failed tests:
> > >>>>
> > >>>> TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > >>>> testListStatus:257
> > >>>> expected:<1> but was:<10>
> > >>>>
> > >>>> Tests in error:
> > >>>>
> > >>>> TestAdlFileContextMainOperationsLive>FileContextMainOperationsBaseT
> > >> est.
> > >>>> testMkdirsFailsForSubdirectoryOfExistingFile:254
> > >>>> » AccessControl
> > >>>>
> > >>>> TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > >>>> testMkdirsFailsForSubdirectoryOfExistingFile:190
> > >>>> » AccessControl
> > >>>>
> > >>>>
> > >>>> Stay tuned...
> > >>>>
> > >>>> John Zhuge
> > >>>> Software Engineer, Cloudera
> > >>>>
> > >>>> On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <[hidden email]>
> > >>> wrote:
> > >>>>
> > >>>>> Thank you for reporting the issue, John! Does this issue only
> affect
> > >>> ADL
> > >>>>> (Azure Data Lake) which is a new feature for 2.8 rather than other
> > >>>> existing
> > >>>>> FS? If so, I think we can leave the fix to 2.8.1 to fix given this
> is
> > >>>> not a
> > >>>>> regression and just a new feature get broken.?
> > >>>>>
> > >>>>>
> > >>>>> Thanks,
> > >>>>>
> > >>>>>
> > >>>>> Junping
> > >>>>> ------------------------------
> > >>>>> *From:* John Zhuge <[hidden email]>
> > >>>>> *Sent:* Monday, March 20, 2017 9:07 AM
> > >>>>> *To:* Junping Du
> > >>>>> *Cc:* [hidden email]; [hidden email];
> > >>>>> [hidden email]; [hidden email]
> > >>>>> *Subject:* Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >>>>>
> > >>>>> Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > >>>>> FileSystem for scheme: adl".
> > >>>>>
> > >>>>> The issue were caused by backporting HADOOP-13037 to branch-2 and
> > >>>> earlier.
> > >>>>> HADOOP-12666 should not be backported, but some changes are needed:
> > >>>>> property fs.adl.impl in core-default.xml and
> > >> hadoop-tools-dist/pom.xml.
> > >>>>>
> > >>>>> I am working on a patch.
> > >>>>>
> > >>>>>
> > >>>>> John Zhuge
> > >>>>> Software Engineer, Cloudera
> > >>>>>
> > >>>>> On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <[hidden email]>
> > >>> wrote:
> > >>>>>
> > >>>>>> Hi all,
> > >>>>>>     With fix of HDFS-11431 get in, I've created a new release
> > >>> candidate
> > >>>>>> (RC3) for Apache Hadoop 2.8.0.
> > >>>>>>
> > >>>>>>     This is the next minor release to follow up 2.7.0 which has
> > >> been
> > >>>>>> released for more than 1 year. It comprises 2,900+ fixes,
> > >>> improvements,
> > >>>> and
> > >>>>>> new features. Most of these commits are released for the first
> time
> > >> in
> > >>>>>> branch-2.
> > >>>>>>
> > >>>>>>     More information about the 2.8.0 release plan can be found
> > >> here:
> > >>>>>> https://cwiki.apache.org/confluence/display/HADOOP/
> > >> Hadoop+2.8+Release
> > >>>>>>
> > >>>>>>     New RC is available at: http://home.apache.org/~junpin
> > >>>>>> g_du/hadoop-2.8.0-RC3
> > >>>>>>
> > >>>>>>     The RC tag in git is: release-2.8.0-RC3, and the latest commit
> > >> id
> > >>>>>> is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >>>>>>
> > >>>>>>     The maven artifacts are available via repository.apache.org
> > >> at:
> > >>>>>> https://repository.apache.org/content/repositories/
> > >>> orgapachehadoop-1057
> > >>>>>>
> > >>>>>>     Please try the release and vote; the vote will run for the
> > >> usual
> > >>> 5
> > >>>>>> days, ending on 03/22/2017 PDT time.
> > >>>
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>>
> > >>>>>> Junping
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> John
> > >>>
> > >>>
> > >>> ------------------------------------------------------------
> ---------
> > >>> To unsubscribe, e-mail: [hidden email]
> > >>> For additional commands, e-mail: [hidden email]
> > >>>
> > >>
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

varunsaxena@apache.org
In reply to this post by Haibo Chen
Thanks Junping for creating the release.

+1 (non-binding)

* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.

Regards,
Varun Saxena.

On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <[hidden email]> wrote:

> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <[hidden email]> wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <[hidden email]>
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: [hidden email]; [hidden email];
> > [hidden email]; [hidden email]
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> >   <property>
> >     <name>fs.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >   </property>
> >
> >   <property>
> >     <name>fs.AbstractFileSystem.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.Adl</value>
> >   </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >   TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <[hidden email]
> <mailto:
> > [hidden email]>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <[hidden email]<mailto:[hidden email]>>
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: [hidden email]<mailto:[hidden email]>;
> > [hidden email]<mailto:[hidden email]>;
> > [hidden email]<mailto:[hidden email]>;
> > [hidden email]<mailto:[hidden email]>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <[hidden email]<mailto:
> jd
> > [hidden email]>> wrote:
> > Hi all,
> >      With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> >      This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> >       More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> >       New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> >       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> >       The maven artifacts are available via repository.apache.org
> <http://
> > repository.apache.org> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> >       Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Karthik Kambatla-2
+1 (binding)

* Built from source
* Started a pseudo-distributed cluster with fairscheduler.
* Ran sample jobs
* Verified WebUI

On Wed, Mar 22, 2017 at 11:56 AM, [hidden email] <
[hidden email]> wrote:

> Thanks Junping for creating the release.
>
> +1 (non-binding)
>
> * Verified signatures.
> * Built from source.
> * Set up a pseudo-distributed cluster.
> * Successfully ran pi and wordcount jobs.
> * Navigated the YARN RM and NM UI.
>
> Regards,
> Varun Saxena.
>
> On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <[hidden email]>
> wrote:
>
> > Thanks Junping for working on the new release!
> >
> > +1 non-binding
> >
> > 1) Downloaded the source, verified the checksum
> > 2) Built natively from source, and deployed it to a pseudo-distributed
> > cluster
> > 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> > 4) Played with yarn + mapreduce command lines
> >
> > Best,
> > Haibo Chen
> >
> > On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <[hidden email]>
> wrote:
> >
> > > ?Thanks for update, John. Then we should be OK with fixing this issue
> in
> > > 2.8.1.
> > >
> > > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> > bump
> > > up to blocker in case we could miss this in releasing 2.8.1. :)
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > >
> > > ________________________________
> > > From: John Zhuge <[hidden email]>
> > > Sent: Monday, March 20, 2017 10:31 AM
> > > To: Junping Du
> > > Cc: [hidden email]; [hidden email];
> > > [hidden email]; [hidden email]
> > > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Yes, it only affects ADL. There is a workaround of adding these 2
> > > properties to core-site.xml:
> > >
> > >   <property>
> > >     <name>fs.adl.impl</name>
> > >     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> > >   </property>
> > >
> > >   <property>
> > >     <name>fs.AbstractFileSystem.adl.impl</name>
> > >     <value>org.apache.hadoop.fs.adl.Adl</value>
> > >   </property>
> > >
> > > I have the initial patch ready but hitting these live unit test
> failures:
> > >
> > > Failed tests:
> > >   TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> > BaseTest.testListStatus:257
> > > expected:<1> but was:<10>
> > >
> > > Tests in error:
> > >   TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> > nsBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> > >   TestAdlFileSystemContractLive.runTest:60->
> FileSystemContractBaseTest.
> > > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> > >
> > >
> > > Stay tuned...
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <[hidden email]
> > <mailto:
> > > [hidden email]>> wrote:
> > >
> > > Thank you for reporting the issue, John! Does this issue only affect
> ADL
> > > (Azure Data Lake) which is a new feature for 2.8 rather than other
> > existing
> > > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> > not a
> > > regression and just a new feature get broken.?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > >
> > > ________________________________
> > > From: John Zhuge <[hidden email]<mailto:[hidden email]>>
> > > Sent: Monday, March 20, 2017 9:07 AM
> > > To: Junping Du
> > > Cc: [hidden email]<mailto:[hidden email]>;
> > > [hidden email]<mailto:[hidden email]>;
> > > [hidden email]<mailto:[hidden email]>;
> > > [hidden email]<mailto:[hidden email]
> >
> > > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> > >
> > > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > > FileSystem for scheme: adl".
> > >
> > > The issue were caused by backporting HADOOP-13037 to branch-2 and
> > earlier.
> > > HADOOP-12666 should not be backported, but some changes are needed:
> > > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> > >
> > > I am working on a patch.
> > >
> > >
> > > John Zhuge
> > > Software Engineer, Cloudera
> > >
> > > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <[hidden email]
> <mailto:
> > jd
> > > [hidden email]>> wrote:
> > > Hi all,
> > >      With fix of HDFS-11431 get in, I've created a new release
> candidate
> > > (RC3) for Apache Hadoop 2.8.0.
> > >
> > >      This is the next minor release to follow up 2.7.0 which has been
> > > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> > and
> > > new features. Most of these commits are released for the first time in
> > > branch-2.
> > >
> > >       More information about the 2.8.0 release plan can be found here:
> > > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> > >
> > >       New RC is available at: http://home.apache.org/~
> > > junping_du/hadoop-2.8.0-RC3
> > >
> > >       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> > >
> > >       The maven artifacts are available via repository.apache.org
> > <http://
> > > repository.apache.org> at: https://repository.apache.org/
> > > content/repositories/orgapachehadoop-1057
> > >
> > >       Please try the release and vote; the vote will run for the usual
> 5
> > > days, ending on 03/22/2017 PDT time.
> > >
> > > Thanks,
> > >
> > > Junping
> > >
> > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

[RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Junping Du-2
Thanks again for all who verified and voted!


I give my binding +1 to conclude the vote for 2.8.0 RC3, based on:
- Build from source and verify signatures
- Deploy pseudo-distributed cluster with capacity scheduler
- Verify UI of daemons, like: NameNode, ResourceManager, NodeManager, etc.
- Run some example MR jobs, like: PI, sleep, etc.

Now, we have:

7 binding +1s, from:
     Wangda Tan, Jason Lowe, Akira Ajisaka, Ravi Prakash,
     Karthik Kambatla, Jian He, Junping Du

18 non-binding +1s, from:
    Miklos Szegedi, Eric Payne, Daniel Templeton, Mingliang Liu,
    Sunil Govind, Marton Elek, Brahma Reddy Battula, Masatake Iwasaki,
    Gergo Pasztor, Haibo Chen, Zhihai Xu, John Zhuge, Eric Badger,
    Kuhu Shukla, Larry Mccay, Rakesh Radhakrishnan, Naganarasimha Garla,
    Varun Saxena

1 binding +0, from:
    Steve Loughran

and no -1s.

So I am glad to announce that the vote of 2.8.0 RC3 passes.

Thanks everyone listed above who tried the release candidate and vote.
Also, kudos to all who ever help with 2.8.0 release effort in all kinds of ways.
Without working together in community, we cannot afford so many issues get found
in RC stage (as a minor release with debit of ~2 years commits) and most of them
get fixed so quickly.

I'll push the release bits and send out an announcement for 2.8.0 soon.


Thanks,

Junping​


________________________________
From: Karthik Kambatla <[hidden email]>
Sent: Wednesday, March 22, 2017 2:10 PM
To: [hidden email]
Cc: Junping Du; [hidden email]; [hidden email]; [hidden email]; [hidden email]
Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)

+1 (binding)

* Built from source
* Started a pseudo-distributed cluster with fairscheduler.
* Ran sample jobs
* Verified WebUI

On Wed, Mar 22, 2017 at 11:56 AM, [hidden email]<mailto:[hidden email]> <[hidden email]<mailto:[hidden email]>> wrote:
Thanks Junping for creating the release.

+1 (non-binding)

* Verified signatures.
* Built from source.
* Set up a pseudo-distributed cluster.
* Successfully ran pi and wordcount jobs.
* Navigated the YARN RM and NM UI.

Regards,
Varun Saxena.

On Wed, Mar 22, 2017 at 12:13 AM, Haibo Chen <[hidden email]<mailto:[hidden email]>> wrote:

> Thanks Junping for working on the new release!
>
> +1 non-binding
>
> 1) Downloaded the source, verified the checksum
> 2) Built natively from source, and deployed it to a pseudo-distributed
> cluster
> 3) Ran sleep and teragen job and checked both YARN and JHS web UI
> 4) Played with yarn + mapreduce command lines
>
> Best,
> Haibo Chen
>
> On Mon, Mar 20, 2017 at 11:18 AM, Junping Du <[hidden email]<mailto:[hidden email]>> wrote:
>
> > ?Thanks for update, John. Then we should be OK with fixing this issue in
> > 2.8.1.
> >
> > Mark the target version of HADOOP-14205 to 2.8.1 instead of 2.8.0 and
> bump
> > up to blocker in case we could miss this in releasing 2.8.1. :)
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <[hidden email]<mailto:[hidden email]>>
> > Sent: Monday, March 20, 2017 10:31 AM
> > To: Junping Du
> > Cc: [hidden email]<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>;
> > [hidden email]<mailto:[hidden email]>; [hidden email]<mailto:[hidden email]>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Yes, it only affects ADL. There is a workaround of adding these 2
> > properties to core-site.xml:
> >
> >   <property>
> >     <name>fs.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.AdlFileSystem</value>
> >   </property>
> >
> >   <property>
> >     <name>fs.AbstractFileSystem.adl.impl</name>
> >     <value>org.apache.hadoop.fs.adl.Adl</value>
> >   </property>
> >
> > I have the initial patch ready but hitting these live unit test failures:
> >
> > Failed tests:
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContract
> BaseTest.testListStatus:257
> > expected:<1> but was:<10>
> >
> > Tests in error:
> >   TestAdlFileContextMainOperationsLive>FileContextMainOperatio
> nsBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:254 » AccessControl
> >   TestAdlFileSystemContractLive.runTest:60->FileSystemContractBaseTest.
> > testMkdirsFailsForSubdirectoryOfExistingFile:190 » AccessControl
> >
> >
> > Stay tuned...
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Mon, Mar 20, 2017 at 10:02 AM, Junping Du <[hidden email]<mailto:[hidden email]>
> <mailto:
> > [hidden email]<mailto:[hidden email]>>> wrote:
> >
> > Thank you for reporting the issue, John! Does this issue only affect ADL
> > (Azure Data Lake) which is a new feature for 2.8 rather than other
> existing
> > FS? If so, I think we can leave the fix to 2.8.1 to fix given this is
> not a
> > regression and just a new feature get broken.?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> > ________________________________
> > From: John Zhuge <[hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>>>
> > Sent: Monday, March 20, 2017 9:07 AM
> > To: Junping Du
> > Cc: [hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>>;
> > [hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>>;
> > [hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>>;
> > [hidden email]<mailto:[hidden email]><mailto:[hidden email]<mailto:[hidden email]>>
> > Subject: Re: [VOTE] Release Apache Hadoop 2.8.0 (RC3)
> >
> > Discovered https://issues.apache.org/jira/browse/HADOOP-14205 "No
> > FileSystem for scheme: adl".
> >
> > The issue were caused by backporting HADOOP-13037 to branch-2 and
> earlier.
> > HADOOP-12666 should not be backported, but some changes are needed:
> > property fs.adl.impl in core-default.xml and hadoop-tools-dist/pom.xml.
> >
> > I am working on a patch.
> >
> >
> > John Zhuge
> > Software Engineer, Cloudera
> >
> > On Fri, Mar 17, 2017 at 2:18 AM, Junping Du <[hidden email]<mailto:[hidden email]><mailto:
> jd
> > [hidden email]<mailto:[hidden email]>>> wrote:
> > Hi all,
> >      With fix of HDFS-11431 get in, I've created a new release candidate
> > (RC3) for Apache Hadoop 2.8.0.
> >
> >      This is the next minor release to follow up 2.7.0 which has been
> > released for more than 1 year. It comprises 2,900+ fixes, improvements,
> and
> > new features. Most of these commits are released for the first time in
> > branch-2.
> >
> >       More information about the 2.8.0 release plan can be found here:
> > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> >       New RC is available at: http://home.apache.org/~
> > junping_du/hadoop-2.8.0-RC3
> >
> >       The RC tag in git is: release-2.8.0-RC3, and the latest commit id
> > is: 91f2b7a13d1e97be65db92ddabc627cc29ac0009
> >
> >       The maven artifacts are available via repository.apache.org<http://repository.apache.org>
> <http://
> > repository.apache.org<http://repository.apache.org>> at: https://repository.apache.org/
> > content/repositories/orgapachehadoop-1057
> >
> >       Please try the release and vote; the vote will run for the usual 5
> > days, ending on 03/22/2017 PDT time.
> >
> > Thanks,
> >
> > Junping
> >
> >
> >
>

Reply | Threaded
Open this post in threaded view
|

Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Allen Wittenauer-6

Just a heads up.  Looks like some removed the Finish Date off of 2.8.0 in JIRA.  It needs to be put back to match what is in the artifacts that we voted on.
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Junping Du-2
Thanks Allen, I already add it back in JIRA's 2.8.0 release date.
However, I met another issue in uploading our 2.8.0 release bits to SVN: hadoop-2.8.0.tar.gz get uploaded failed with following exception:
....
Adding  (bin)  hadoop-2.8.0/hadoop-2.8.0.tar.gz
Transmitting file data .svn: E175002: Commit failed (details follow):
svn: E175002: PUT request on '/repos/dist/!svn/txr/18902-g1s/release/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz' failed

Other (smaller) files get uploaded successfully. Once I suspected it could due to no space, so I removed some old release bits (like: 2.5.2) but still no luck. So may be the size of hadoop-2.8.0.tar.gz become a problem here?
I already reported the issue in https://issues.apache.org/jira/browse/INFRA-13749, but will appreciate if someone got idea on how to get through this.

Thanks,

Junping
________________________________________
From: Allen Wittenauer <[hidden email]>
Sent: Thursday, March 23, 2017 5:27 PM
To: Junping Du
Cc: [hidden email]; [hidden email]; [hidden email]; [hidden email]
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Just a heads up.  Looks like some removed the Finish Date off of 2.8.0 in JIRA.  It needs to be put back to match what is in the artifacts that we voted on.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Junping Du-2
Quick update: the missing bits in svn release directories (reported in https://issues.apache.org/jira/browse/INFRA-13749) already get resolved with workaround of bypassing 200M file size limitation. We should document this in our release wiki.
However, a new issue is found that releasing bits are missing from mirror sites which get reported in https://issues.apache.org/jira/browse/INFRA-13755. The Apache infrastructure team is investigating right now. Will send release news when issue get fixed. Stay tuned!

Thanks,

Junping
________________________________________
From: Junping Du <[hidden email]>
Sent: Friday, March 24, 2017 6:21 PM
To: Allen Wittenauer
Cc: [hidden email]; [hidden email]; [hidden email]; [hidden email]
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Thanks Allen, I already add it back in JIRA's 2.8.0 release date.
However, I met another issue in uploading our 2.8.0 release bits to SVN: hadoop-2.8.0.tar.gz get uploaded failed with following exception:
....
Adding  (bin)  hadoop-2.8.0/hadoop-2.8.0.tar.gz
Transmitting file data .svn: E175002: Commit failed (details follow):
svn: E175002: PUT request on '/repos/dist/!svn/txr/18902-g1s/release/hadoop/common/hadoop-2.8.0/hadoop-2.8.0.tar.gz' failed

Other (smaller) files get uploaded successfully. Once I suspected it could due to no space, so I removed some old release bits (like: 2.5.2) but still no luck. So may be the size of hadoop-2.8.0.tar.gz become a problem here?
I already reported the issue in https://issues.apache.org/jira/browse/INFRA-13749, but will appreciate if someone got idea on how to get through this.

Thanks,

Junping
________________________________________
From: Allen Wittenauer <[hidden email]>
Sent: Thursday, March 23, 2017 5:27 PM
To: Junping Du
Cc: [hidden email]; [hidden email]; [hidden email]; [hidden email]
Subject: Re: [RESULT] [VOTE] Release Apache Hadoop 2.8.0 (RC3)

Just a heads up.  Looks like some removed the Finish Date off of 2.8.0 in JIRA.  It needs to be put back to match what is in the artifacts that we voted on.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

123