test failures - newbie help needed

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

test failures - newbie help needed

Simon Scott
Hi

Please excuse me. I am new to building Hadoop. I am attempting to build and run tests for the trunk and encountering a series of test failures which for sure indicates that I am doing something wrong. I have scoured the wiki and can't see what I'm missing. Can anybody offer any advice?

I am running the tests in Ubuntu 14.04 hosted in VirtualBox on Windows 8.1 host.

Using this command: "mvn clean test"

List of failed tests below.

Many thanks
Simon

Failed tests:
  TestHDFSFileSystemContract>FileSystemContractBaseTest.testRenameDirectoryAsExistingDirectory:536 Renamed nested file1 exists
  TestTrashWithEncryptionZones.testDeleteEZWithMultipleUsers:164 Non-admin could delete an encryption zone with multiple users : /zones/zone1 expected:<1> but was:<0>
  TestDataNodeVolumeFailure.testFailedVolumeBeingRemovedFromDataNode:247 expected:<1> but was:<2>
  TestFileContextXAttr>FSXAttrBaseTest.testUnreadableBySuperuserXAttr:1142->FSXAttrBaseTest.access$000:67->FSXAttrBaseTest.doTestUnreadableBySuperuserXAttr:1158->FSXAttrBaseTest.doTUBSXAInt:1213->FSXAttrBaseTest.verifyFileAccess:1241 open failed but expected it to succeed
  TestFileContextXAttr>FSXAttrBaseTest.testRawXAttrs:1024 setXAttr should have thrown
  TestFileContextXAttr>FSXAttrBaseTest.testListXAttrs:820 expected IOException
  TestFileContextXAttr>FSXAttrBaseTest.testGetXAttrs:458 expected IOException
  TestNameNodeXAttr>FSXAttrBaseTest.testUnreadableBySuperuserXAttr:1142->FSXAttrBaseTest.access$000:67->FSXAttrBaseTest.doTestUnreadableBySuperuserXAttr:1158->FSXAttrBaseTest.doTUBSXAInt:1213->FSXAttrBaseTest.verifyFileAccess:1241 open failed but expected it to succeed
  TestNameNodeXAttr>FSXAttrBaseTest.testRawXAttrs:1024 setXAttr should have thrown
  TestNameNodeXAttr>FSXAttrBaseTest.testListXAttrs:820 expected IOException
  TestNameNodeXAttr>FSXAttrBaseTest.testGetXAttrs:458 expected IOException
  TestNamenodeCapacityReport.testXceiverCount:198->testXceiverCountInternal:344 Test resulted in an unexpected exit
  TestEncryptionZonesWithKMS>TestEncryptionZones.testGetEZAsNonSuperUser:581 expected AccessControlException
  TestEncryptionZonesWithKMS>TestEncryptionZones.testBasicOperations:441 createEncryptionZone is superuser-only operation
  TestReservedRawPaths.testAdminAccessOnly:262 access to /.reserved/raw is superuser-only operation
  TestDFSShell.testSetXAttrPermission:2992 Returned should be 1 expected:<1> but was:<0>
  TestWebHDFSXAttr>FSXAttrBaseTest.testUnreadableBySuperuserXAttr:1142->FSXAttrBaseTest.access$000:67->FSXAttrBaseTest.doTestUnreadableBySuperuserXAttr:1158->FSXAttrBaseTest.doTUBSXAInt:1213->FSXAttrBaseTest.verifyFileAccess:1241 open failed but expected it to succeed
  TestWebHDFSXAttr>FSXAttrBaseTest.testRawXAttrs:1024 setXAttr should have thrown
  TestWebHDFSXAttr>FSXAttrBaseTest.testListXAttrs:820 expected IOException
  TestWebHDFSXAttr>FSXAttrBaseTest.testGetXAttrs:458 expected IOException
  TestWebHdfsFileSystemContract>FileSystemContractBaseTest.testRenameDirectoryAsExistingDirectory:536 Renamed nested file1 exists
  TestEncryptionZones.testGetEZAsNonSuperUser:581 expected AccessControlException
  TestEncryptionZones.testBasicOperations:441 createEncryptionZone is superuser-only operation

Tests in error:
  TestRBWBlockInvalidation.testBlockInvalidationWhenRBWReplicaMissedInDN:122 >  ...
  TestDataNodeVolumeFailure.tearDown:142->Object.wait:-2 >  test timed out after...
  TestDataNodeVolumeFailureReporting.tearDown:109->Object.wait:-2 >  test timed ...
  TestDataNodeVolumeFailureReporting.tearDown:109 >  test timed out after 120000...
  TestDataNodeVolumeFailureReporting.tearDown:109 >  test timed out after 120000...
  TestDataNodeVolumeFailureReporting.tearDown:109->Object.wait:-2 >  test timed ...
  TestFileContextXAttr>FSXAttrBaseTest.testRemoveXAttrPermissions:656 > Remote N...
  TestNameNodeMetadataConsistency.testGenerationStampInFuture:109->waitForNumBytes:161 > Timeout
  TestNameNodeXAttr>FSXAttrBaseTest.testRemoveXAttrPermissions:656 > Remote No m...
  TestFSImage.testCompression:96->setCompressCodec:102->testPersistHelper:108 > IO
  TestDiskBalancerCommand.testRunMultipleCommandsUnderOneSetup > Remote File /sy...
  TestDiskBalancer.testBalanceDataBetweenMultiplePairsOfVolumes:187 > IllegalArgument
  TestDiskBalancerRPC.testMoveBlockAcrossVolume > Remote File /tmp.txt could onl...
  TestDFSRSDefault10x4StripedOutputStream>TestDFSStripedOutputStream.setup:91 > DiskError
  TestDFSRSDefault10x4StripedOutputStream>TestDFSStripedOutputStream.testFileSmallerThanOneStripe2:132->TestDFSStripedOutputStream.testOneFile:187 > Timeout
  TestDFSRSDefault10x4StripedOutputStream>TestDFSStripedOutputStream.testFileMoreThanABlockGroup3:176->TestDFSStripedOutputStream.testOneFile:186 > IO
  TestParallelRead.setupCluster:37->TestParallelReadUtil.setupCluster:71 > IO Ca...
  TestParallelRead.teardownCluster:42->TestParallelReadUtil.teardownCluster:394 > NullPointer
  TestSaslDataTransfer.testNoSaslAndSecurePortsIgnored:175->startCluster:207 > FileNotFound
  TestDatanodeLayoutUpgrade.testUpgradeToIdBasedLayout:52 > IO Timed out waiting...
  TestDFSMkdirs.testMkdir:94 > DiskError Too many failed volumes - current valid...
  TestWebHDFSXAttr>FSXAttrBaseTest.testRemoveXAttrPermissions:656 > IO No matchi...

Tests run: 4997, Failures: 23, Errors: 22, Skipped: 150
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: test failures - newbie help needed

Steve Loughran-3
some of this looks network related. Ubuntu often makes a mess of the loopback address of localhost: find the localhost entry in /etc/hosts and change it to 127.0.0.1 and see if that helps

otherwise, the hdfs-dev list is the one to really ping about HDFS issues, which all of these appear to be. Mostly related to XAttrs, interestingly enough


Pick one isolated test, like testWebHDFSXAttr#FSXAttrBaseTest.testListXAttr ; run it with the log4j.properties in hdfs src/test/resources set to log org.apache.hadoop at debug and see what is going on underneath. That's generally how I start debugging test failures, before switching to the IDE and stepping through.

Good luck

Steve

PS if it makes you feel any better, staring a test run failures is a core skill of all Hadoop developers


> On 16 Mar 2017, at 10:33, Simon Scott <[hidden email]> wrote:
>
> Hi
>
> Please excuse me. I am new to building Hadoop. I am attempting to build and run tests for the trunk and encountering a series of test failures which for sure indicates that I am doing something wrong. I have scoured the wiki and can't see what I'm missing. Can anybody offer any advice?
>
> I am running the tests in Ubuntu 14.04 hosted in VirtualBox on Windows 8.1 host.
>
> Using this command: "mvn clean test"
>
> List of failed tests below.
>
> Many thanks
> Simon
>
> Failed tests:
>  TestHDFSFileSystemContract>FileSystemContractBaseTest.testRenameDirectoryAsExistingDirectory:536 Renamed nested file1 exists
>  TestTrashWithEncryptionZones.testDeleteEZWithMultipleUsers:164 Non-admin could delete an encryption zone with multiple users : /zones/zone1 expected:<1> but was:<0>
>  TestDataNodeVolumeFailure.testFailedVolumeBeingRemovedFromDataNode:247 expected:<1> but was:<2>
>  TestFileContextXAttr>FSXAttrBaseTest.testUnreadableBySuperuserXAttr:1142->FSXAttrBaseTest.access$000:67->FSXAttrBaseTest.doTestUnreadableBySuperuserXAttr:1158->FSXAttrBaseTest.doTUBSXAInt:1213->FSXAttrBaseTest.verifyFileAccess:1241 open failed but expected it to succeed
>  TestFileContextXAttr>FSXAttrBaseTest.testRawXAttrs:1024 setXAttr should have thrown
>  TestFileContextXAttr>FSXAttrBaseTest.testListXAttrs:820 expected IOException
>  TestFileContextXAttr>FSXAttrBaseTest.testGetXAttrs:458 expected IOException
>  TestNameNodeXAttr>FSXAttrBaseTest.testUnreadableBySuperuserXAttr:1142->FSXAttrBaseTest.access$000:67->FSXAttrBaseTest.doTestUnreadableBySuperuserXAttr:1158->FSXAttrBaseTest.doTUBSXAInt:1213->FSXAttrBaseTest.verifyFileAccess:1241 open failed but expected it to succeed
>  TestNameNodeXAttr>FSXAttrBaseTest.testRawXAttrs:1024 setXAttr should have thrown
>  TestNameNodeXAttr>FSXAttrBaseTest.testListXAttrs:820 expected IOException
>  TestNameNodeXAttr>FSXAttrBaseTest.testGetXAttrs:458 expected IOException
>  TestNamenodeCapacityReport.testXceiverCount:198->testXceiverCountInternal:344 Test resulted in an unexpected exit
>  TestEncryptionZonesWithKMS>TestEncryptionZones.testGetEZAsNonSuperUser:581 expected AccessControlException
>  TestEncryptionZonesWithKMS>TestEncryptionZones.testBasicOperations:441 createEncryptionZone is superuser-only operation
>  TestReservedRawPaths.testAdminAccessOnly:262 access to /.reserved/raw is superuser-only operation
>  TestDFSShell.testSetXAttrPermission:2992 Returned should be 1 expected:<1> but was:<0>
>  TestWebHDFSXAttr>FSXAttrBaseTest.testUnreadableBySuperuserXAttr:1142->FSXAttrBaseTest.access$000:67->FSXAttrBaseTest.doTestUnreadableBySuperuserXAttr:1158->FSXAttrBaseTest.doTUBSXAInt:1213->FSXAttrBaseTest.verifyFileAccess:1241 open failed but expected it to succeed
>  TestWebHDFSXAttr>FSXAttrBaseTest.testRawXAttrs:1024 setXAttr should have thrown
>  TestWebHDFSXAttr>FSXAttrBaseTest.testListXAttrs:820 expected IOException
>  TestWebHDFSXAttr>FSXAttrBaseTest.testGetXAttrs:458 expected IOException
>  TestWebHdfsFileSystemContract>FileSystemContractBaseTest.testRenameDirectoryAsExistingDirectory:536 Renamed nested file1 exists
>  TestEncryptionZones.testGetEZAsNonSuperUser:581 expected AccessControlException
>  TestEncryptionZones.testBasicOperations:441 createEncryptionZone is superuser-only operation
>
> Tests in error:
>  TestRBWBlockInvalidation.testBlockInvalidationWhenRBWReplicaMissedInDN:122 >  ...
>  TestDataNodeVolumeFailure.tearDown:142->Object.wait:-2 >  test timed out after...
>  TestDataNodeVolumeFailureReporting.tearDown:109->Object.wait:-2 >  test timed ...
>  TestDataNodeVolumeFailureReporting.tearDown:109 >  test timed out after 120000...
>  TestDataNodeVolumeFailureReporting.tearDown:109 >  test timed out after 120000...
>  TestDataNodeVolumeFailureReporting.tearDown:109->Object.wait:-2 >  test timed ...
>  TestFileContextXAttr>FSXAttrBaseTest.testRemoveXAttrPermissions:656 > Remote N...
>  TestNameNodeMetadataConsistency.testGenerationStampInFuture:109->waitForNumBytes:161 > Timeout
>  TestNameNodeXAttr>FSXAttrBaseTest.testRemoveXAttrPermissions:656 > Remote No m...
>  TestFSImage.testCompression:96->setCompressCodec:102->testPersistHelper:108 > IO
>  TestDiskBalancerCommand.testRunMultipleCommandsUnderOneSetup > Remote File /sy...
>  TestDiskBalancer.testBalanceDataBetweenMultiplePairsOfVolumes:187 > IllegalArgument
>  TestDiskBalancerRPC.testMoveBlockAcrossVolume > Remote File /tmp.txt could onl...
>  TestDFSRSDefault10x4StripedOutputStream>TestDFSStripedOutputStream.setup:91 > DiskError
>  TestDFSRSDefault10x4StripedOutputStream>TestDFSStripedOutputStream.testFileSmallerThanOneStripe2:132->TestDFSStripedOutputStream.testOneFile:187 > Timeout
>  TestDFSRSDefault10x4StripedOutputStream>TestDFSStripedOutputStream.testFileMoreThanABlockGroup3:176->TestDFSStripedOutputStream.testOneFile:186 > IO
>  TestParallelRead.setupCluster:37->TestParallelReadUtil.setupCluster:71 > IO Ca...


>  TestParallelRead.teardownCluster:42->TestParallelReadUtil.teardownCluster:394 > NullPointer

that's a clear bug in the teardown logic; I'd run that standalone then file a JIRA on it...it's not checking for a field being set before invoking. Root cause is probably a failure in test setup tho


>  TestSaslDataTransfer.testNoSaslAndSecurePortsIgnored:175->startCluster:207 > FileNotFound
>  TestDatanodeLayoutUpgrade.testUpgradeToIdBasedLayout:52 > IO Timed out waiting...
>  TestDFSMkdirs.testMkdir:94 > DiskError Too many failed volumes - current valid...
>  TestWebHDFSXAttr>FSXAttrBaseTest.testRemoveXAttrPermissions:656 > IO No matchi...
>
> Tests run: 4997, Failures: 23, Errors: 22, Skipped: 150


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Loading...