[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1531 - Failure

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1531 - Failure

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1531/

2 tests failed.
FAILED:  org.apache.solr.uninverting.TestDocTermOrds.testTriggerUnInvertLimit

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space
        at __randomizedtesting.SeedInfo.seed([B873F742D5FBF37D:8BC1DF86D84C29CA]:0)
        at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:78)
        at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:51)
        at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
        at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
        at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
        at org.apache.lucene.codecs.lucene50.Lucene50PostingsWriter.finishTerm(Lucene50PostingsWriter.java:417)
        at org.apache.lucene.codecs.PushPostingsWriterBase.writeTerm(PushPostingsWriterBase.java:176)
        at org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter$TermsWriter.write(BlockTreeTermsWriter.java:865)
        at org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter.write(BlockTreeTermsWriter.java:344)
        at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:141)
        at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:110)
        at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:173)
        at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:463)
        at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:556)
        at org.apache.lucene.index.DocumentsWriter.postUpdate(DocumentsWriter.java:416)
        at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:517)
        at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1774)
        at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1466)
        at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:185)
        at org.apache.solr.uninverting.TestDocTermOrds.testTriggerUnInvertLimit(TestDocTermOrds.java:171)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)


FAILED:  org.apache.solr.uninverting.TestDocTermOrds.testNumericEncoded64

Error Message:
Captured an uncaught exception in thread: Thread[id=158876, name=Lucene Merge Thread #34, state=RUNNABLE, group=TGRP-TestDocTermOrds]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=158876, name=Lucene Merge Thread #34, state=RUNNABLE, group=TGRP-TestDocTermOrds]
        at __randomizedtesting.SeedInfo.seed([B873F742D5FBF37D:9461C646A1138FC4]:0)
Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError: Java heap space
        at __randomizedtesting.SeedInfo.seed([B873F742D5FBF37D]:0)
        at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:704)
        at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)
Caused by: java.lang.OutOfMemoryError: Java heap space
        at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:78)
        at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:51)
        at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
        at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
        at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
        at org.apache.lucene.store.RateLimitedIndexOutput.writeBytes(RateLimitedIndexOutput.java:73)
        at org.apache.lucene.store.DataOutput.copyBytes(DataOutput.java:278)
        at org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.write(Lucene50CompoundFormat.java:100)
        at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:5042)
        at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4545)
        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4142)
        at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:625)
        at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:662)




Build Log:
[...truncated 15579 lines...]
   [junit4] Suite: org.apache.solr.uninverting.TestDocTermOrds
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDocTermOrds -Dtests.method=testTriggerUnInvertLimit -Dtests.seed=B873F742D5FBF37D -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=hr-HR -Dtests.timezone=US/Pacific-New -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   57.8s J2 | TestDocTermOrds.testTriggerUnInvertLimit <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > at __randomizedtesting.SeedInfo.seed([B873F742D5FBF37D:8BC1DF86D84C29CA]:0)
   [junit4]    > at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:78)
   [junit4]    > at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:51)
   [junit4]    > at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
   [junit4]    > at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
   [junit4]    > at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
   [junit4]    > at org.apache.lucene.codecs.lucene50.Lucene50PostingsWriter.finishTerm(Lucene50PostingsWriter.java:417)
   [junit4]    > at org.apache.lucene.codecs.PushPostingsWriterBase.writeTerm(PushPostingsWriterBase.java:176)
   [junit4]    > at org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter$TermsWriter.write(BlockTreeTermsWriter.java:865)
   [junit4]    > at org.apache.lucene.codecs.blocktree.BlockTreeTermsWriter.write(BlockTreeTermsWriter.java:344)
   [junit4]    > at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.write(PerFieldPostingsFormat.java:141)
   [junit4]    > at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:110)
   [junit4]    > at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:173)
   [junit4]    > at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:463)
   [junit4]    > at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:556)
   [junit4]    > at org.apache.lucene.index.DocumentsWriter.postUpdate(DocumentsWriter.java:416)
   [junit4]    > at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:517)
   [junit4]    > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1774)
   [junit4]    > at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1466)
   [junit4]    > at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:185)
   [junit4]    > at org.apache.solr.uninverting.TestDocTermOrds.testTriggerUnInvertLimit(TestDocTermOrds.java:171)
   [junit4]   2> tra 15, 2018 4:55:53 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Lucene Merge Thread #34,5,TGRP-TestDocTermOrds]
   [junit4]   2> org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError: Java heap space
   [junit4]   2> at __randomizedtesting.SeedInfo.seed([B873F742D5FBF37D]:0)
   [junit4]   2> at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:704)
   [junit4]   2> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)
   [junit4]   2> Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]   2> at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:78)
   [junit4]   2> at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:51)
   [junit4]   2> at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
   [junit4]   2> at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
   [junit4]   2> at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
   [junit4]   2> at org.apache.lucene.store.RateLimitedIndexOutput.writeBytes(RateLimitedIndexOutput.java:73)
   [junit4]   2> at org.apache.lucene.store.DataOutput.copyBytes(DataOutput.java:278)
   [junit4]   2> at org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.write(Lucene50CompoundFormat.java:100)
   [junit4]   2> at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:5042)
   [junit4]   2> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4545)
   [junit4]   2> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4142)
   [junit4]   2> at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:625)
   [junit4]   2> at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:662)
   [junit4]   2>
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDocTermOrds -Dtests.method=testNumericEncoded64 -Dtests.seed=B873F742D5FBF37D -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=hr-HR -Dtests.timezone=US/Pacific-New -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.02s J2 | TestDocTermOrds.testNumericEncoded64 <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=158876, name=Lucene Merge Thread #34, state=RUNNABLE, group=TGRP-TestDocTermOrds]
   [junit4]    > at __randomizedtesting.SeedInfo.seed([B873F742D5FBF37D:9461C646A1138FC4]:0)
   [junit4]    > Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > at __randomizedtesting.SeedInfo.seed([B873F742D5FBF37D]:0)
   [junit4]    > at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:704)
   [junit4]    > at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)
   [junit4]    > Caused by: java.lang.OutOfMemoryError: Java heap space
   [junit4]    > at org.apache.lucene.store.RAMFile.newBuffer(RAMFile.java:78)
   [junit4]    > at org.apache.lucene.store.RAMFile.addBuffer(RAMFile.java:51)
   [junit4]    > at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:164)
   [junit4]    > at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:150)
   [junit4]    > at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:141)
   [junit4]    > at org.apache.lucene.store.RateLimitedIndexOutput.writeBytes(RateLimitedIndexOutput.java:73)
   [junit4]    > at org.apache.lucene.store.DataOutput.copyBytes(DataOutput.java:278)
   [junit4]    > at org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.write(Lucene50CompoundFormat.java:100)
   [junit4]    > at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:5042)
   [junit4]    > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4545)
   [junit4]    > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4142)
   [junit4]    > at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:625)
   [junit4]    > at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:662)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {field=Lucene50(blocksize=128), foo=Lucene50(blocksize=128), id=PostingsFormat(name=LuceneFixedGap)}, docValues:{}, maxPointsInLeafNode=1806, maxMBSortInHeap=6.392852599222817, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@36f1bbf7), locale=hr-HR, timezone=US/Pacific-New
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=125835216,total=477626368
   [junit4]   2> NOTE: All tests run in this JVM: [TestPolicyCloud, GraphQueryTest, TestSha256AuthenticationProvider, TestRawResponseWriter, BasicDistributedZk2Test, TestAnalyzeInfixSuggestions, SuggesterTSTTest, DisMaxRequestHandlerTest, TestStressUserVersions, DeleteShardTest, TestExclusionRuleCollectionAccess, SolrCmdDistributorTest, TestConfigsApi, NodeAddedTriggerIntegrationTest, SearchRateTriggerIntegrationTest, TimeZoneUtilsTest, TestCorePropertiesReload, NotRequiredUniqueKeyTest, TestJettySolrRunner, HighlighterMaxOffsetTest, TestGraphMLResponseWriter, TestPivotHelperCode, CopyFieldTest, TestBadConfig, TestCopyFieldCollectionResource, ResponseLogComponentTest, HdfsBasicDistributedZkTest, ConvertedLegacyTest, MultiSolrCloudTestCaseTest, TestSegmentSorting, BaseCdcrDistributedZkTest, TestLegacyTerms, VersionInfoTest, TestNumericRangeQuery64, TestLeaderInitiatedRecoveryThread, DistributedFacetPivotLargeTest, CoreAdminHandlerTest, TestJsonFacets, HdfsNNFailoverTest, SSLMigrationTest, TestEmbeddedSolrServerAdminHandler, TestDFRSimilarityFactory, AtomicUpdatesTest, StatelessScriptUpdateProcessorFactoryTest, IndexSchemaRuntimeFieldTest, SolrTestCaseJ4Test, LegacyCloudClusterPropTest, TestRestoreCore, TestManagedSynonymGraphFilterFactory, TestXmlQParserPlugin, PreAnalyzedFieldManagedSchemaCloudTest, HdfsRestartWhileUpdatingTest, DistanceFunctionTest, PropertiesRequestHandlerTest, TestRealTimeGet, CdcrReplicationDistributedZkTest, SolrGangliaReporterTest, TestGraphTermsQParserPlugin, MoreLikeThisHandlerTest, V2ApiIntegrationTest, TestLFUCache, TestRebalanceLeaders, TestSkipOverseerOperations, PreAnalyzedFieldTest, TestSolrConfigHandlerCloud, TestFastWriter, RankQueryTest, TestSolrCloudWithHadoopAuthPlugin, CursorMarkTest, ExecutePlanActionTest, MetricUtilsTest, SpellCheckComponentTest, TestPullReplicaErrorHandling, TimeRoutedAliasUpdateProcessorTest, CollectionTooManyReplicasTest, DateMathParserTest, TestTolerantUpdateProcessorRandomCloud, TestObjectReleaseTracker, ConcurrentCreateRoutedAliasTest, OverseerTaskQueueTest, PingRequestHandlerTest, CollectionPropsTest, FileBasedSpellCheckerTest, TestSolrXml, TestFieldCollectionResource, TestLMJelinekMercerSimilarityFactory, StatsReloadRaceTest, TestSubQueryTransformerCrossCore, TestInitParams, TestFiltering, TestUtils, DistribDocExpirationUpdateProcessorTest, TestDistribDocBasedVersion, TestStressLiveNodes, JSONWriterTest, TestShardHandlerFactory, TestAnalyzedSuggestions, ResponseHeaderTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestAuthorizationFramework, ZkStateWriterTest, CoreAdminCreateDiscoverTest, TestSolrCoreParser, RulesTest, ChangedSchemaMergeTest, TestDynamicLoading, TestClusterProperties, TestRangeQuery, TestBlobHandler, TriggerSetPropertiesIntegrationTest, DirectSolrSpellCheckerTest, TestReplicaProperties, TestPayloadCheckQParserPlugin, ShowFileRequestHandlerTest, TestCloudSchemaless, TestPointFields, HdfsTlogReplayBufferedWhileIndexingTest, NumericFieldsTest, DistributedDebugComponentTest, IndexSizeTriggerTest, URLClassifyProcessorTest, TestDFISimilarityFactory, DirectoryFactoryTest, LeaderElectionTest, ShardSplitTest, TestCollectionsAPIViaSolrCloudCluster, TestDistributedStatsComponentCardinality, TestLargeCluster, OutOfBoxZkACLAndCredentialsProvidersTest, DistribJoinFromCollectionTest, DocValuesNotIndexedTest, HealthCheckHandlerTest, HttpPartitionOnCommitTest, LeaderVoteWaitTimeoutTest, MultiThreadedOCPTest, OverseerModifyCollectionTest, OverseerRolesTest, OverseerStatusTest, PeerSyncReplicationTest, RecoveryAfterSoftCommitTest, RemoteQueryErrorTest, ReplaceNodeNoTargetTest, ReplicationFactorTest, RollingRestartTest, CollectionsAPIDistributedZkTest, ScheduledTriggerTest, TestDistribStateManager, TestExecutePlanAction, TestTriggerIntegration, CdcrBootstrapTest, CdcrVersionReplicationTest, HdfsChaosMonkeySafeLeaderTest, HdfsRecoveryZkTest, HdfsSyncSliceTest, TestDirectoryFactory, TestReloadAndDeleteDocs, TestSolrCoreSnapshots, RequestLoggingTest, SearchHandlerTest, TestCoreBackup, TestReqParamsAPI, TestSystemCollAutoCreate, DistributedFacetPivotWhiteBoxTest, FacetPivotSmallTest, InfixSuggestersTest, TestLegacyField, TestLegacyFieldReuse, TestLegacyNumericUtils, TestNumericTokenStream, JvmMetricsTest, MetricsConfigTest, SolrCoreMetricManagerTest, SolrMetricManagerTest, SolrMetricsIntegrationTest, SolrGraphiteReporterTest, SolrJmxReporterCloudTest, SolrJmxReporterTest, SolrCloudReportersTest, TestUnInvertedFieldException, TestMacros, SmileWriterTest, TestExportWriter, TestBulkSchemaAPI, DocValuesMissingTest, TestCollationFieldDocValues, TestManagedSchema, TestSortableTextField, AnalyticsMergeStrategyTest, TestReloadDeadlock, TestSearcherReuse, TestSimpleQParserPlugin, TestSolr4Spatial2, TestSolrFieldCacheBean, TestOrdValues, TestSortByMinMaxFunction, BlockJoinFacetDistribTest, BlockJoinFacetRandomTest, TestCloudNestedDocsSort, TestClassicSimilarityFactory, TestNonDefinedSimilarityFactory, TestPerFieldSimilarityWithDefaultOverride, TestDistribIDF, TestDocTermOrds]
   [junit4] Completed [769/802 (1!)] on J2 in 60.53s, 10 tests, 2 errors <<< FAILURES!

[...truncated 118 lines...]
   [junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/temp/junit4-J2-20180415_224610_4911415298189908880098.sysout
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/heapdumps/java_pid20173.hprof ...
   [junit4] Heap dump file created [456750629 bytes in 2.772 secs]
   [junit4] <<< JVM J2: EOF ----

[...truncated 9420 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:651: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid20173.hprof

Total time: 313 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1532 - Still Failing

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1532/

8 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.ltr.feature.TestExternalFeatures

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.ltr.feature.TestExternalFeatures:     1) Thread[id=126, name=qtp784058621-126, state=TIMED_WAITING, group=TGRP-TestExternalFeatures]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.ltr.feature.TestExternalFeatures:
   1) Thread[id=126, name=qtp784058621-126, state=TIMED_WAITING, group=TGRP-TestExternalFeatures]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([83E976A018F08D7C]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.ltr.feature.TestExternalFeatures

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=126, name=qtp784058621-126, state=TIMED_WAITING, group=TGRP-TestExternalFeatures]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=126, name=qtp784058621-126, state=TIMED_WAITING, group=TGRP-TestExternalFeatures]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([83E976A018F08D7C]:0)


FAILED:  org.apache.solr.uninverting.TestDocTermOrds.testEmptyIndex

Error Message:
GC overhead limit exceeded

Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded


FAILED:  org.apache.solr.uninverting.TestDocTermOrds.testActuallySingleValued

Error Message:
GC overhead limit exceeded

Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded


FAILED:  org.apache.solr.uninverting.TestDocTermOrds.testSortedTermsEnum

Error Message:
GC overhead limit exceeded

Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded


FAILED:  org.apache.solr.uninverting.TestDocTermOrds.testRandom

Error Message:
GC overhead limit exceeded

Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded


FAILED:  junit.framework.TestSuite.org.apache.solr.uninverting.TestDocTermOrds

Error Message:
GC overhead limit exceeded

Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded


FAILED:  org.apache.solr.uninverting.TestDocTermOrds.testNumericEncoded64

Error Message:
GC overhead limit exceeded

Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded




Build Log:
[...truncated 15649 lines...]
   [junit4] Suite: org.apache.solr.uninverting.TestDocTermOrds
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDocTermOrds -Dtests.method=testTriggerUnInvertLimit -Dtests.seed=74EC5E24D6A423CC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=no-NO -Dtests.timezone=Antarctica/Palmer -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDocTermOrds -Dtests.method=testEmptyIndex -Dtests.seed=74EC5E24D6A423CC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=no-NO -Dtests.timezone=Antarctica/Palmer -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   1.56s J0 | TestDocTermOrds.testEmptyIndex <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDocTermOrds -Dtests.method=testActuallySingleValued -Dtests.seed=74EC5E24D6A423CC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=no-NO -Dtests.timezone=Antarctica/Palmer -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   2.42s J0 | TestDocTermOrds.testActuallySingleValued <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDocTermOrds -Dtests.method=testSortedTermsEnum -Dtests.seed=74EC5E24D6A423CC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=no-NO -Dtests.timezone=Antarctica/Palmer -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   2.46s J0 | TestDocTermOrds.testSortedTermsEnum <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDocTermOrds -Dtests.method=testNumericEncoded64 -Dtests.seed=74EC5E24D6A423CC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=no-NO -Dtests.timezone=Antarctica/Palmer -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   2.26s J0 | TestDocTermOrds.testNumericEncoded64 <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDocTermOrds -Dtests.method=testRandom -Dtests.seed=74EC5E24D6A423CC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=no-NO -Dtests.timezone=Antarctica/Palmer -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   2.56s J0 | TestDocTermOrds.testRandom <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4]   2> NOTE: test params are: codec=SimpleText, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@17c014), locale=no-NO, timezone=Antarctica/Palmer
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=383835512,total=477626368
   [junit4]   2> NOTE: All tests run in this JVM: [PeerSyncWithIndexFingerprintCachingTest, MoveReplicaHDFSTest, RestartWhileUpdatingTest, BigEndianAscendingWordDeserializerTest, ExplicitHLLTest, OverseerRolesTest, SuggesterWFSTTest, TestIntervalFaceting, DistributedSuggestComponentTest, TestComponentsName, TestManagedSchemaAPI, TestHashQParserPlugin, ZkNodePropsTest, TestScoreJoinQPScore, ConjunctionSolrSpellCheckerTest, TokenizerChainTest, TestJsonFacetsWithNestedObjects, TestIBSimilarityFactory, SpellCheckCollatorTest, TestExecutePlanAction, MBeansHandlerTest, TestSimpleTextCodec, MetricsConfigTest, ChaosMonkeyNothingIsSafeTest, TestCSVLoader, SortSpecParsingTest, SolrSlf4jReporterTest, InfoHandlerTest, FullSolrCloudDistribCmdsTest, TestLuceneMatchVersion, CreateRoutedAliasTest, TestClusterStateProvider, TestCrossCoreJoin, DeleteStatusTest, TestOmitPositions, SynonymTokenizerTest, TestCloudNestedDocsSort, TestDelegationWithHadoopAuth, SuggestComponentContextFilterQueryTest, CursorPagingTest, TermVectorComponentDistributedTest, TestCloudRecovery, MultiThreadedOCPTest, DocumentAnalysisRequestHandlerTest, TestTlogReplica, CollectionStateFormat2Test, TestFieldSortValues, HdfsAutoAddReplicasIntegrationTest, TestReplicationHandlerBackup, ReplicaListTransformerTest, BadCopyFieldTest, SearchRateTriggerTest, OpenCloseCoreStressTest, UtilsToolTest, TestMacroExpander, TestStressCloudBlindAtomicUpdates, TestConfigReload, SolrIndexConfigTest, TestReloadDeadlock, OpenExchangeRatesOrgProviderTest, TestExportWriter, TestCloudJSONFacetJoinDomain, DistributedTermsComponentTest, TestSchemaNameResource, HdfsUnloadDistributedZkTest, CurrencyFieldTypeTest, PluginInfoTest, TestRTimerTree, SimpleFacetsTest, TemplateUpdateProcessorTest, TestDocSet, TestNodeAddedTrigger, ReplaceNodeNoTargetTest, TestShortCircuitedRequests, SchemaApiFailureTest, AliasIntegrationTest, DistributedFacetPivotWhiteBoxTest, ExitableDirectoryReaderTest, IndexBasedSpellCheckerTest, PeerSyncReplicationTest, TestSolrCloudWithDelegationTokens, TestBackupRepositoryFactory, DistributedFacetPivotLongTailTest, TestUnifiedSolrHighlighter, CleanupOldIndexTest, TestGroupingSearch, TestCollationField, TestSolrDeletionPolicy1, TriggerCooldownIntegrationTest, TestBlendedInfixSuggestions, TestIndexingPerformance, TestCoreAdminApis, ScheduledTriggerIntegrationTest, CloneFieldUpdateProcessorFactoryTest, UUIDFieldTest, SimpleMLTQParserTest, BadIndexSchemaTest, TestHdfsCloudBackupRestore, ExternalFileFieldSortTest, BinaryUpdateRequestHandlerTest, TestPseudoReturnFields, TestClusterStateMutator, FieldAnalysisRequestHandlerTest, MetricTriggerIntegrationTest, DistributedExpandComponentTest, TestJavabinTupleStreamParser, TestBulkSchemaConcurrent, TestCustomStream, TestExtendedDismaxParser, XmlUpdateRequestHandlerTest, DocValuesNotIndexedTest, DistributedQueryElevationComponentTest, TestDynamicFieldCollectionResource, EnumFieldTest, VMParamsZkACLAndCredentialsProvidersTest, OverriddenZkACLAndCredentialsProvidersTest, TestDistributedSearch, TestSolrConfigHandlerConcurrent, TestImplicitCoreProperties, TestExactStatsCache, LargeFieldTest, TestConfig, TestWriterPerf, TestDefaultStatsCache, TestPHPSerializedResponseWriter, TestRequestStatusCollectionAPI, TestDistributedMap, TestNoOpRegenerator, ConnectionReuseTest, HdfsRecoverLeaseTest, UpdateParamsTest, TestRawTransformer, BasicDistributedZkTest, SyncSliceTest, ShardRoutingTest, ClusterStateUpdateTest, ZkSolrClientTest, ZkCLITest, TestFaceting, DistributedSpellCheckComponentTest, TestReload, SolrCmdDistributorTest, ConvertedLegacyTest, TestSort, DirectUpdateHandlerTest, TestBadConfig, SoftAutoCommitTest, SpellPossibilityIteratorTest, TestCharFilters, TestXIncludeConfig, EchoParamsTest, TestDFRSimilarityFactory, TestLMDirichletSimilarityFactory, ResourceLoaderTest, URLClassifyProcessorTest, DateMathParserTest, SpellingQueryConverterTest, RAMDirectoryFactoryTest, TestLRUCache, TestDocumentBuilder, FileUtilsTest, TestRTGBase, TestHighlightDedupGrouping, TestTolerantSearch, TestEmbeddedSolrServerAdminHandler, TestEmbeddedSolrServerConstructors, AddReplicaTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, DistribJoinFromCollectionTest, DistributedVersionInfoTest, HttpPartitionOnCommitTest, LIRRollingUpdatesTest, LegacyCloudClusterPropTest, MigrateRouteKeyTest, MultiSolrCloudTestCaseTest, NodeMutatorTest, OverseerCollectionConfigSetProcessorTest, OverseerStatusTest, ReplaceNodeTest, RollingRestartTest, TestRequestForwarding, TestSSLRandomization, TestSolrCloudWithKerberosAlt, TestStressInPlaceUpdates, TestUtilizeNode, ZkShardTermsTest, AssignTest, CollectionReloadTest, CollectionsAPIAsyncDistributedZkTest, CustomCollectionTest, HdfsCollectionsAPIDistributedZkTest, MetricTriggerTest, NodeLostTriggerIntegrationTest, ScheduledTriggerTest, TestComputePlanAction, TestDistribStateManager, TestSimDistributedQueue, CdcrBidirectionalTest, HDFSCollectionsAPITest, HdfsChaosMonkeyNothingIsSafeTest, HdfsNNFailoverTest, HdfsTlogReplayBufferedWhileIndexingTest, HdfsWriteToMultipleCollectionsTest, ImplicitSnitchTest, RuleEngineTest, BlobRepositoryCloudTest, ConfigureRecoveryStrategyTest, CoreSorterTest, HdfsDirectoryFactoryTest, TestConfigSetImmutable, TestConfigSetProperties, TestConfigSets, TestDynamicLoading, TestInitParams, TestNRTOpen, TestReloadAndDeleteDocs, TestShardHandlerFactory, TestSolrConfigHandler, ResponseBuilderTest, TestCoreBackup, TestSQLHandler, TestNumericRangeQuery64, TestSubQueryTransformerDistrib, AnalyticsQueryTest, TestComplexPhraseLeadingWildcard, TestInitQParser, TestMissingGroups, TestXmlQParser, TestJsonFacets, TestZkAclsWithHadoopAuth, HttpSolrCallGetCoreTest, BufferStoreTest, TestDocTermOrds]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDocTermOrds -Dtests.seed=74EC5E24D6A423CC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=no-NO -Dtests.timezone=Antarctica/Palmer -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J0 | TestDocTermOrds (suite) <<<
   [junit4]    > Throwable #1: java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Completed [785/802 (1!)] on J0 in 252.15s, 10 tests, 6 errors <<< FAILURES!

[...truncated 46 lines...]
   [junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/temp/junit4-J0-20180416_153327_527822181018809972841.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/heapdumps/java_pid24166.hprof ...
   [junit4] Heap dump file created [467861673 bytes in 1.113 secs]
   [junit4] <<< JVM J0: EOF ----

   [junit4] JVM J0: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/temp/junit4-J0-20180416_153327_5273173350778028939000.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4]
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Lucene Merge Thread #207"
   [junit4] WARN: Event serializer exception. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J0: EOF ----

[...truncated 7683 lines...]
   [junit4] Suite: org.apache.solr.ltr.feature.TestExternalFeatures
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J2/temp/solr.ltr.feature.TestExternalFeatures_83E976A018F08D7C-001/init-core-data-001
   [junit4]   2> 40140 WARN  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=3 numCloses=3
   [junit4]   2> 40152 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 40154 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 40154 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 40155 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 40176 WARN  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/contrib/ltr/src/test-files/solr/models filtered by null to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/contrib/ltr/src/test-files/solr/models
   [junit4]   2> 40191 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 40222 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=example
   [junit4]   2> 40234 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.s.IndexSchema Loaded schema example/1.5 with uniqueid field id
   [junit4]   2> 40289 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7645a22f
   [junit4]   2> 40303 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7645a22f
   [junit4]   2> 40303 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7645a22f
   [junit4]   2> 40308 INFO  (coreLoadExecutor-84-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 40327 WARN  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/contrib/ltr/src/test-files/solr/models filtered by null to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/contrib/ltr/src/test-files/solr/models
   [junit4]   2> 40338 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 40366 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=example
   [junit4]   2> 40407 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema example/1.5 with uniqueid field id
   [junit4]   2> 40409 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/contrib/ltr/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 40410 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7645a22f
   [junit4]   2> 40410 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 40410 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/contrib/ltr/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J2/temp/solr.ltr.feature.TestExternalFeatures_83E976A018F08D7C-001/init-core-data-001/]
   [junit4]   2> 40523 WARN  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.c.RequestHandlers no default request handler is registered (either '/select' or 'standard')
   [junit4]   2> 40524 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 40524 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 40525 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
   [junit4]   2> 40525 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: if uncommited for 1000ms;
   [junit4]   2> 40528 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@389e6dad[collection1] main]
   [junit4]   2> 40528 WARN  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/contrib/ltr/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 40528 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.r.RestManager Registered ManagedResource impl org.apache.solr.ltr.store.rest.ManagedFeatureStore for path /schema/feature-store
   [junit4]   2> 40529 WARN  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.r.ManagedResource No stored data found for /schema/feature-store
   [junit4]   2> 40529 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.l.s.r.ManagedFeatureStore ------ managed feature ~ loading ------
   [junit4]   2> 40529 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.r.ManagedResource Notified 1 observers of /schema/feature-store
   [junit4]   2> 40529 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.r.RestManager Registered new managed resource /schema/feature-store
   [junit4]   2> 40529 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.r.RestManager Registered ManagedResource impl org.apache.solr.ltr.store.rest.ManagedModelStore for path /schema/model-store
   [junit4]   2> 40529 WARN  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.r.ManagedResource No stored data found for /schema/model-store
   [junit4]   2> 40529 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.l.s.r.ManagedModelStore INIT model store
   [junit4]   2> 40529 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.l.s.r.ManagedModelStore ------ managed models ~ loading ------
   [junit4]   2> 40529 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.r.ManagedResource Notified 1 observers of /schema/model-store
   [junit4]   2> 40529 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.r.RestManager Registered new managed resource /schema/model-store
   [junit4]   2> 40530 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 40530 INFO  (coreLoadExecutor-84-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1597922889723543552
   [junit4]   2> 40534 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 40539 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 40539 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 40540 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J2/temp/solr.ltr.feature.TestExternalFeatures_83E976A018F08D7C-001/tempDir-003/cores/core
   [junit4]   2> 40547 INFO  (searcherExecutor-85-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@389e6dad[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 40548 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 40550 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 40550 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 40550 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 40551 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1c7c6669{/solr,null,AVAILABLE}
   [junit4]   2> 40551 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@280b2a4c{HTTP/1.1,[http/1.1]}{127.0.0.1:34624}
   [junit4]   2> 40551 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.e.j.s.Server Started @40657ms
   [junit4]   2> 40551 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34624, configSetBaseDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J2/temp/solr.ltr.feature.TestExternalFeatures_83E976A018F08D7C-001/tempDir-001, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J2/temp/solr.ltr.feature.TestExternalFeatures_83E976A018F08D7C-001/tempDir-003/cores}
   [junit4]   2> 40554 ERROR (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 40559 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 40559 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 40559 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 40559 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 40559 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-04-16T17:00:19.556Z
   [junit4]   2> 40560 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J2/temp/solr.ltr.feature.TestExternalFeatures_83E976A018F08D7C-001/tempDir-001/solr.xml
   [junit4]   2> 40581 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7645a22f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 40694 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7645a22f
   [junit4]   2> 40710 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7645a22f
   [junit4]   2> 40711 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7645a22f
   [junit4]   2> 40718 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J2/temp/solr.ltr.feature.TestExternalFeatures_83E976A018F08D7C-001/tempDir-003/cores
   [junit4]   2> 40719 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 40733 INFO  (coreLoadExecutor-95-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 40749 WARN  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J2/temp/solr.ltr.feature.TestExternalFeatures_83E976A018F08D7C-001/tempDir-001/models filtered by null to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J2/temp/solr.ltr.feature.TestExternalFeatures_83E976A018F08D7C-001/tempDir-001/models
   [junit4]   2> 40752 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 40765 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=example
   [junit4]   2> 40813 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema example/1.5 with uniqueid field id
   [junit4]   2> 40827 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J2/temp/solr.ltr.feature.TestExternalFeatures_83E976A018F08D7C-001/tempDir-001/collection1, trusted=true
   [junit4]   2> 40828 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7645a22f
   [junit4]   2> 40828 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 40828 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J2/temp/solr.ltr.feature.TestExternalFeatures_83E976A018F08D7C-001/tempDir-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J2/temp/solr.ltr.feature.TestExternalFeatures_83E976A018F08D7C-001/tempDir-003/cores/core/data/]
   [junit4]   2> 40941 WARN  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.c.RequestHandlers no default request handler is registered (either '/select' or 'standard')
   [junit4]   2> 40942 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 40942 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 40943 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
   [junit4]   2> 40943 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: if uncommited for 1000ms;
   [junit4]   2> 40946 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@74678992[collection1] main]
   [junit4]   2> 40947 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J2/temp/solr.ltr.feature.TestExternalFeatures_83E976A018F08D7C-001/tempDir-001/collection1/conf
   [junit4]   2> 40947 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.r.RestManager Registered ManagedResource impl org.apache.solr.ltr.store.rest.ManagedFeatureStore for path /schema/feature-store
   [junit4]   2> 40947 WARN  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.r.ManagedResource No stored data found for /schema/feature-store
   [junit4]   2> 40948 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.l.s.r.ManagedFeatureStore ------ managed feature ~ loading ------
   [junit4]   2> 40948 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.r.ManagedResource Notified 1 observers of /schema/feature-store
   [junit4]   2> 40948 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.r.RestManager Registered new managed resource /schema/feature-store
   [junit4]   2> 40948 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.r.RestManager Registered ManagedResource impl org.apache.solr.ltr.store.rest.ManagedModelStore for path /schema/model-store
   [junit4]   2> 40948 WARN  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.r.ManagedResource No stored data found for /schema/model-store
   [junit4]   2> 40948 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.l.s.r.ManagedModelStore INIT model store
   [junit4]   2> 40948 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.l.s.r.ManagedModelStore ------ managed models ~ loading ------
   [junit4]   2> 40948 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.r.ManagedResource Notified 1 observers of /schema/model-store
   [junit4]   2> 40948 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.r.RestManager Registered new managed resource /schema/model-store
   [junit4]   2> 40949 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 40949 INFO  (coreLoadExecutor-95-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1597922890162896896
   [junit4]   2> 40953 INFO  (searcherExecutor-96-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@74678992[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 41478 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.SolrJettyTestBase Jetty Assigned Port#34624
   [junit4]   2> 41548 INFO  (qtp784058621-121) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<add><doc><field+name%3D"id">1</field><field+name%3D"title">w1</field><field+name%3D"description">w1</field><field+name%3D"popularity">1</field></doc></add>}{add=[1 (1597922890749050880)]} 0 46
   [junit4]   2> 41556 INFO  (qtp784058621-123) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<add><doc><field+name%3D"id">2</field><field+name%3D"title">w2</field><field+name%3D"description">w2</field><field+name%3D"popularity">2</field></doc></add>}{add=[2 (1597922890797285376)]} 0 2
   [junit4]   2> 41559 INFO  (qtp784058621-125) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<add><doc><field+name%3D"id">3</field><field+name%3D"title">w3</field><field+name%3D"description">w3</field><field+name%3D"popularity">3</field></doc></add>}{add=[3 (1597922890801479680)]} 0 1
   [junit4]   2> 41581 INFO  (qtp784058621-123) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<add><doc><field+name%3D"id">4</field><field+name%3D"title">w4</field><field+name%3D"description">w4</field><field+name%3D"popularity">4</field></doc></add>}{add=[4 (1597922890824548352)]} 0 17
   [junit4]   2> 41585 INFO  (qtp784058621-125) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<add><doc><field+name%3D"id">5</field><field+name%3D"title">w5</field><field+name%3D"description">w5</field><field+name%3D"popularity">5</field></doc></add>}{add=[5 (1597922890828742656)]} 0 1
   [junit4]   2> 41597 INFO  (qtp784058621-123) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1597922890842374144,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 41597 INFO  (qtp784058621-123) [    x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1a493621 commitCommandVersion:1597922890842374144
   [junit4]   2> 41614 INFO  (qtp784058621-123) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@31d26759[collection1] main]
   [junit4]   2> 41617 INFO  (searcherExecutor-96-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@31d26759[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C5)))}
   [junit4]   2> 41633 INFO  (qtp784058621-123) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 41634 INFO  (qtp784058621-123) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<commit/>}{commit=} 0 37
   [junit4]   2> 41637 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.l.TestRerankBase send
   [junit4]   2> [ {
   [junit4]   2>     "name" : "matchedTitle",
   [junit4]   2>     "class" : "org.apache.solr.ltr.feature.SolrFeature",
   [junit4]   2>     "params" : {
   [junit4]   2>         "q" : "{!terms f=title}${user_query}"
   [junit4]   2>     }
   [junit4]   2> }, {
   [junit4]   2>     "name" : "docPopularity",
   [junit4]   2>     "class" : "org.apache.solr.ltr.feature.FieldValueFeature",
   [junit4]   2>     "store": "fstore2",
   [junit4]   2>     "params" : {
   [junit4]   2>         "field" : "popularity"
   [junit4]   2>     }
   [junit4]   2> }, {
   [junit4]   2>     "name":"originalScore",
   [junit4]   2>     "class":"org.apache.solr.ltr.feature.OriginalScoreFeature",
   [junit4]   2>     "store": "fstore2",
   [junit4]   2>     "params":{}
   [junit4]   2> }, {
   [junit4]   2>     "name" : "occurrences",
   [junit4]   2>     "class" : "org.apache.solr.ltr.feature.ValueFeature",
   [junit4]   2>     "store": "fstore3",
   [junit4]   2>     "params" : {
   [junit4]   2>         "value" : "${myOcc}",
   [junit4]   2>         "required" : false
   [junit4]   2>     }
   [junit4]   2> }, {
   [junit4]   2>     "name":"originalScore",
   [junit4]   2>     "class":"org.apache.solr.ltr.feature.OriginalScoreFeature",
   [junit4]   2>     "store": "fstore3",
   [junit4]   2>     "params":{}
   [junit4]   2> }, {
   [junit4]   2>     "name" : "popularity",
   [junit4]   2>     "class" : "org.apache.solr.ltr.feature.ValueFeature",
   [junit4]   2>     "store": "fstore4",
   [junit4]   2>     "params" : {
   [junit4]   2>         "value" : "${myPop}",
   [junit4]   2>         "required" : true
   [junit4]   2>     }
   [junit4]   2> }, {
   [junit4]   2>     "name":"originalScore",
   [junit4]   2>     "class":"org.apache.solr.ltr.feature.OriginalScoreFeature",
   [junit4]   2>     "store": "fstore4",
   [junit4]   2>     "params":{}
   [junit4]   2> }, {
   [junit4]   2>     "name" : "titlePhraseMatch",
   [junit4]   2>     "class" : "org.apache.solr.ltr.feature.SolrFeature",
   [junit4]   2>     "params" : {
   [junit4]   2>         "q" : "{!field f=title}${user_query}"
   [junit4]   2>     }
   [junit4]   2> }, {
   [junit4]   2>     "name" : "titlePhrasesMatch",
   [junit4]   2>     "class" : "org.apache.solr.ltr.feature.SolrFeature",
   [junit4]   2>     "params" : {
   [junit4]   2>         "fq" : [ "{!field f=title}${userTitlePhrase1}", "{!field f=title}${userTitlePhrase2}"]
   [junit4]   2>     }
   [junit4]   2> } ]
   [junit4]   2>
   [junit4]   2> 41640 INFO  (qtp784058621-125) [    x:collection1] o.e.j.s.h.C.solr SolrSchemaRestApi: [Restlet] ServerServlet: component class is null
   [junit4]   2> 41642 INFO  (qtp784058621-125) [    x:collection1] o.e.j.s.h.C.solr SolrSchemaRestApi: [Restlet] Attaching application: org.apache.solr.rest.SolrSchemaRestApi@5bbb9a9d to URI: /solr/schema
   [junit4]   2> ápr. 16, 2018 2:00:20 DU org.restlet.Application start
   [junit4]   2> INFO: Starting org.apache.solr.rest.SolrSchemaRestApi application
   [junit4]   2> 41723 INFO  (qtp784058621-125) [    x:collection1] o.a.s.r.SolrSchemaRestApi createInboundRoot started for /schema
   [junit4]   2> 41725 INFO  (qtp784058621-125) [    x:collection1] o.a.s.r.RestManager Attached managed resource at path: /feature-store
   [junit4]   2> 41726 INFO  (qtp784058621-125) [    x:collection1] o.a.s.r.RestManager Attached managed resource at path: /managed
   [junit4]   2> 41726 INFO  (qtp784058621-125) [    x:collection1] o.a.s.r.RestManager Attached managed resource at path: /model-store
   [junit4]   2> 41726 INFO  (qtp784058621-125) [    x:collection1] o.a.s.r.RestManager Attached 3 ManagedResource endpoints to Restlet router: /schema
   [junit4]   2> 41726 INFO  (qtp784058621-125) [    x:collection1] o.a.s.r.SolrSchemaRestApi createInboundRoot complete for /schema
   [junit4]   2> 41740 INFO  (qtp784058621-125) [    x:collection1] o.a.s.r.RestManager Found ManagedResource [org.apache.solr.ltr.store.rest.ManagedFeatureStore@18309245] for /schema/feature-store
   [junit4]   2> 41741 INFO  (qtp784058621-125) [    x:collection1] o.a.s.r.ManagedResource Processing update to /schema/feature-store: [{name=matchedTitle, class=org.apache.solr.ltr.feature.SolrFeature, params={q={!terms f=title}${user_query}}}, {name=docPopularity, class=org.apache.solr.ltr.feature.FieldValueFeature, store=fstore2, params={field=popularity}}, {name=originalScore, class=org.apache.solr.ltr.feature.OriginalScoreFeature, store=fstore2, params={}}, {name=occurrences, class=org.apache.solr.ltr.feature.ValueFeature, store=fstore3, params={value=${myOcc}, required=false}}, {name=originalScore, class=org.apache.solr.ltr.feature.OriginalScoreFeature, store=fstore3, params={}}, {name=popularity, class=org.apache.solr.ltr.feature.ValueFeature, store=fstore4, params={value=${myPop}, required=true}}, {name=originalScore, class=org.apache.solr.ltr.feature.OriginalScoreFeature, store=fstore4, params={}}, {name=titlePhraseMatch, class=org.apache.solr.ltr.feature.SolrFeature, params={q={!field f=title}${user_query}}}, {name=titlePhrasesMatch, class=org.apache.solr.ltr.feature.SolrFeature, params={fq=[{!field f=title}${userTitlePhrase1}, {!field f=title}${userTitlePhrase2}]}}] is a java.util.ArrayList
   [junit4]   2> 41742 INFO  (qtp784058621-125) [    x:collection1] o.a.s.l.s.r.ManagedFeatureStore register feature based on {name=matchedTitle, class=org.apache.solr.ltr.feature.SolrFeature, params={q={!terms f=title}${user_query}}}
   [junit4]   2> 41757 INFO  (qtp784058621-125) [    x:collection1] o.a.s.l.s.r.ManagedFeatureStore register feature based on {name=docPopularity, class=org.apache.solr.ltr.feature.FieldValueFeature, store=fstore2, params={field=popularity}}
   [junit4]   2> 41772 INFO  (qtp784058621-125) [    x:collection1] o.a.s.l.s.r.ManagedFeatureStore register feature based on {name=originalScore, class=org.apache.solr.ltr.feature.OriginalScoreFeature, store=fstore2, params={}}
   [junit4]   2> 41774 INFO  (qtp784058621-125) [    x:collection1] o.a.s.l.s.r.ManagedFeatureStore register feature based on {name=occurrences, class=org.apache.solr.ltr.feature.ValueFeature, store=fstore3, params={value=${myOcc}, required=false}}
   [junit4]   2> 41788 INFO  (qtp784058621-125) [    x:collection1] o.a.s.l.s.r.ManagedFeatureStore register feature based on {name=originalScore, class=org.apache.solr.ltr.feature.OriginalScoreFeature, store=fstore3, params={}}
   [junit4]   2> 41788 INFO  (qtp784058621-125) [    x:collection1] o.a.s.l.s.r.ManagedFeatureStore register feature based on {name=popularity, class=org.apache.solr.ltr.feature.ValueFeature, store=fstore4, params={value=${myPop}, required=true}}
   [junit4]   2> 41791 INFO  (qtp784058621-125) [    x:collection1] o.a.s.l.s.r.ManagedFeatureStore register feature based on {name=originalScore, class=org.apache.solr.ltr.feature.OriginalScoreFeature, store=fstore4, params={}}
   [junit4]   2> 41791 INFO  (qtp784058621-125) [    x:collection1] o.a.s.l.s.r.ManagedFeatureStore register feature based on {name=titlePhraseMatch, class=org.apache.solr.ltr.feature.SolrFeature, params={q={!field f=title}${user_query}}}
   [junit4]   2> 41796 INFO  (qtp784058621-125) [    x:collection1] o.a.s.l.s.r.ManagedFeatureStore register feature based on {name=titlePhrasesMatch, class=org.apache.solr.ltr.feature.SolrFeature, params={fq=[{!field f=title}${userTitlePhrase1}, {!field f=title}${userTitlePhrase2}]}}
   [junit4]   2> 41800 INFO  (qtp784058621-125) [    x:collection1] o.a.s.r.ManagedResourceStorage Saved JSON object to path _schema_feature-store.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J2/temp/solr.ltr.feature.TestExternalFeatures_83E976A018F08D7C-001/tempDir-001/collection1/conf
   [junit4]   2> 41800 INFO  (qtp784058621-125) [    x:collection1] o.a.s.r.RestManager [collection1]  webapp=/solr path=/schema/feature-store params={indent=on&wt=json} status=0 QTime=160
   [junit4]   2> ápr. 16, 2018 2:00:20 DU org.restlet.engine.log.LogFilter afterHandle
   [junit4]   2> INFO: 2018-04-16 14:00:20 127.0.0.1 - 127.0.0.1 34624 PUT /solr/schema/feature-store wt=json&indent=on 200 - 1516 158 http://127.0.0.1:34624        Apache-HttpClient/4.5.3 (Java/1.8.0_152) -
   [junit4]   2> 41807 INFO  (qtp784058621-125) [    x:collection1] o.a.s.r.RestManager Found ManagedResource [ManagedModelStore [store=ModelStore [availableModels=[]], featureStores=org.apache.solr.ltr.store.rest.ManagedFeatureStore@18309245]] for /schema/model-store
   [junit4]   2> 41816 INFO  (qtp784058621-125) [    x:collection1] o.a.s.r.ManagedResource Processing update to /schema/model-store: {class=org.apache.solr.ltr.model.LinearModel, name=externalmodel, features=[{name=titlePhraseMatch}], params={weights={titlePhraseMatch=0.555}}} is a java.util.LinkedHashMap
   [junit4]   2> 41831 INFO  (qtp784058621-125) [    x:collection1] o.a.s.l.s.r.ManagedModelStore adding model externalmodel
   [junit4]   2> 41834 INFO  (qtp784058621-125) [    x:collection1] o.a.s.r.ManagedResourceStorage Saved JSON object to path _schema_model-store.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J2/temp/solr.ltr.feature.TestExternalFeatures_83E976A018F08D7C-001/tempDir-001/collection1/conf
   [junit4]   2> 41834 INFO  (qtp784058621-125) [    x:collection1] o.a.s.r.RestManager [collection1]  webapp=/solr path=/schema/model-store params={indent=on&wt=json} status=0 QTime=28
   [junit4]   2> ápr. 16, 2018 2:00:20 DU org.restlet.engine.log.LogFilter afterHandle
   [junit4]   2> INFO: 2018-04-16 14:00:20 127.0.0.1 - 127.0.0.1 34624 PUT /solr/schema/model-store wt=json&indent=on 200 - 237 29 http://127.0.0.1:34624        Apache-HttpClient/4.5.3 (Java/1.8.0_152) -
   [junit4]   2> 41847 INFO  (TEST-TestExternalFeatures.featureExtraction_valueFeatureRequiredInFq_shouldThrowException-seed#[83E976A018F08D7C]) [    ] o.a.s.SolrTestCaseJ4 ###Starting featureExtraction_valueFeatureRequiredInFq_shouldThrowException
   [junit4]   2> 41849 INFO  (qtp784058621-123) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=score,features:[fv+efi.user_query%3Duq+userTitlePhrase1%3Dutpp]&rows=1&wt=json} hits=5 status=0 QTime=0
   [junit4]   2> 41851 ERROR (qtp784058621-123) [    x:collection1] o.a.s.s.HttpSolrCall null:java.lang.RuntimeException: Exception from createWeight for SolrFeature [name=titlePhrasesMatch, params={fq=[{!field f=title}${userTitlePhrase1}, {!field f=title}${userTitlePhrase2}]}] SolrFeatureWeight requires efi parameter that was not passed in request.
   [junit4]   2> at org.apache.solr.ltr.LTRScoringQuery.createWeights(LTRScoringQuery.java:242)
   [junit4]   2> at org.apache.solr.ltr.LTRScoringQuery.createWeight(LTRScoringQuery.java:209)
   [junit4]   2> at org.apache.solr.ltr.response.transform.LTRFeatureLoggerTransformerFactory$FeatureTransformer.setContext(LTRFeatureLoggerTransformerFactory.java:246)
   [junit4]   2> at org.apache.solr.response.transform.DocTransformers.setContext(DocTransformers.java:69)
   [junit4]   2> at org.apache.solr.response.DocsStreamer.<init>(DocsStreamer.java:82)
   [junit4]   2> at org.apache.solr.response.ResultContext.getProcessedDocuments(ResultContext.java:55)
   [junit4]   2> at org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:270)
   [junit4]   2> at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:162)
   [junit4]   2> at org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:210)
   [junit4]   2> at org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:326)
   [junit4]   2> at org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:121)
   [junit4]   2> at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:72)
   [junit4]   2> at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:789)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:393)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:339)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
   [junit4]   2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
   [junit4]   2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
   [junit4]   2> at org.eclipse.jetty.server.Server.handle(Server.java:530)
   [junit4]   2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
   [junit4]   2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
   [junit4]   2> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
   [junit4]   2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: org.apache.solr.ltr.feature.FeatureException: SolrFeatureWeight requires efi parameter that was not passed in request.
   [junit4]   2> at org.apache.solr.ltr.feature.SolrFeature$SolrFeatureWeight.<init>(SolrFeature.java:162)
   [junit4]   2> at org.apache.solr.ltr.feature.SolrFeature.createWeight(SolrFeature.java:114)
   [junit4]   2> at org.apache.solr.ltr.LTRScoringQuery.createWeights(LTRScoringQuery.java:238)
   [junit4]   2> ... 45 more
   [junit4]   2>
   [junit4]   2> 41854 INFO  (TEST-TestExternalFeatures.featureExtraction_valueFeatureRequiredInFq_shouldThrowException-seed#[83E976A018F08D7C]) [    ] o.a.s.SolrTestCaseJ4 ###Ending featureExtraction_valueFeatureRequiredInFq_shouldThrowException
   [junit4]   2> 41883 INFO  (TEST-TestExternalFeatures.featureExtraction_valueFeatureRequired_shouldThrowException-seed#[83E976A018F08D7C]) [    ] o.a.s.SolrTestCaseJ4 ###Starting featureExtraction_valueFeatureRequired_shouldThrowException
   [junit4]   2> 41886 INFO  (qtp784058621-126) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=fvalias:[fv+store%3Dfstore4]&rows=1&wt=json} hits=5 status=0 QTime=2
   [junit4]   2> 41887 ERROR (qtp784058621-126) [    x:collection1] o.a.s.s.HttpSolrCall null:java.lang.RuntimeException: Exception from createWeight for ValueFeature [name=popularity, params={value=${myPop}, required=true}] ValueFeatureWeight requires efi parameter that was not passed in request.
   [junit4]   2> at org.apache.solr.ltr.LTRScoringQuery.createWeights(LTRScoringQuery.java:242)
   [junit4]   2> at org.apache.solr.ltr.LTRScoringQuery.createWeight(LTRScoringQuery.java:209)
   [junit4]   2> at org.apache.solr.ltr.response.transform.LTRFeatureLoggerTransformerFactory$FeatureTransformer.setContext(LTRFeatureLoggerTransformerFactory.java:246)
   [junit4]   2> at org.apache.solr.response.DocsStreamer.<init>(DocsStreamer.java:82)
   [junit4]   2> at org.apache.solr.response.ResultContext.getProcessedDocuments(ResultContext.java:55)
   [junit4]   2> at org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:270)
   [junit4]   2> at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:162)
   [junit4]   2> at org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:210)
   [junit4]   2> at org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:326)
   [junit4]   2> at org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:121)
   [junit4]   2> at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:72)
   [junit4]   2> at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:789)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:393)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:339)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
   [junit4]   2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
   [junit4]   2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
   [junit4]   2> at org.eclipse.jetty.server.Server.handle(Server.java:530)
   [junit4]   2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
   [junit4]   2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
   [junit4]   2> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
   [junit4]   2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: org.apache.solr.ltr.feature.FeatureException: ValueFeatureWeight requires efi parameter that was not passed in request.
   [junit4]   2> at org.apache.solr.ltr.feature.ValueFeature$ValueFeatureWeight.<init>(ValueFeature.java:123)
   [junit4]   2> at org.apache.solr.ltr.feature.ValueFeature.createWeight(ValueFeature.java:108)
   [junit4]   2> at org.apache.solr.ltr.LTRScoringQuery.createWeights(LTRScoringQuery.java:238)
   [junit4]   2> ... 44 more
   [junit4]   2>
   [junit4]   2> 41893 INFO  (TEST-TestExternalFeatures.featureExtraction_valueFeatureRequired_shouldThrowException-seed#[83E976A018F08D7C]) [    ] o.a.s.SolrTestCaseJ4 ###Ending featureExtraction_valueFeatureRequired_shouldThrowException
   [junit4]   2> 41904 INFO  (TEST-TestExternalFeatures.testFeaturesUseStopwordQueryReturnEmptyFeatureVector-seed#[83E976A018F08D7C]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testFeaturesUseStopwordQueryReturnEmptyFeatureVector
   [junit4]   2> 41909 INFO  (qtp784058621-123) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=*,score,fv:[fv]&rows=1&wt=json&rq={!ltr+reRankDocs%3D10+model%3Dexternalmodel+efi.user_query%3D'a'+efi.userTitlePhrase1%3D'b'+efi.userTitlePhrase2%3D'c'}} hits=5 status=0 QTime=3
   [junit4]   2> 41912 INFO  (TEST-TestExternalFeatures.testFeaturesUseStopwordQueryReturnEmptyFeatureVector-seed#[83E976A018F08D7C]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testFeaturesUseStopwordQueryReturnEmptyFeatureVector
   [junit4]   2> 41926 INFO  (TEST-TestExternalFeatures.featureExtraction_valueFeatureImplicitlyNotRequired_shouldNotScoreFeature-seed#[83E976A018F08D7C]) [    ] o.a.s.SolrTestCaseJ4 ###Starting featureExtraction_valueFeatureImplicitlyNotRequired_shouldNotScoreFeature
   [junit4]   2> 41928 INFO  (qtp784058621-125) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=fvalias:[fv+store%3Dfstore3]&rows=1&wt=json} hits=5 status=0 QTime=0
   [junit4]   2> 41933 INFO  (TEST-TestExternalFeatures.featureExtraction_valueFeatureImplicitlyNotRequired_shouldNotScoreFeature-seed#[83E976A018F08D7C]) [    ] o.a.s.SolrTestCaseJ4 ###Ending featureExtraction_valueFeatureImplicitlyNotRequired_shouldNotScoreFeature
   [junit4]   2> 42035 INFO  (TEST-TestExternalFeatures.testEfiInTransformerShouldNotChangeOrderOfRerankedResults-seed#[83E976A018F08D7C]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testEfiInTransformerShouldNotChangeOrderOfRerankedResults
   [junit4]   2> 42038 INFO  (qtp784058621-123) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=*,score&rows=3&wt=json} hits=5 status=0 QTime=0
   [junit4]   2> 42044 INFO  (qtp784058621-125) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=*,score&rows=3&wt=json} hits=5 status=0 QTime=0
   [junit4]   2> 42055 INFO  (qtp784058621-123) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=*,score&rows=3&wt=json} hits=5 status=0 QTime=0
   [junit4]   2> 42060 INFO  (qtp784058621-125) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=*,score&rows=3&wt=json} hits=5 status=0 QTime=0
   [junit4]   2> 42069 INFO  (qtp784058621-123) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=*,score&rows=3&wt=json} hits=5 status=0 QTime=0
   [junit4]   2> 42071 INFO  (qtp784058621-125) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=*,score&rows=3&wt=json} hits=5 status=0 QTime=0
   [junit4]   2> 42075 INFO  (qtp784058621-123) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=*,score,[fv]&rows=3&wt=json&rq={!ltr+reRankDocs%3D10+model%3Dexternalmodel+efi.user_query%3Dw3+efi.userTitlePhrase1%3Dw4+efi.userTitlePhrase2%3Dw5}} hits=5 status=0 QTime=2
   [junit4]   2> 42079 INFO  (qtp784058621-125) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=*,score,[fv]&rows=3&wt=json&rq={!ltr+reRankDocs%3D10+model%3Dexternalmodel+efi.user_query%3Dw3+efi.userTitlePhrase1%3Dw4+efi.userTitlePhrase2%3Dw5}} hits=5 status=0 QTime=1
   [junit4]   2> 42084 INFO  (qtp784058621-123) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=*,score,[fv]&rows=3&wt=json&rq={!ltr+reRankDocs%3D10+model%3Dexternalmodel+efi.user_query%3Dw3+efi.userTitlePhrase1%3Dw4+efi.userTitlePhrase2%3Dw5}} hits=5 status=0 QTime=1
   [junit4]   2> 42088 INFO  (qtp784058621-125) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=*,score,[fv]&rows=3&wt=json&rq={!ltr+reRankDocs%3D10+model%3Dexternalmodel+efi.user_query%3Dw3+efi.userTitlePhrase1%3Dw4+efi.userTitlePhrase2%3Dw5}} hits=5 status=0 QTime=1
   [junit4]   2> 42107 INFO  (qtp784058621-123) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=*,score,[fv+efi.user_query%3Dw2+efi.userTitlePhrase1%3Dw4+efi.userTitlePhrase2%3Dw5]&rows=3&wt=json&rq={!ltr+reRankDocs%3D10+model%3Dexternalmodel+efi.user_query%3Dw3+efi.userTitlePhrase1%3Dw4+efi.userTitlePhrase2%3Dw5}} hits=5 status=0 QTime=1
   [junit4]   2> 42386 INFO  (qtp784058621-125) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=*,score,[fv+efi.user_query%3Dw2+efi.userTitlePhrase1%3Dw4+efi.userTitlePhrase2%3Dw5]&rows=3&wt=json&rq={!ltr+reRankDocs%3D10+model%3Dexternalmodel+efi.user_query%3Dw3+efi.userTitlePhrase1%3Dw4+efi.userTitlePhrase2%3Dw5}} hits=5 status=0 QTime=268
   [junit4]   2> 42401 INFO  (qtp784058621-125) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=*,score,[fv+efi.user_query%3Dw2+efi.userTitlePhrase1%3Dw4+efi.userTitlePhrase2%3Dw5]&rows=3&wt=json&rq={!ltr+reRankDocs%3D10+model%3Dexternalmodel+efi.user_query%3Dw3+efi.userTitlePhrase1%3Dw4+efi.userTitlePhrase2%3Dw5}} hits=5 status=0 QTime=1
   [junit4]   2> 42404 INFO  (qtp784058621-125) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=*,score,[fv+efi.user_query%3Dw2+efi.userTitlePhrase1%3Dw4+efi.userTitlePhrase2%3Dw5]&rows=3&wt=json&rq={!ltr+reRankDocs%3D10+model%3Dexternalmodel+efi.user_query%3Dw3+efi.userTitlePhrase1%3Dw4+efi.userTitlePhrase2%3Dw5}} hits=5 status=0 QTime=1
   [junit4]   2> 42406 INFO  (TEST-TestExternalFeatures.testEfiInTransformerShouldNotChangeOrderOfRerankedResults-seed#[83E976A018F08D7C]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testEfiInTransformerShouldNotChangeOrderOfRerankedResults
   [junit4]   2> 42418 INFO  (TEST-TestExternalFeatures.testEfiFeatureExtraction-seed#[83E976A018F08D7C]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testEfiFeatureExtraction
   [junit4]   2> 42422 INFO  (qtp784058621-125) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=[fv]&rows=1&wt=json} hits=5 status=0 QTime=1
   [junit4]   2> 42425 ERROR (qtp784058621-125) [    x:collection1] o.a.s.s.HttpSolrCall null:java.lang.RuntimeException: Exception from createWeight for SolrFeature [name=matchedTitle, params={q={!terms f=title}${user_query}}] SolrFeatureWeight requires efi parameter that was not passed in request.
   [junit4]   2> at org.apache.solr.ltr.LTRScoringQuery.createWeights(LTRScoringQuery.java:242)
   [junit4]   2> at org.apache.solr.ltr.LTRScoringQuery.createWeight(LTRScoringQuery.java:209)
   [junit4]   2> at org.apache.solr.ltr.response.transform.LTRFeatureLoggerTransformerFactory$FeatureTransformer.setContext(LTRFeatureLoggerTransformerFactory.java:246)
   [junit4]   2> at org.apache.solr.response.DocsStreamer.<init>(DocsStreamer.java:82)
   [junit4]   2> at org.apache.solr.response.ResultContext.getProcessedDocuments(ResultContext.java:55)
   [junit4]   2> at org.apache.solr.response.TextResponseWriter.writeDocuments(TextResponseWriter.java:270)
   [junit4]   2> at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:162)
   [junit4]   2> at org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:210)
   [junit4]   2> at org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:326)
   [junit4]   2> at org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:121)
   [junit4]   2> at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:72)
   [junit4]   2> at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:789)
   [junit4]   2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:393)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:339)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
   [junit4]   2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
   [junit4]   2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
   [junit4]   2> at org.eclipse.jetty.server.Server.handle(Server.java:530)
   [junit4]   2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
   [junit4]   2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
   [junit4]   2> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
   [junit4]   2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: org.apache.solr.ltr.feature.FeatureException: SolrFeatureWeight requires efi parameter that was not passed in request.
   [junit4]   2> at org.apache.solr.ltr.feature.SolrFeature$SolrFeatureWeight.<init>(SolrFeature.java:146)
   [junit4]   2> at org.apache.solr.ltr.feature.SolrFeature.createWeight(SolrFeature.java:114)
   [junit4]   2> at org.apache.solr.ltr.LTRScoringQuery.createWeights(LTRScoringQuery.java:238)
   [junit4]   2> ... 44 more
   [junit4]   2>
   [junit4]   2> 42428 INFO  (qtp784058621-125) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=score,fvalias:[fv+store%3Dfstore3+efi.myOcc%3D2.3]&rows=1&wt=json} hits=5 status=0 QTime=0
   [junit4]   2> 42431 INFO  (qtp784058621-125) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=score,fvalias:[fv+store%3Dfstore3+efi.myOcc%3D2.3]&rows=1&wt=json&rq={!ltr+reRankDocs%3D10+model%3Dexternalmodel+efi.user_query%3Dw3}} hits=5 status=0 QTime=0
   [junit4]   2> 42432 INFO  (TEST-TestExternalFeatures.testEfiFeatureExtraction-seed#[83E976A018F08D7C]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testEfiFeatureExtraction
   [junit4]   2> 42439 INFO  (TEST-TestExternalFeatures.featureExtraction_valueFeatureExplicitlyNotRequired_shouldNotScoreFeature-seed#[83E976A018F08D7C]) [    ] o.a.s.SolrTestCaseJ4 ###Starting featureExtraction_valueFeatureExplicitlyNotRequired_shouldNotScoreFeature
   [junit4]   2> 42483 INFO  (qtp784058621-125) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=fvalias:[fv+store%3Dfstore3]&rows=1&wt=json} hits=5 status=0 QTime=0
   [junit4]   2> 42485 INFO  (TEST-TestExternalFeatures.featureExtraction_valueFeatureExplicitlyNotRequired_shouldNotScoreFeature-seed#[83E976A018F08D7C]) [    ] o.a.s.SolrTestCaseJ4 ###Ending featureExtraction_valueFeatureExplicitlyNotRequired_shouldNotScoreFeature
   [junit4]   2> 42490 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@280b2a4c{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 42495 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=464665807
   [junit4]   2> 42495 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 42495 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@72818452: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@3fde3b8a
   [junit4]   2> 42517 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 42517 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6603faed: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@599fc216
   [junit4]   2> 42522 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 42522 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2002a407: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@5d8f1de8
   [junit4]   2> 42531 INFO  (coreCloseExecutor-101-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@5638523a
   [junit4]   2> 42531 INFO  (coreCloseExecutor-101-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=5638523a
   [junit4]   2> 42532 INFO  (coreCloseExecutor-101-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@222a8288: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@d50c865
   [junit4]   2> 42593 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1c7c6669{/solr,null,UNAVAILABLE}
   [junit4]   2> 42593 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 72593 WARN  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.e.j.u.t.QueuedThreadPool QueuedThreadPool@qtp784058621{STOPPING,8<=8<=10000,i=0,q=3} Couldn't stop Thread[qtp784058621-125,5,TGRP-TestExternalFeatures]
   [junit4]   2> 72593 WARN  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.e.j.u.t.QueuedThreadPool QueuedThreadPool@qtp784058621{STOPPING,8<=8<=10000,i=0,q=3} Couldn't stop Thread[qtp784058621-123,5,TGRP-TestExternalFeatures]
   [junit4]   2> 72593 WARN  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.e.j.u.t.QueuedThreadPool QueuedThreadPool@qtp784058621{STOPPING,8<=8<=10000,i=0,q=3} Couldn't stop Thread[qtp784058621-126,5,TGRP-TestExternalFeatures]
   [junit4]   2> 72596 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 72596 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=416643198
   [junit4]   2> 72596 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 72596 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4183cec1: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@204e831b
   [junit4]   2> 72597 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 72597 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@fdf36a1: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@599fc216
   [junit4]   2> 72598 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 72598 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@24e92dfc: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@5d8f1de8
   [junit4]   2> 72600 INFO  (coreCloseExecutor-103-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@350c4429
   [junit4]   2> 72600 INFO  (coreCloseExecutor-103-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=350c4429
   [junit4]   2> 72601 INFO  (coreCloseExecutor-103-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4134dd29: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@33b5a759
   [junit4]   2> 72614 INFO  (SUITE-TestExternalFeatures-seed#[83E976A018F08D7C]-worker) [    ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
   [junit4]   2> Apr 16, 2018 5:00:51 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 4 leaked thread(s).
   [junit4]   2> Apr 16, 2018 5:02:11 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.ltr.feature.TestExternalFeatures:
   [junit4]   2>    1) Thread[id=126, name=qtp784058621-126, state=TIMED_WAITING, group=TGRP-TestExternalFeatures]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]   2>         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]   2>         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Apr 16, 2018 5:02:11 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=126, name=qtp784058621-126, state=TIMED_WAITING, group=TGRP-TestExternalFeatures]
   [junit4]   2> Apr 16, 2018 5:02:14 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=126, name=qtp784058621-126, state=TIMED_WAITING, group=TGRP-TestExternalFeatures]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]   2>         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]   2>         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=31556, maxDocsPerChunk=8, blockSize=258), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=31556, blockSize=258)), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@4eb9dde4), locale=hu-HU, timezone=America/Halifax
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=2,free=226781008,total=295174144
   [junit4]   2> NOTE: All tests run in this JVM: [TestOriginalScoreScorer, TestFilterSolrFeature, TestRerankBase, TestLTRQParserPlugin, TestAdapterModel, TestExternalFeatures]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestExternalFeatures -Dtests.seed=83E976A018F08D7C -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=hu-HU -Dtests.timezone=America/Halifax -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J2 | TestExternalFeatures (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.ltr.feature.TestExternalFeatures:
   [junit4]    >    1) Thread[id=126, name=qtp784058621-126, state=TIMED_WAITING, group=TGRP-TestExternalFeatures]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]    >         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]    >         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    > at __randomizedtesting.SeedInfo.seed([83E976A018F08D7C]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=126, name=qtp784058621-126, state=TIMED_WAITING, group=TGRP-TestExternalFeatures]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
   [junit4]    >         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
   [junit4]    >         at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    > at __randomizedtesting.SeedInfo.seed([83E976A018F08D7C]:0)
   [junit4] Completed [37/37 (1!)] on J2 in 115.58s, 7 tests, 2 errors <<< FAILURES!

[...truncated 1754 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:651: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid24166.hprof

Total time: 317 minutes 3 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1533 - Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1533/

4 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger

Error Message:
waitFor not elapsed but produced an event

Stack Trace:
java.lang.AssertionError: waitFor not elapsed but produced an event
        at __randomizedtesting.SeedInfo.seed([44BEF462524B4CCB:2775C2E0CB843FE6]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:180)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.handler.admin.SegmentsInfoRequestHandlerTest.testSegmentInfos

Error Message:


Stack Trace:
java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([44BEF462524B4CCB:8E963DD9847949B]:0)
        at org.apache.lucene.index.IndexWriter$ReaderPool.noDups(IndexWriter.java:867)
        at org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:857)
        at org.apache.lucene.index.IndexWriter.numDeletesToMerge(IndexWriter.java:5233)
        at org.apache.lucene.index.LogMergePolicy.sizeDocs(LogMergePolicy.java:153)
        at org.apache.lucene.index.LogDocMergePolicy.size(LogDocMergePolicy.java:44)
        at org.apache.lucene.index.LogMergePolicy.findMerges(LogMergePolicy.java:469)
        at org.apache.solr.handler.admin.SegmentsInfoRequestHandler.getMergeCandidatesNames(SegmentsInfoRequestHandler.java:100)
        at org.apache.solr.handler.admin.SegmentsInfoRequestHandler.getSegmentsInfo(SegmentsInfoRequestHandler.java:59)
        at org.apache.solr.handler.admin.SegmentsInfoRequestHandler.handleRequestBody(SegmentsInfoRequestHandler.java:48)
        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:2508)
        at org.apache.solr.util.TestHarness.query(TestHarness.java:337)
        at org.apache.solr.util.TestHarness.query(TestHarness.java:319)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:890)
        at org.apache.solr.handler.admin.SegmentsInfoRequestHandlerTest.testSegmentInfos(SegmentsInfoRequestHandlerTest.java:61)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.handler.admin.SegmentsInfoRequestHandlerTest.testSegmentInfosVersion

Error Message:


Stack Trace:
java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([44BEF462524B4CCB:BC60618A2A279D98]:0)
        at org.apache.lucene.index.IndexWriter$ReaderPool.noDups(IndexWriter.java:867)
        at org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:857)
        at org.apache.lucene.index.IndexWriter.numDeletesToMerge(IndexWriter.java:5233)
        at org.apache.lucene.index.LogMergePolicy.sizeDocs(LogMergePolicy.java:153)
        at org.apache.lucene.index.LogDocMergePolicy.size(LogDocMergePolicy.java:44)
        at org.apache.lucene.index.LogMergePolicy.findMerges(LogMergePolicy.java:469)
        at org.apache.solr.handler.admin.SegmentsInfoRequestHandler.getMergeCandidatesNames(SegmentsInfoRequestHandler.java:100)
        at org.apache.solr.handler.admin.SegmentsInfoRequestHandler.getSegmentsInfo(SegmentsInfoRequestHandler.java:59)
        at org.apache.solr.handler.admin.SegmentsInfoRequestHandler.handleRequestBody(SegmentsInfoRequestHandler.java:48)
        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:2508)
        at org.apache.solr.util.TestHarness.query(TestHarness.java:337)
        at org.apache.solr.util.TestHarness.query(TestHarness.java:319)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:890)
        at org.apache.solr.handler.admin.SegmentsInfoRequestHandlerTest.testSegmentInfosVersion(SegmentsInfoRequestHandlerTest.java:68)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.handler.admin.SegmentsInfoRequestHandlerTest.testSegmentInfosData

Error Message:


Stack Trace:
java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([44BEF462524B4CCB:3817D639E4A3FB54]:0)
        at org.apache.lucene.index.IndexWriter$ReaderPool.noDups(IndexWriter.java:867)
        at org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:857)
        at org.apache.lucene.index.IndexWriter.numDeletesToMerge(IndexWriter.java:5233)
        at org.apache.lucene.index.LogMergePolicy.sizeDocs(LogMergePolicy.java:153)
        at org.apache.lucene.index.LogDocMergePolicy.size(LogDocMergePolicy.java:44)
        at org.apache.lucene.index.LogMergePolicy.findMerges(LogMergePolicy.java:469)
        at org.apache.solr.handler.admin.SegmentsInfoRequestHandler.getMergeCandidatesNames(SegmentsInfoRequestHandler.java:100)
        at org.apache.solr.handler.admin.SegmentsInfoRequestHandler.getSegmentsInfo(SegmentsInfoRequestHandler.java:59)
        at org.apache.solr.handler.admin.SegmentsInfoRequestHandler.handleRequestBody(SegmentsInfoRequestHandler.java:48)
        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:2508)
        at org.apache.solr.util.TestHarness.query(TestHarness.java:337)
        at org.apache.solr.util.TestHarness.query(TestHarness.java:319)
        at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:890)
        at org.apache.solr.handler.admin.SegmentsInfoRequestHandlerTest.testSegmentInfosData(SegmentsInfoRequestHandlerTest.java:75)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13507 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
   [junit4]   2> 845437 INFO  (SUITE-IndexSizeTriggerTest-seed#[44BEF462524B4CCB]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_44BEF462524B4CCB-001/init-core-data-001
   [junit4]   2> 845443 INFO  (SUITE-IndexSizeTriggerTest-seed#[44BEF462524B4CCB]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 845444 INFO  (SUITE-IndexSizeTriggerTest-seed#[44BEF462524B4CCB]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 845444 INFO  (SUITE-IndexSizeTriggerTest-seed#[44BEF462524B4CCB]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_44BEF462524B4CCB-001/tempDir-001
   [junit4]   2> 845444 INFO  (SUITE-IndexSizeTriggerTest-seed#[44BEF462524B4CCB]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 845444 INFO  (Thread-25009) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 845444 INFO  (Thread-25009) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 845446 ERROR (Thread-25009) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 845544 INFO  (SUITE-IndexSizeTriggerTest-seed#[44BEF462524B4CCB]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:32797
   [junit4]   2> 845578 INFO  (zkConnectionManagerCallback-1531-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 845588 INFO  (jetty-launcher-1528-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 845589 INFO  (jetty-launcher-1528-thread-2) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 845603 INFO  (jetty-launcher-1528-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 845603 INFO  (jetty-launcher-1528-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 845603 INFO  (jetty-launcher-1528-thread-1) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 845603 INFO  (jetty-launcher-1528-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@695dee0d{/solr,null,AVAILABLE}
   [junit4]   2> 845605 INFO  (jetty-launcher-1528-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@6f21cf61{HTTP/1.1,[http/1.1]}{127.0.0.1:59054}
   [junit4]   2> 845605 INFO  (jetty-launcher-1528-thread-1) [    ] o.e.j.s.Server Started @845650ms
   [junit4]   2> 845605 INFO  (jetty-launcher-1528-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59054}
   [junit4]   2> 845605 ERROR (jetty-launcher-1528-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 845605 INFO  (jetty-launcher-1528-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 845605 INFO  (jetty-launcher-1528-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 845605 INFO  (jetty-launcher-1528-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 845605 INFO  (jetty-launcher-1528-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 845605 INFO  (jetty-launcher-1528-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-04-17T11:39:54.885Z
   [junit4]   2> 845606 INFO  (jetty-launcher-1528-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 845606 INFO  (jetty-launcher-1528-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 845606 INFO  (jetty-launcher-1528-thread-2) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 845607 INFO  (jetty-launcher-1528-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3db76c05{/solr,null,AVAILABLE}
   [junit4]   2> 845607 INFO  (jetty-launcher-1528-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@21051619{HTTP/1.1,[http/1.1]}{127.0.0.1:58192}
   [junit4]   2> 845607 INFO  (jetty-launcher-1528-thread-2) [    ] o.e.j.s.Server Started @845652ms
   [junit4]   2> 845607 INFO  (jetty-launcher-1528-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58192}
   [junit4]   2> 845607 ERROR (jetty-launcher-1528-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 845607 INFO  (jetty-launcher-1528-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 845607 INFO  (jetty-launcher-1528-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 845607 INFO  (jetty-launcher-1528-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 845607 INFO  (jetty-launcher-1528-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 845607 INFO  (jetty-launcher-1528-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-04-17T11:39:54.887Z
   [junit4]   2> 845627 INFO  (zkConnectionManagerCallback-1533-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 845628 INFO  (jetty-launcher-1528-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 845628 INFO  (zkConnectionManagerCallback-1535-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 845629 INFO  (jetty-launcher-1528-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 845637 INFO  (jetty-launcher-1528-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32797/solr
   [junit4]   2> 845641 INFO  (jetty-launcher-1528-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32797/solr
   [junit4]   2> 845647 INFO  (zkConnectionManagerCallback-1539-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 845651 INFO  (zkConnectionManagerCallback-1543-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 845678 INFO  (zkConnectionManagerCallback-1545-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 845699 INFO  (zkConnectionManagerCallback-1547-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 845706 INFO  (jetty-launcher-1528-thread-1) [n:127.0.0.1:59054_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 845707 INFO  (jetty-launcher-1528-thread-1) [n:127.0.0.1:59054_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59054_solr
   [junit4]   2> 845708 INFO  (jetty-launcher-1528-thread-1) [n:127.0.0.1:59054_solr    ] o.a.s.c.Overseer Overseer (id=73644685449691142-127.0.0.1:59054_solr-n_0000000000) starting
   [junit4]   2> 845726 INFO  (zkConnectionManagerCallback-1556-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 845728 INFO  (jetty-launcher-1528-thread-1) [n:127.0.0.1:59054_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:32797/solr ready
   [junit4]   2> 845762 INFO  (jetty-launcher-1528-thread-1) [n:127.0.0.1:59054_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59054_solr
   [junit4]   2> 845765 DEBUG (OverseerAutoScalingTriggerThread-73644685449691142-127.0.0.1:59054_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 845776 INFO  (zkCallback-1546-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 845776 INFO  (zkCallback-1555-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 845777 DEBUG (OverseerAutoScalingTriggerThread-73644685449691142-127.0.0.1:59054_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 845780 DEBUG (OverseerAutoScalingTriggerThread-73644685449691142-127.0.0.1:59054_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 845780 DEBUG (OverseerAutoScalingTriggerThread-73644685449691142-127.0.0.1:59054_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 845800 INFO  (jetty-launcher-1528-thread-2) [n:127.0.0.1:58192_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 845800 DEBUG (OverseerAutoScalingTriggerThread-73644685449691142-127.0.0.1:59054_solr-n_0000000000) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:59054_solr]
   [junit4]   2> 845802 INFO  (jetty-launcher-1528-thread-2) [n:127.0.0.1:58192_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 845805 INFO  (jetty-launcher-1528-thread-2) [n:127.0.0.1:58192_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 845805 INFO  (jetty-launcher-1528-thread-2) [n:127.0.0.1:58192_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58192_solr
   [junit4]   2> 845818 DEBUG (OverseerAutoScalingTriggerThread-73644685449691142-127.0.0.1:59054_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 845821 INFO  (zkCallback-1555-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 845821 DEBUG (OverseerAutoScalingTriggerThread-73644685449691142-127.0.0.1:59054_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 845834 DEBUG (ScheduledTrigger-2246-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 845854 INFO  (zkCallback-1544-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 845874 INFO  (zkCallback-1546-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 845910 INFO  (jetty-launcher-1528-thread-1) [n:127.0.0.1:59054_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59054.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12c696ce
   [junit4]   2> 845910 INFO  (jetty-launcher-1528-thread-1) [n:127.0.0.1:59054_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59054.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12c696ce
   [junit4]   2> 845910 INFO  (jetty-launcher-1528-thread-1) [n:127.0.0.1:59054_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59054.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12c696ce
   [junit4]   2> 845911 INFO  (jetty-launcher-1528-thread-1) [n:127.0.0.1:59054_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_44BEF462524B4CCB-001/tempDir-001/node1/.
   [junit4]   2> 845987 INFO  (jetty-launcher-1528-thread-2) [n:127.0.0.1:58192_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58192.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12c696ce
   [junit4]   2> 845994 INFO  (jetty-launcher-1528-thread-2) [n:127.0.0.1:58192_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58192.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12c696ce
   [junit4]   2> 845994 INFO  (jetty-launcher-1528-thread-2) [n:127.0.0.1:58192_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58192.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12c696ce
   [junit4]   2> 845995 INFO  (jetty-launcher-1528-thread-2) [n:127.0.0.1:58192_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_44BEF462524B4CCB-001/tempDir-001/node2/.
   [junit4]   2> 846018 INFO  (zkConnectionManagerCallback-1561-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 846020 INFO  (jetty-launcher-1528-thread-2) [n:127.0.0.1:58192_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 846020 INFO  (jetty-launcher-1528-thread-2) [n:127.0.0.1:58192_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:32797/solr ready
   [junit4]   2> 846113 INFO  (zkConnectionManagerCallback-1563-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 846130 INFO  (zkConnectionManagerCallback-1568-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 846131 INFO  (SUITE-IndexSizeTriggerTest-seed#[44BEF462524B4CCB]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 846138 INFO  (SUITE-IndexSizeTriggerTest-seed#[44BEF462524B4CCB]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:32797/solr ready
   [junit4]   2> 846174 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 846174 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 846181 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 846181 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 846186 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 846187 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 846187 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 846187 DEBUG (ScheduledTrigger-2255-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846208 DEBUG (ScheduledTrigger-2255-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846229 DEBUG (ScheduledTrigger-2255-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846249 DEBUG (ScheduledTrigger-2255-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846269 DEBUG (ScheduledTrigger-2255-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846289 DEBUG (ScheduledTrigger-2255-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846310 DEBUG (ScheduledTrigger-2255-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846330 DEBUG (ScheduledTrigger-2255-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846350 DEBUG (ScheduledTrigger-2255-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846373 DEBUG (ScheduledTrigger-2255-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846394 DEBUG (ScheduledTrigger-2255-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846414 DEBUG (ScheduledTrigger-2255-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846416 INFO  (TEST-IndexSizeTriggerTest.testSplitIntegration-seed#[44BEF462524B4CCB]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSplitIntegration
   [junit4]   2> 846437 DEBUG (ScheduledTrigger-2255-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846457 DEBUG (ScheduledTrigger-2255-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846477 DEBUG (ScheduledTrigger-2255-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846498 DEBUG (ScheduledTrigger-2255-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846512 DEBUG (simCloudManagerPool-2254-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard2: {"core_node3":{
   [junit4]   2>     "core":"testSplitIntegration_collection_shard2_replica_n3",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testSplitIntegration_collection"}}
   [junit4]   2> 846515 DEBUG (simCloudManagerPool-2254-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard1: {"core_node1":{
   [junit4]   2>     "core":"testSplitIntegration_collection_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testSplitIntegration_collection"}}
   [junit4]   2> 846518 DEBUG (ScheduledTrigger-2255-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846538 DEBUG (ScheduledTrigger-2255-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846548 DEBUG (simCloudManagerPool-2254-thread-8) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 846548 DEBUG (simCloudManagerPool-2254-thread-8) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 846552 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 846552 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 846552 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 846558 DEBUG (ScheduledTrigger-2255-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846562 DEBUG (simCloudManagerPool-2254-thread-9) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 846562 DEBUG (simCloudManagerPool-2254-thread-9) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 846562 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 846563 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 846563 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 846564 DEBUG (simCloudManagerPool-2254-thread-10) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 846564 DEBUG (simCloudManagerPool-2254-thread-10) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 846564 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 846565 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 846565 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 846578 DEBUG (ScheduledTrigger-2255-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846598 DEBUG (ScheduledTrigger-2255-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846619 DEBUG (ScheduledTrigger-2255-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846639 DEBUG (ScheduledTrigger-2255-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846659 DEBUG (ScheduledTrigger-2255-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846679 DEBUG (ScheduledTrigger-2255-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846699 DEBUG (ScheduledTrigger-2255-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846711 DEBUG (ScheduledTrigger-2255-thread-4) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"56095660215865T5y5l4msagmlelppy1cjjz2jd",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":24217114581751909,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":11,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":11,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":11,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}
   [junit4]   2> 846727 DEBUG (ScheduledTrigger-2255-thread-4) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [index_size_trigger, .auto_add_replicas, .scheduled_maintenance]
   [junit4]   2> 846730 INFO  (ScheduledTrigger-2255-thread-4) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=24217123186475459, stage=STARTED, actionName='null', event={
   [junit4]   2>   "id":"56095660215865T5y5l4msagmlelppy1cjjz2jd",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":24217114581751909,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":11,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":11,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":11,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":24217123082606759,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}, context={}, config={
   [junit4]   2>   "trigger":"index_size_trigger",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 846754 DEBUG (AutoscalingActionExecutor-2256-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"56095660215865T5y5l4msagmlelppy1cjjz2jd",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":24217114581751909,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":11,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":11,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":11,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":24217123082606759,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}
   [junit4]   2> 846756 INFO  (AutoscalingActionExecutor-2256-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=24217124494836909, stage=BEFORE_ACTION, actionName='compute_plan', event={
   [junit4]   2>   "id":"56095660215865T5y5l4msagmlelppy1cjjz2jd",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":24217114581751909,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":11,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":11,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":11,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":24217123082606759,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger}, config={
   [junit4]   2>   "trigger":"index_size_trigger",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 846756 DEBUG (AutoscalingActionExecutor-2256-thread-1) [    ] o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"56095660215865T5y5l4msagmlelppy1cjjz2jd",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":24217114581751909,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":11,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":11,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":11,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":24217123082606759,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}} with context properties: {BEFORE_ACTION=[compute_plan]}
   [junit4]   2> 846759 INFO  (AutoscalingActionExecutor-2256-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=SPLITSHARD&collection=testSplitIntegration_collection&shard=shard1
   [junit4]   2> 846759 INFO  (AutoscalingActionExecutor-2256-thread-1) [    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=SPLITSHARD&collection=testSplitIntegration_collection&shard=shard2
   [junit4]   2> 846759 INFO  (AutoscalingActionExecutor-2256-thread-1) [    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 24217124652 sessionWrapper.createTime 24217124554018759, this.sessionWrapper.createTime 24217124554018759
   [junit4]   2> 846760 INFO  (AutoscalingActionExecutor-2256-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=24217124666527209, stage=AFTER_ACTION, actionName='compute_plan', event={
   [junit4]   2>   "id":"56095660215865T5y5l4msagmlelppy1cjjz2jd",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":24217114581751909,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "__start__":2,
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":11,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":11,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":11,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":24217123082606759,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard2}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger, properties.AFTER_ACTION=[compute_plan]}, config={
   [junit4]   2>   "trigger":"index_size_trigger",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 846760 INFO  (AutoscalingActionExecutor-2256-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=24217124691141159, stage=BEFORE_ACTION, actionName='execute_plan', event={
   [junit4]   2>   "id":"56095660215865T5y5l4msagmlelppy1cjjz2jd",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":24217114581751909,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "__start__":2,
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":11,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":11,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":11,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":24217123082606759,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard2}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger, properties.AFTER_ACTION=[compute_plan]}, config={
   [junit4]   2>   "trigger":"index_size_trigger",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 846760 DEBUG (AutoscalingActionExecutor-2256-thread-1) [    ] o.a.s.c.a.ExecutePlanAction -- processing event: {
   [junit4]   2>   "id":"56095660215865T5y5l4msagmlelppy1cjjz2jd",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":24217114581751909,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "__start__":2,
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":11,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":11,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":11,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":24217123082606759,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard@4673d9f, org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard@6273677d], AFTER_ACTION=[compute_plan], BEFORE_ACTION=[compute_plan, execute_plan]}
   [junit4]   2> 846761 DEBUG (AutoscalingActionExecutor-2256-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=SPLITSHARD&collection=testSplitIntegration_collection&shard=shard1
   [junit4]   2> 846837 DEBUG (ScheduledTrigger-2246-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 846935 DEBUG (simCloudManagerPool-2254-thread-41) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard1_1: {"core_node10":{
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":7,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>     "base_url":"http://127.0.0.1:10000/solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":7,
   [junit4]   2>     "core":"testSplitIntegration_collection_shard1_1_replica1",
   [junit4]   2>     "shard":"shard1_1",
   [junit4]   2>     "collection":"testSplitIntegration_collection"}}
   [junit4]   2> 846935 DEBUG (simCloudManagerPool-2254-thread-42) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard1_0: {"core_node7":{
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":7,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "base_url":"http://127.0.0.1:10001/solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":7,
   [junit4]   2>     "core":"testSplitIntegration_collection_shard1_0_replica0",
   [junit4]   2>     "shard":"shard1_0",
   [junit4]   2>     "collection":"testSplitIntegration_collection"}}
   [junit4]   2> 846949 DEBUG (AutoscalingActionExecutor-2256-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=SPLITSHARD&collection=testSplitIntegration_collection&shard=shard2
   [junit4]   2> 847099 DEBUG (simCloudManagerPool-2254-thread-48) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard2_1: {"core_node16":{
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":5,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>     "base_url":"http://127.0.0.1:10000/solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":5,
   [junit4]   2>     "core":"testSplitIntegration_collection_shard2_1_replica1",
   [junit4]   2>     "shard":"shard2_1",
   [junit4]   2>     "collection":"testSplitIntegration_collection"}}
   [junit4]   2> 847102 DEBUG (simCloudManagerPool-2254-thread-51) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSplitIntegration_collection / shard2_0: {"core_node14":{
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":6,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>     "base_url":"http://127.0.0.1:10000/solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":6,
   [junit4]   2>     "core":"testSplitIntegration_collection_shard2_0_replica1",
   [junit4]   2>     "shard":"shard2_0",
   [junit4]   2>     "collection":"testSplitIntegration_collection"}}
   [junit4]   2> 847103 INFO  (AutoscalingActionExecutor-2256-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=24217141851034409, stage=AFTER_ACTION, actionName='execute_plan', event={
   [junit4]   2>   "id":"56095660215865T5y5l4msagmlelppy1cjjz2jd",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":24217114581751909,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "__start__":2,
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":11,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":11,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":11,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":24217123082606759,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger/56095660215865T5y5l4msagmlelppy1cjjz2jd/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger/56095660215865T5y5l4msagmlelppy1cjjz2jd/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard2}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger/56095660215865T5y5l4msagmlelppy1cjjz2jd/0] in completed tasks},success=}, {status={state=completed,msg=found [index_size_trigger/56095660215865T5y5l4msagmlelppy1cjjz2jd/0] in completed tasks},success=}], source=index_size_trigger, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={
   [junit4]   2>   "trigger":"index_size_trigger",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 847105 INFO  (AutoscalingActionExecutor-2256-thread-1) [    ] o.a.s.c.a.IndexSizeTriggerTest =======> CapturedEvent{timestamp=24217141924754009, stage=SUCCEEDED, actionName='null', event={
   [junit4]   2>   "id":"56095660215865T5y5l4msagmlelppy1cjjz2jd",
   [junit4]   2>   "source":"index_size_trigger",
   [junit4]   2>   "eventTime":24217114581751909,
   [junit4]   2>   "eventType":"INDEXSIZE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "__start__":2,
   [junit4]   2>     "aboveSize":{"testSplitIntegration_collection":[
   [junit4]   2>         "{\"core_node1\":{\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":14,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":14,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":14,\n    \"__bytes__\":1,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
   [junit4]   2>         "{\"core_node3\":{\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":11,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":1,\n    \"__docs__\":11,\n    \"violationType\":\"aboveDocs\",\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":11,\n    \"__bytes__\":1,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
   [junit4]   2>     "belowSize":{},
   [junit4]   2>     "_enqueue_time_":24217123082606759,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard2\"}]}}"]}}, context={}, config={
   [junit4]   2>   "trigger":"index_size_trigger",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"],
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":[
   [junit4]   2>     "compute_plan",
   [junit4]   2>     "execute_plan"]}, message='null'}
   [junit4]   2> 847115 INFO  (TEST-IndexSizeTriggerTest.testSplitIntegration-seed#[44BEF462524B4CCB]) [    ] o.a.s.c.a.IndexSizeTriggerTest #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes: 2
   [junit4]   2> ## Empty nodes: 0
   [junit4]   2> ## Dead nodes: 0
   [junit4]   2> ## Collections: [testSplitIntegration_collection]
   [junit4]   2> DocCollection(testSplitIntegration_collection//clusterstate.json/35)={
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"2",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "autoCreated":"true",
   [junit4]   2>   "shards":{
   [junit4]   2>     "shard2":{
   [junit4]   2>       "stateTimestamp":"1523965243084663850",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testSplitIntegration_collection_shard2_replica_n3",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":11,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":11},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testSplitIntegration_collection_shard2_replica_n4",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":11,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":11}},
   [junit4]   2>       "range":"0-7fffffff",
   [junit4]   2>       "state":"inactive"},
   [junit4]   2>     "shard1":{
   [junit4]   2>       "stateTimestamp":"1523965234475678600",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node1":{
   [junit4]   2>           "core":"testSplitIntegration_collection_shard1_replica_n1",
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":14,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":14},
   [junit4]   2>         "core_node2":{
   [junit4]   2>           "core":"testSplitIntegration_collection_shard1_replica_n2",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":14,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":14}},
   [junit4]   2>       "range":"80000000-ffffffff",
   [junit4]   2>       "state":"inactive"},
   [junit4]   2>     "shard2_1":{
   [junit4]   2>       "parent":"shard2",
   [junit4]   2>       "stateTimestamp":"1523965243085707500",
   [junit4]   2>       "range":"40000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node16":{
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "core":"testSplitIntegration_collection_shard2_1_replica1",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":5,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>           "base_url":"http://127.0.0.1:10000/solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":5},
   [junit4]   2>         "core_node15":{
   [junit4]   2>           "core":"testSplitIntegration_collection_shard2_1_replica0",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":5,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>           "base_url":"http://127.0.0.1:10001/solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":5}}},
   [junit4]   2>     "shard1_1":{
   [junit4]   2>       "parent":"shard1",
   [junit4]   2>       "stateTimestamp":"1523965234497316600",
   [junit4]   2>       "range":"c0000000-ffffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node10":{
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "core":"testSplitIntegration_collection_shard1_1_replica1",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":7,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>           "base_url":"http://127.0.0.1:10000/solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":7},
   [junit4]   2>         "core_node9":{
   [junit4]   2>           "core":"testSplitIntegration_collection_shard1_1_replica0",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":7,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>           "base_url":"http://127.0.0.1:10001/solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":7}}},
   [junit4]   2>     "shard1_0":{
   [junit4]   2>       "parent":"shard1",
   [junit4]   2>       "stateTimestamp":"1523965234497061700",
   [junit4]   2>       "range":"80000000-bfffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node7":{
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "core":"testSplitIntegration_collection_shard1_0_replica0",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":7,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>           "base_url":"http://127.0.0.1:10001/solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":7},
   [junit4]   2>         "core_node8":{
   [junit4]   2>           "core":"testSplitIntegration_collection_shard1_0_replica1",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":7,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>           "base_url":"http://127.0.0.1:10000/solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":7}}},
   [junit4]   2>     "shard2_0":{
   [junit4]   2>       "parent":"shard2",
   [junit4]   2>       "stateTimestamp":"1523965243085471150",
   [junit4]   2>       "range":"0-3fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node14":{
   [junit4]   2>           "leader":"true",
   [junit4]   2>           "core":"testSplitIntegration_collection_shard2_0_replica1",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":6,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>           "base_url":"http://127.0.0.1:10000/solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":6},
   [junit4]   2>         "core_node13":{
   [junit4]   2>           "core":"testSplitIntegration_collection_shard2_0_replica0",
   [junit4]   2>           "SEARCHER.searcher.maxDoc":6,
   [junit4]   2>           "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>           "INDEX.sizeInBytes":1,
   [junit4]   2>           "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>           "base_url":"http://127.0.0.1:10001/solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "SEARCHER.searcher.numDocs":6}}}}}
   [junit4]   2> ## Max replicas per node: 6
   [junit4]   2> ## Min replicas per node: 6
   [junit4]   2> ## Total replicas: 12
   [junit4]   2> ## * testSplitIntegration_collection 12
   [junit4]   2> ## - active          12
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ## - CREATE             1
   [junit4]   2> ## - DELETESTATUS       2
   [junit4]   2> ## - REQUESTSTATUS      2
   [junit4]   2> ## - SPLITSHARD         2
   [junit4]   2> ## - autoscaling        3
   [junit4]   2> ## - update            27
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> ## * Trigger: index_size_trigger
   [junit4]   2> ## - STARTED         1
   [junit4]   2>
   [junit4]   2> 847116 DEBUG (TEST-IndexSizeTriggerTest.testSplitIntegration-seed#[44BEF462524B4CCB]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 847116 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 847116 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 847116 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 847116 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 847122 DEBUG (AutoscalingActionExecutor-2256-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 18390 ms for event id=56095660215865T5y5l4msagmlelppy1cjjz2jd
   [junit4]   2> 847216 INFO  (TEST-IndexSizeTriggerTest.testSplitIntegration-seed#[44BEF462524B4CCB]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testSplitIntegration
   [junit4]   2> 847259 INFO  (TEST-IndexSizeTriggerTest.testMergeIntegration-seed#[44BEF462524B4CCB]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testMergeIntegration
   [junit4]   2> 847303 DEBUG (simCloudManagerPool-2254-thread-60) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testMergeIntegration_collection / shard2: {"core_node3":{
   [junit4]   2>     "core":"testMergeIntegration_collection_shard2_replica_n3",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testMergeIntegration_collection"}}
   [junit4]   2> 847304 DEBUG (simCloudManagerPool-2254-thread-61) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testMergeIntegration_collection / shard1: {"core_node1":{
   [junit4]   2>     "core":"testMergeIntegration_collection_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testMergeIntegration_collection"}}
   [junit4]   2> 847330 DEBUG (simCloudManagerPool-2254-thread-73) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 847330 DEBUG (simCloudManagerPool-2254-thread-73) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 847330 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 847330 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 847330 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 847330 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 847331 DEBUG (simCloudManagerPool-2254-thread-74) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 847331 DEBUG (simCloudManagerPool-2254-thread-74) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 847331 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 847331 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 847331 DEBUG (Simulated Over

[...truncated too long message...]

Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/null2103383181
     [copy] Copying 40 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/null2103383181
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/null2103383181

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/null678195048
     [copy] Copying 246 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/null678195048
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/null678195048

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 43ms :: artifacts dl 1ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
        ---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 273 minutes 31 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1534 - Failure

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1534/

6 tests failed.
FAILED:  org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig

Error Message:
CheckIndex failed

Stack Trace:
java.lang.RuntimeException: CheckIndex failed
        at __randomizedtesting.SeedInfo.seed([DFB9317353BCF3DC:58EE4CFCC2E58F5C]:0)
        at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:305)
        at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:285)
        at org.apache.lucene.store.BaseDirectoryWrapper.close(BaseDirectoryWrapper.java:45)
        at org.apache.lucene.util.IOUtils.close(IOUtils.java:88)
        at org.apache.lucene.util.IOUtils.close(IOUtils.java:76)
        at org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:293)
        at org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:160)
        at org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:75)
        at org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig(TestInetAddressRangeQueries.java:81)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
        at __randomizedtesting.SeedInfo.seed([C5031A749CF2A0AE]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
        at __randomizedtesting.SeedInfo.seed([C5031A749CF2A0AE]:0)


FAILED:  org.apache.solr.cloud.RollingRestartTest.test

Error Message:
Unable to restart (#4): CloudJettyRunner [url=http://127.0.0.1:33556/b_/collection1_shard1_replica_n59]

Stack Trace:
java.lang.AssertionError: Unable to restart (#4): CloudJettyRunner [url=http://127.0.0.1:33556/b_/collection1_shard1_replica_n59]
        at __randomizedtesting.SeedInfo.seed([C5031A749CF2A0AE:4D5725AE320ECD56]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:103)
        at org.apache.solr.cloud.RollingRestartTest.test(RollingRestartTest.java:52)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:34002/dd: ADDREPLICA failed to create replica

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:34002/dd: ADDREPLICA failed to create replica
        at __randomizedtesting.SeedInfo.seed([C5031A749CF2A0AE:4D5725AE320ECD56]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:125)
        at org.apache.solr.cloud.UnloadDistributedZkTest.test(UnloadDistributedZkTest.java:72)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.hdfs.StressHdfsTest.test

Error Message:
Could not find collection:delete_data_dir

Stack Trace:
java.lang.AssertionError: Could not find collection:delete_data_dir
        at __randomizedtesting.SeedInfo.seed([C5031A749CF2A0AE:4D5725AE320ECD56]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNotNull(Assert.java:526)
        at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
        at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:140)
        at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:135)
        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:915)
        at org.apache.solr.cloud.hdfs.StressHdfsTest.createAndDeleteCollection(StressHdfsTest.java:161)
        at org.apache.solr.cloud.hdfs.StressHdfsTest.test(StressHdfsTest.java:105)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 9216 lines...]
   [junit4] Suite: org.apache.lucene.search.TestInetAddressRangeQueries
   [junit4]   1> CheckIndex failed
   [junit4]   1> Segments file=segments_1 numSegments=1 version=8.0.0 id=bzclxnd6cce5v24stbr4q4rjw
   [junit4]   1>   1 of 1: name=_2v maxDoc=1037626
   [junit4]   1>     version=8.0.0
   [junit4]   1>     id=bzclxnd6cce5v24stbr4q4rjv
   [junit4]   1>     codec=Asserting(Lucene70)
   [junit4]   1>     compound=false
   [junit4]   1>     numFiles=13
   [junit4]   1>     size (MB)=21.667
   [junit4]   1>     diagnostics = {os=Linux, java.vendor=Oracle Corporation, java.version=1.8.0_152, java.vm.version=25.152-b16, lucene.version=8.0.0, mergeMaxNumSegments=1, os.arch=amd64, java.runtime.version=1.8.0_152-b16, source=merge, mergeFactor=5, os.version=3.13.0-88-generic, timestamp=1524071467459}
   [junit4]   1>     no deletions
   [junit4]   1>     test: open reader.........FAILED
   [junit4]   1>     WARNING: exorciseIndex() would remove reference to this segment; full exception:
   [junit4]   1> java.lang.OutOfMemoryError: Java heap space
   [junit4]   1> at java.util.Arrays.copyOf(Arrays.java:3236)
   [junit4]   1> at org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:285)
   [junit4]   1> at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:354)
   [junit4]   1> at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:132)
   [junit4]   1> at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:116)
   [junit4]   1> at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:293)
   [junit4]   1> at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:373)
   [junit4]   1> at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:113)
   [junit4]   1> at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:78)
   [junit4]   1> at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:702)
   [junit4]   1> at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:300)
   [junit4]   1> at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:285)
   [junit4]   1> at org.apache.lucene.store.BaseDirectoryWrapper.close(BaseDirectoryWrapper.java:45)
   [junit4]   1> at org.apache.lucene.util.IOUtils.close(IOUtils.java:88)
   [junit4]   1> at org.apache.lucene.util.IOUtils.close(IOUtils.java:76)
   [junit4]   1> at org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:293)
   [junit4]   1> at org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:160)
   [junit4]   1> at org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:75)
   [junit4]   1> at org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig(TestInetAddressRangeQueries.java:81)
   [junit4]   1> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   1> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   1> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   1> at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   1> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
   [junit4]   1> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
   [junit4]   1> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
   [junit4]   1> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
   [junit4]   1> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   [junit4]   1> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   1> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   [junit4]   1> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   1> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   1>
   [junit4]   1> WARNING: 1 broken segments (containing 1037626 documents) detected
   [junit4]   1> Took 8.811 sec total.
   [junit4]   1>
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestInetAddressRangeQueries -Dtests.method=testRandomBig -Dtests.seed=DFB9317353BCF3DC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=hr -Dtests.timezone=Australia/Brisbane -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    464s J1 | TestInetAddressRangeQueries.testRandomBig <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: CheckIndex failed
   [junit4]    > at __randomizedtesting.SeedInfo.seed([DFB9317353BCF3DC:58EE4CFCC2E58F5C]:0)
   [junit4]    > at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:305)
   [junit4]    > at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:285)
   [junit4]    > at org.apache.lucene.store.BaseDirectoryWrapper.close(BaseDirectoryWrapper.java:45)
   [junit4]    > at org.apache.lucene.util.IOUtils.close(IOUtils.java:88)
   [junit4]    > at org.apache.lucene.util.IOUtils.close(IOUtils.java:76)
   [junit4]    > at org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:293)
   [junit4]    > at org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:160)
   [junit4]    > at org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:75)
   [junit4]    > at org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig(TestInetAddressRangeQueries.java:81)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/misc/test/J1/temp/lucene.search.TestInetAddressRangeQueries_DFB9317353BCF3DC-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=Direct)}, docValues:{ipRangeField=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=1266, maxMBSortInHeap=7.319658649705669, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@7e59a5af), locale=hr, timezone=Australia/Brisbane
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=301864032,total=364380160
   [junit4]   2> NOTE: All tests run in this JVM: [TestInetAddressRangeQueries]
   [junit4] Completed [12/13 (1!)] on J1 in 472.27s, 4 tests, 1 error <<< FAILURES!

[...truncated 4 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/misc/test/temp/junit4-J1-20180418_170712_4728517081836439621798.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/heapdumps/java_pid15388.hprof ...
   [junit4] Heap dump file created [586144610 bytes in 3.546 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 4266 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.StressHdfsTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001/init-core-data-001
   [junit4]   2> 1291741 WARN  (SUITE-StressHdfsTest-seed#[C5031A749CF2A0AE]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 1291741 INFO  (SUITE-StressHdfsTest-seed#[C5031A749CF2A0AE]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1291742 INFO  (SUITE-StressHdfsTest-seed#[C5031A749CF2A0AE]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 1291742 INFO  (SUITE-StressHdfsTest-seed#[C5031A749CF2A0AE]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1291742 INFO  (SUITE-StressHdfsTest-seed#[C5031A749CF2A0AE]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 1291785 WARN  (SUITE-StressHdfsTest-seed#[C5031A749CF2A0AE]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 1291803 WARN  (SUITE-StressHdfsTest-seed#[C5031A749CF2A0AE]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1291809 INFO  (SUITE-StressHdfsTest-seed#[C5031A749CF2A0AE]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1291821 INFO  (SUITE-StressHdfsTest-seed#[C5031A749CF2A0AE]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_51663_hdfs____.wk0mwv/webapp
   [junit4]   2> 1292606 INFO  (SUITE-StressHdfsTest-seed#[C5031A749CF2A0AE]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:51663
   [junit4]   2> 1292729 WARN  (SUITE-StressHdfsTest-seed#[C5031A749CF2A0AE]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1292730 INFO  (SUITE-StressHdfsTest-seed#[C5031A749CF2A0AE]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1292743 INFO  (SUITE-StressHdfsTest-seed#[C5031A749CF2A0AE]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_39452_datanode____y0ntaq/webapp
   [junit4]   2> 1293192 INFO  (SUITE-StressHdfsTest-seed#[C5031A749CF2A0AE]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39452
   [junit4]   2> 1293352 WARN  (SUITE-StressHdfsTest-seed#[C5031A749CF2A0AE]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 1293353 INFO  (SUITE-StressHdfsTest-seed#[C5031A749CF2A0AE]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 1293365 INFO  (SUITE-StressHdfsTest-seed#[C5031A749CF2A0AE]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_56368_datanode____.tbs8x1/webapp
   [junit4]   2> 1293667 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001/tempDir-001/hdfsBaseDir/data/data2/]]  heartbeating to localhost/127.0.0.1:34043) [    ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
   [junit4]   2> 1293676 INFO  (Block report processor) [    ] BlockStateChange BLOCK* processReport 0x566e379f719f5c: from storage DS-30e5800d-9a59-4209-8779-e7426acb4fe4 node DatanodeRegistration(127.0.0.1:59067, datanodeUuid=f44b66a1-bc99-4494-a975-b6de39329fd7, infoPort=40745, infoSecurePort=0, ipcPort=33455, storageInfo=lv=-56;cid=testClusterID;nsid=530579525;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 1293677 INFO  (Block report processor) [    ] BlockStateChange BLOCK* processReport 0x566e379f719f5c: from storage DS-6e6cd7a1-edb3-4705-9ece-0ee5c8b86eb9 node DatanodeRegistration(127.0.0.1:59067, datanodeUuid=f44b66a1-bc99-4494-a975-b6de39329fd7, infoPort=40745, infoSecurePort=0, ipcPort=33455, storageInfo=lv=-56;cid=testClusterID;nsid=530579525;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 1293864 INFO  (SUITE-StressHdfsTest-seed#[C5031A749CF2A0AE]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:56368
   [junit4]   2> 1293946 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001/tempDir-001/hdfsBaseDir/data/data4/]]  heartbeating to localhost/127.0.0.1:34043) [    ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
   [junit4]   2> 1293955 INFO  (Block report processor) [    ] BlockStateChange BLOCK* processReport 0x566e37b012e306: from storage DS-2d78e635-0b15-404e-b60f-bf734edccb46 node DatanodeRegistration(127.0.0.1:46420, datanodeUuid=921b8df4-4d26-4e48-8614-bd78ecae9d80, infoPort=59119, infoSecurePort=0, ipcPort=52982, storageInfo=lv=-56;cid=testClusterID;nsid=530579525;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 1293955 INFO  (Block report processor) [    ] BlockStateChange BLOCK* processReport 0x566e37b012e306: from storage DS-82a0001b-00f8-48f1-9e10-11220ef37583 node DatanodeRegistration(127.0.0.1:46420, datanodeUuid=921b8df4-4d26-4e48-8614-bd78ecae9d80, infoPort=59119, infoSecurePort=0, ipcPort=52982, storageInfo=lv=-56;cid=testClusterID;nsid=530579525;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 1294034 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1294034 INFO  (Thread-24127) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1294034 INFO  (Thread-24127) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1294041 ERROR (Thread-24127) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1294134 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.ZkTestServer start zk server on port:34393
   [junit4]   2> 1294142 INFO  (zkConnectionManagerCallback-1075-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1294160 INFO  (zkConnectionManagerCallback-1077-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1294174 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1294176 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1294177 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1294178 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1294179 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1294180 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1294181 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1294182 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1294183 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1294184 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1294185 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1294187 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
   [junit4]   2> 1294275 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T01:27:37+04:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1294276 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1294276 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1294276 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1294280 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3cd02886{/,null,AVAILABLE}
   [junit4]   2> 1294281 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@6e40c77e{HTTP/1.1,[http/1.1]}{127.0.0.1:41717}
   [junit4]   2> 1294281 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.Server Started @1294333ms
   [junit4]   2> 1294281 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:34043/hdfs__localhost_34043__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001_tempDir-002_control_data, hostContext=/, hostPort=41717, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001/control-001/cores}
   [junit4]   2> 1294282 ERROR (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1294282 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1294282 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1294282 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1294282 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1294282 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-04-18T18:28:55.605Z
   [junit4]   2> 1294284 INFO  (zkConnectionManagerCallback-1079-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1294289 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1294289 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001/control-001/solr.xml
   [junit4]   2> 1294292 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1294292 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1294294 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1a886b63, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1294296 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34393/solr
   [junit4]   2> 1294300 INFO  (zkConnectionManagerCallback-1083-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1294302 INFO  (zkConnectionManagerCallback-1085-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1294377 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:41717_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1294377 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:41717_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41717_
   [junit4]   2> 1294387 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:41717_    ] o.a.s.c.Overseer Overseer (id=73651956043546628-127.0.0.1:41717_-n_0000000000) starting
   [junit4]   2> 1294400 INFO  (zkConnectionManagerCallback-1092-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1294402 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:41717_    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34393/solr ready
   [junit4]   2> 1294403 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:41717_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41717_
   [junit4]   2> 1294404 INFO  (zkCallback-1084-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1294417 INFO  (zkCallback-1091-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1294539 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:41717_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a886b63
   [junit4]   2> 1294546 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:41717_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a886b63
   [junit4]   2> 1294546 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:41717_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a886b63
   [junit4]   2> 1294547 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:41717_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001/control-001/cores
   [junit4]   2> 1294577 INFO  (zkConnectionManagerCallback-1097-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1294578 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1294579 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34393/solr ready
   [junit4]   2> 1294580 INFO  (qtp624361061-27442) [n:127.0.0.1:41717_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:41717_&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1294581 INFO  (OverseerThreadFactory-1414-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 1294689 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1294690 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1295707 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1295726 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 1295855 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1295870 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 1295870 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a886b63
   [junit4]   2> 1295870 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:34043/solr_hdfs_home
   [junit4]   2> 1295870 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 1295870 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1295871 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[hdfs://localhost:34043/solr_hdfs_home/control_collection/core_node2/data/]
   [junit4]   2> 1295872 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:34043/solr_hdfs_home/control_collection/core_node2/data/snapshot_metadata
   [junit4]   2> 1295879 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 1295879 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 1295879 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 1295898 WARN  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 1300903 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 1300904 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:34043/solr_hdfs_home/control_collection/core_node2/data
   [junit4]   2> 1300923 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:34043/solr_hdfs_home/control_collection/core_node2/data/index
   [junit4]   2> 1300931 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 1300931 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 1300931 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 1300955 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 1300956 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1522732887, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1300999 INFO  (Block report processor) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:46420 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-6e6cd7a1-edb3-4705-9ece-0ee5c8b86eb9:NORMAL:127.0.0.1:59067|RBW], ReplicaUC[[DISK]DS-82a0001b-00f8-48f1-9e10-11220ef37583:NORMAL:127.0.0.1:46420|RBW]]} size 0
   [junit4]   2> 1301002 INFO  (Block report processor) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:59067 is added to blk_1073741825_1001 size 69
   [junit4]   2> 1301009 WARN  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 1301049 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 1301050 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1301050 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
   [junit4]   2> 1301059 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1301059 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1301061 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=18, maxMergeAtOnceExplicit=45, maxMergedSegmentMB=35.0869140625, floorSegmentMB=1.73828125, forceMergeDeletesPctAllowed=25.482285968205282, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1301067 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@49d6fa07[control_collection_shard1_replica_n1] main]
   [junit4]   2> 1301068 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1301068 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1301069 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1301069 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1598109665084833792
   [junit4]   2> 1301073 INFO  (searcherExecutor-1417-thread-1-processing-n:127.0.0.1:41717_ x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@49d6fa07[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1301083 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
   [junit4]   2> 1301086 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1301086 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1301086 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:41717/control_collection_shard1_replica_n1/
   [junit4]   2> 1301086 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1301086 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:41717/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 1301086 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1301088 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:41717/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 1301140 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1301144 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=6454
   [junit4]   2> 1301147 INFO  (qtp624361061-27442) [n:127.0.0.1:41717_    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1301245 INFO  (zkCallback-1084-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1302147 INFO  (qtp624361061-27442) [n:127.0.0.1:41717_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:41717_&wt=javabin&version=2} status=0 QTime=7567
   [junit4]   2> 1302153 INFO  (zkConnectionManagerCallback-1102-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1302154 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1302154 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34393/solr ready
   [junit4]   2> 1302154 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1302155 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1302157 INFO  (OverseerCollectionConfigSetProcessor-73651956043546628-127.0.0.1:41717_-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1302157 INFO  (OverseerThreadFactory-1414-thread-2) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 1302158 WARN  (OverseerThreadFactory-1414-thread-2) [    ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 1302364 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1302365 INFO  (qtp624361061-27440) [n:127.0.0.1:41717_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=210
   [junit4]   2> 1302460 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001/shard-1-001 of type TLOG
   [junit4]   2> 1302461 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T01:27:37+04:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1302461 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1302462 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1302462 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1302462 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5064989c{/,null,AVAILABLE}
   [junit4]   2> 1302462 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3baad178{HTTP/1.1,[http/1.1]}{127.0.0.1:37524}
   [junit4]   2> 1302462 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.Server Started @1302515ms
   [junit4]   2> 1302462 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:34043/hdfs__localhost_34043__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001_tempDir-002_jetty1, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/, hostPort=37524, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001/shard-1-001/cores}
   [junit4]   2> 1302463 ERROR (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1302463 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1302463 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1302463 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1302463 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1302463 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-04-18T18:29:03.786Z
   [junit4]   2> 1302464 INFO  (zkConnectionManagerCallback-1104-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1302465 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1302465 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001/shard-1-001/solr.xml
   [junit4]   2> 1302469 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1302469 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1302470 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1a886b63, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1302474 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34393/solr
   [junit4]   2> 1302475 INFO  (zkConnectionManagerCallback-1108-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1302477 INFO  (zkConnectionManagerCallback-1110-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1302483 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:37524_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1302486 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:37524_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1302488 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:37524_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1302488 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:37524_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37524_
   [junit4]   2> 1302489 INFO  (zkCallback-1091-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1302489 INFO  (zkCallback-1084-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1302490 INFO  (zkCallback-1109-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1302490 INFO  (zkCallback-1101-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1302582 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:37524_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a886b63
   [junit4]   2> 1302591 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:37524_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a886b63
   [junit4]   2> 1302592 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:37524_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a886b63
   [junit4]   2> 1302593 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:37524_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001/shard-1-001/cores
   [junit4]   2> 1302596 INFO  (zkConnectionManagerCallback-1117-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1302597 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:37524_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1302597 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [n:127.0.0.1:37524_    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34393/solr ready
   [junit4]   2> 1302619 INFO  (qtp1272599099-27498) [n:127.0.0.1:37524_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:37524_&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1302621 INFO  (OverseerCollectionConfigSetProcessor-73651956043546628-127.0.0.1:41717_-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1302624 INFO  (OverseerThreadFactory-1414-thread-3) [    ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:37524_ for creating new replica
   [junit4]   2> 1302626 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 1303670 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1303680 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.s.IndexSchema [collection1_shard1_replica_t21] Schema name=test
   [junit4]   2> 1303771 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1303783 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_t21' using configuration from collection collection1, trusted=true
   [junit4]   2> 1303784 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_t21' (registry 'solr.core.collection1.shard1.replica_t21') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a886b63
   [junit4]   2> 1303784 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:34043/solr_hdfs_home
   [junit4]   2> 1303784 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 1303784 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1303784 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.SolrCore [[collection1_shard1_replica_t21] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001/shard-1-001/cores/collection1_shard1_replica_t21], dataDir=[hdfs://localhost:34043/solr_hdfs_home/collection1/core_node22/data/]
   [junit4]   2> 1303785 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:34043/solr_hdfs_home/collection1/core_node22/data/snapshot_metadata
   [junit4]   2> 1303793 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 1303793 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 1303793 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 1303814 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 1303815 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:34043/solr_hdfs_home/collection1/core_node22/data
   [junit4]   2> 1303833 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:34043/solr_hdfs_home/collection1/core_node22/data/index
   [junit4]   2> 1303838 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 1303838 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 1303838 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 1303844 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 1303851 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1522732887, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1303867 INFO  (Block report processor) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:59067 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-82a0001b-00f8-48f1-9e10-11220ef37583:NORMAL:127.0.0.1:46420|RBW], ReplicaUC[[DISK]DS-6e6cd7a1-edb3-4705-9ece-0ee5c8b86eb9:NORMAL:127.0.0.1:59067|RBW]]} size 0
   [junit4]   2> 1303868 INFO  (Block report processor) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:46420 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-6e6cd7a1-edb3-4705-9ece-0ee5c8b86eb9:NORMAL:127.0.0.1:59067|RBW], ReplicaUC[[DISK]DS-2d78e635-0b15-404e-b60f-bf734edccb46:NORMAL:127.0.0.1:46420|FINALIZED]]} size 0
   [junit4]   2> 1303874 WARN  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 1303911 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 1303911 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1303911 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
   [junit4]   2> 1303920 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1303920 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1303925 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=18, maxMergeAtOnceExplicit=45, maxMergedSegmentMB=35.0869140625, floorSegmentMB=1.73828125, forceMergeDeletesPctAllowed=25.482285968205282, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1303935 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.s.SolrIndexSearcher Opening [Searcher@6c3a51a8[collection1_shard1_replica_t21] main]
   [junit4]   2> 1303936 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1303936 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1303937 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1303937 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1598109668092149760
   [junit4]   2> 1303941 INFO  (searcherExecutor-1428-thread-1-processing-n:127.0.0.1:37524_ x:collection1_shard1_replica_t21 c:collection1 s:shard1) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.SolrCore [collection1_shard1_replica_t21] Registered new searcher Searcher@6c3a51a8[collection1_shard1_replica_t21] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1303944 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node22=0}, version=0}
   [junit4]   2> 1303946 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1303946 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1303946 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37524/collection1_shard1_replica_t21/
   [junit4]   2> 1303947 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1303947 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.SyncStrategy http://127.0.0.1:37524/collection1_shard1_replica_t21/ has no replicas
   [junit4]   2> 1303947 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1303947 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.ZkController collection1_shard1_replica_t21 stopping background replication from leader
   [junit4]   2> 1303950 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37524/collection1_shard1_replica_t21/ shard1
   [junit4]   2> 1304102 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1304108 INFO  (qtp1272599099-27502) [n:127.0.0.1:37524_ c:collection1 s:shard1  x:collection1_shard1_replica_t21] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1481
   [junit4]   2> 1304112 INFO  (qtp1272599099-27498) [n:127.0.0.1:37524_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:37524_&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2} status=0 QTime=1492
   [junit4]   2> 1304208 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001/shard-2-001 of type TLOG
   [junit4]   2> 1304209 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T01:27:37+04:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1304209 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1304209 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1304209 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 1304210 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2c85fbd3{/,null,AVAILABLE}
   [junit4]   2> 1304210 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2675cddf{HTTP/1.1,[http/1.1]}{127.0.0.1:36558}
   [junit4]   2> 1304210 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.e.j.s.Server Started @1304262ms
   [junit4]   2> 1304210 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:34043/hdfs__localhost_34043__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001_tempDir-002_jetty2, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/, hostPort=36558, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001/shard-2-001/cores}
   [junit4]   2> 1304210 ERROR (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1304211 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1304212 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1304212 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1304212 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1304212 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-04-18T18:29:05.535Z
   [junit4]   2> 1304212 INFO  (zkCallback-1109-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1304214 INFO  (zkConnectionManagerCallback-1119-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1304215 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1304215 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_C5031A749CF2A0AE-001/shard-2-001/solr.xml
   [junit4]   2> 1304218 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1304218 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1304219 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1a886b63, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1304223 INFO  (TEST-StressHdfsTest.test-seed#[C5031A749CF2A0AE]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34393/solr
   [junit4]   2> 1304224 INFO  (zkConnectionM

[...truncated too long message...]

Ord50, 43parent_f2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 5child1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 10parent_f1_s=PostingsFormat(name=Direct), 33parent_f2_s=PostingsFormat(name=Direct), 22parent_f2_s=PostingsFormat(name=Direct), 4child0_s=PostingsFormat(name=Memory), 12parent_f2_s=PostingsFormat(name=Memory), 52parent_f2_s=PostingsFormat(name=Memory), 42parent_f2_s=FSTOrd50, 32parent_f2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 31parent_f1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 5grand1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 6grand4_s=PostingsFormat(name=LuceneVarGapFixedInterval), 11parent_f1_s=PostingsFormat(name=Memory), 51parent_f1_s=PostingsFormat(name=Memory), 54parent_f1_s=PostingsFormat(name=Direct), 21parent_f1_s=PostingsFormat(name=Direct), 4grand2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 7grand3_s=PostingsFormat(name=LuceneVarGapFixedInterval), 8grand2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 9grand5_s=PostingsFormat(name=LuceneVarGapFixedInterval), 8grand6_s=PostingsFormat(name=LuceneVarGapFixedInterval), 9grand1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 41parent_f1_s=FSTOrd50, 37parent_f1_s=PostingsFormat(name=Memory), 40parent_f2_s=PostingsFormat(name=Direct), 50parent_f2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 47parent_f1_s=PostingsFormat(name=Direct), 14parent_f1_s=PostingsFormat(name=Direct), 57parent_f1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 20parent_f2_s=FSTOrd50, 34parent_f1_s=FSTOrd50, 24parent_f1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 44parent_f1_s=PostingsFormat(name=Memory), 30parent_f2_s=PostingsFormat(name=Memory), 5parent_f1_s=PostingsFormat(name=Direct), 6grand0_s=PostingsFormat(name=LuceneVarGapFixedInterval), 56parent_f2_s=PostingsFormat(name=Memory), 46parent_f2_s=FSTOrd50, 10parent_f2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 5child0_s=PostingsFormat(name=Direct), 8parent_f1_s=PostingsFormat(name=Memory), 27parent_f2_s=PostingsFormat(name=Memory), 2parent_f2_s=PostingsFormat(name=Direct), 4child1_s=FSTOrd50, 2grand1_s=PostingsFormat(name=Direct), 17parent_f2_s=PostingsFormat(name=Direct), 3child2_s=FSTOrd50, 16parent_f2_s=PostingsFormat(name=Memory), 26parent_f2_s=PostingsFormat(name=Direct), 1child0_s=FSTOrd50, 36parent_f2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 7grand6_s=FSTOrd50, 18parent_f1_s=FSTOrd50, 28parent_f1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 5grand0_s=FSTOrd50, 6grand5_s=FSTOrd50, 7grand2_s=FSTOrd50, 48parent_f1_s=PostingsFormat(name=Memory), 15parent_f1_s=PostingsFormat(name=Memory), 4grand3_s=FSTOrd50, 9grand8_s=PostingsFormat(name=Direct), 8grand3_s=FSTOrd50, 6parent_f1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 25parent_f1_s=PostingsFormat(name=Direct), 9grand4_s=PostingsFormat(name=Direct), 8grand7_s=FSTOrd50, 0parent_f1_s=PostingsFormat(name=Memory), 3grand2_s=FSTOrd50, 35parent_f1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 9grand0_s=PostingsFormat(name=Direct), 19parent_f2_s=FSTOrd50, 4parent_f2_s=FSTOrd50, 37parent_f2_s=PostingsFormat(name=Direct), 47parent_f2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 57parent_f2_s=FSTOrd50, 5child4_s=PostingsFormat(name=Direct), 38parent_f1_s=FSTOrd50, 5grand4_s=FSTOrd50, 6grand1_s=FSTOrd50, 5parent_f2_s=PostingsFormat(name=Memory), 46parent_f1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 16parent_f1_s=PostingsFormat(name=Direct), 56parent_f1_s=FSTOrd50, 8parent_f2_s=FSTOrd50, 39parent_f2_s=FSTOrd50, 29parent_f2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 2parent_f1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 4child2_s=PostingsFormat(name=LuceneVarGapFixedInterval), rnd_b=FSTOrd50, 27parent_f1_s=FSTOrd50, 45parent_f1_s=FSTOrd50, 17parent_f1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 2grand0_s=PostingsFormat(name=LuceneVarGapFixedInterval), 49parent_f2_s=PostingsFormat(name=Memory), 3child1_s=PostingsFormat(name=Memory), 55parent_f1_s=PostingsFormat(name=Memory), 7parent_f2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 26parent_f1_s=PostingsFormat(name=Memory), 2child0_s=PostingsFormat(name=LuceneVarGapFixedInterval), 3parent_f1_s=FSTOrd50, 36parent_f1_s=PostingsFormat(name=Direct)}, docValues:{_version_=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=1096, maxMBSortInHeap=7.217955519589295, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@53572230), locale=sr, timezone=Asia/Tel_Aviv
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=2,free=171736840,total=477626368
   [junit4]   2> NOTE: All tests run in this JVM: [TestWriterPerf, TestClusterStateMutator, BadCopyFieldTest, SuggesterWFSTTest, UpdateParamsTest, TestImplicitCoreProperties, OverriddenZkACLAndCredentialsProvidersTest, HdfsSyncSliceTest, TestFieldSortValues, TestMacroExpander, TestLockTree, DistributedSuggestComponentTest, TestSQLHandler, TestNonDefinedSimilarityFactory, NodeMarkersRegistrationTest, DistanceUnitsTest, TestFastOutputStream, ConjunctionSolrSpellCheckerTest, TestIBSimilarityFactory, ConfigureRecoveryStrategyTest, ConfigSetsAPITest, TestTrieFacet, TestCloudConsistency, TestNumericTokenStream, TermVectorComponentDistributedTest, TestPrepRecovery, TestSchemaManager, TestSolrCloudWithDelegationTokens, SpellCheckCollatorTest, TimeRoutedAliasUpdateProcessorTest, RequestHandlersTest, XmlUpdateRequestHandlerTest, ExternalFileFieldSortTest, TestDocumentBuilder, OpenCloseCoreStressTest, ScheduledTriggerIntegrationTest, VersionInfoTest, TriggerCooldownIntegrationTest, DefaultValueUpdateProcessorTest, TestConfig, MigrateRouteKeyTest, TestLuceneMatchVersion, RegexBytesRefFilterTest, TestPhraseSuggestions, InfoHandlerTest, TestTrackingShardHandlerFactory, ZkCLITest, TestConfigReload, DistribJoinFromCollectionTest, SchemaApiFailureTest, DistributedFacetPivotLongTailTest, TestRequestStatusCollectionAPI, HDFSCollectionsAPITest, TestJsonFacetsWithNestedObjects, LargeFieldTest, SpellingQueryConverterTest, TestCSVLoader, MetricsConfigTest, TestExactStatsCache, BinaryUpdateRequestHandlerTest, ConnectionReuseTest, TestSolrDeletionPolicy1, WrapperMergePolicyFactoryTest, TestCoreAdminApis, BlockJoinFacetRandomTest, CursorPagingTest, AliasIntegrationTest, TestShortCircuitedRequests, EnumFieldTest, TestCloudJSONFacetJoinDomain, ShardRoutingTest, DocValuesNotIndexedTest, TestDynamicFieldCollectionResource, ReplaceNodeNoTargetTest, IndexBasedSpellCheckerTest, TestQuerySenderNoQuery, TestNodeAddedTrigger, SortSpecParsingTest, TestSolrQueryParser, SimpleFacetsTest, BasicDistributedZkTest, TestConfigSetProperties, TokenizerChainTest, TestTestInjection, TestJavabinTupleStreamParser, TestSmileRequest, TestFieldCacheVsDocValues, OutputWriterTest, TestComponentsName, ResponseBuilderTest, TestSubQueryTransformerDistrib, FieldAnalysisRequestHandlerTest, SecurityConfHandlerTest, TestRandomFaceting, V2StandaloneTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestSimpleTextCodec, TestEmbeddedSolrServerConstructors, TestSolrCloudSnapshots, SaslZkACLProviderTest, VMParamsZkACLAndCredentialsProvidersTest, OpenExchangeRatesOrgProviderTest, TestRequestForwarding, PeerSyncReplicationTest, ReplicaListTransformerTest, TestDocBasedVersionConstraints, TestWordDelimiterFilterFactory, TestNoOpRegenerator, BadIndexSchemaTest, FullSolrCloudDistribCmdsTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=FullSolrCloudDistribCmdsTest -Dtests.seed=C5031A749CF2A0AE -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=sr -Dtests.timezone=Asia/Tel_Aviv -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J0 | FullSolrCloudDistribCmdsTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > at __randomizedtesting.SeedInfo.seed([C5031A749CF2A0AE]:0)
   [junit4] Completed [786/803 (4!)] on J0 in 7283.73s, 1 test, 2 errors <<< FAILURES!

[...truncated 66 lines...]
   [junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/temp/junit4-J0-20180418_180721_2237685864155380272581.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: Java heap space
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/heapdumps/java_pid650.hprof ...
   [junit4] Heap dump file created [598325556 bytes in 3.753 secs]
   [junit4] <<< JVM J0: EOF ----

   [junit4] JVM J0: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/temp/junit4-J0-20180418_180721_2231560352756849589943.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4]
   [junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "TEST-FullSolrCloudDistribCmdsTest.test-seed#[C5031A749CF2A0AE]-SendThread(127.0.0.1:55015)"
   [junit4] <<< JVM J0: EOF ----

[...truncated 9416 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:651: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid15388.hprof
* java_pid650.hprof

Total time: 394 minutes 59 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [JENKINS] Lucene-Solr-NightlyTests-master - Build # 1534 - Failure

Dawid Weiss-2
This is due to an out of memory exception in

   [junit4]   1> at
org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig(TestInetAddressRangeQueries.java:81)

Seems like mem codec has been picked -- should we add suppression to this test?

@SuppressCodecs({"Direct", "Memory"})

Dawid

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [JENKINS] Lucene-Solr-NightlyTests-master - Build # 1534 - Failure

Alan Woodward-3
+1.  It’s a shame that @SuppressCodecs doesn’t work on test methods, only on classes, which makes things a little trickier.

> On 20 Apr 2018, at 10:13, Dawid Weiss <[hidden email]> wrote:
>
> This is due to an out of memory exception in
>
>  [junit4]   1> at
> org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig(TestInetAddressRangeQueries.java:81)
>
> Seems like mem codec has been picked -- should we add suppression to this test?
>
> @SuppressCodecs({"Direct", "Memory"})
>
> Dawid
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [JENKINS] Lucene-Solr-NightlyTests-master - Build # 1534 - Failure

Dawid Weiss-2
> +1.  It’s a shame that @SuppressCodecs doesn’t work on test methods, only on classes, which makes things a little trickier.

The default codec is picked per-class, not per-test (part of the
reason for that is codecs are used in pre-test hooks, for example). We
could make the annotation apply to test methods too and just ignore
the test if a suppressed codec was picked at the class level. This
would be one workaround.

D.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [JENKINS] Lucene-Solr-NightlyTests-master - Build # 1534 - Failure

Robert Muir
are these memory codecs worth the trouble? i propose dropping them.

On Fri, Apr 20, 2018 at 5:29 AM, Dawid Weiss <[hidden email]> wrote:

>> +1.  It’s a shame that @SuppressCodecs doesn’t work on test methods, only on classes, which makes things a little trickier.
>
> The default codec is picked per-class, not per-test (part of the
> reason for that is codecs are used in pre-test hooks, for example). We
> could make the annotation apply to test methods too and just ignore
> the test if a suppressed codec was picked at the class level. This
> would be one workaround.
>
> D.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [JENKINS] Lucene-Solr-NightlyTests-master - Build # 1534 - Failure

Michael McCandless-2
On Fri, Apr 20, 2018 at 6:32 AM, Robert Muir <[hidden email]> wrote:
are these memory codecs worth the trouble? i propose dropping them.

+1
Reply | Threaded
Open this post in threaded view
|

Re: [JENKINS] Lucene-Solr-NightlyTests-master - Build # 1534 - Failure

Dawid Weiss-2
 I created LUCENE-8267 for this.

D.

On Sat, Apr 21, 2018 at 11:44 PM, Michael McCandless
<[hidden email]> wrote:

> On Fri, Apr 20, 2018 at 6:32 AM, Robert Muir <[hidden email]> wrote:
>>
>> are these memory codecs worth the trouble? i propose dropping them.
>
>
> +1
>
> Mike McCandless
>
> http://blog.mikemccandless.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]