[JENKINS] Lucene-Solr-Tests-master - Build # 2422 - Unstable

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2422 - Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2422/

3 tests failed.
FAILED:  org.apache.lucene.index.TestIndexWriterWithThreads.testCloseWithThreads

Error Message:
thread failed before indexing a single document

Stack Trace:
java.lang.AssertionError: thread failed before indexing a single document
        at __randomizedtesting.SeedInfo.seed([F74EDD534544EE8:12B0C4A257A1544E]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.lucene.index.TestIndexWriterWithThreads.testCloseWithThreads(TestIndexWriterWithThreads.java:223)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.ConcurrentCreateRoutedAliasTest.testConcurrentCreateRoutedAliasComplex

Error Message:
concurrent alias creation failed org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:35643/solr: KeeperErrorCode = Session expired for /configs/_default

Stack Trace:
java.lang.AssertionError: concurrent alias creation failed org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:35643/solr: KeeperErrorCode = Session expired for /configs/_default
        at __randomizedtesting.SeedInfo.seed([79DE20C20FACE14D:138C5C8B4DF222D6]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNull(Assert.java:551)
        at org.apache.solr.cloud.ConcurrentCreateRoutedAliasTest.testConcurrentCreateRoutedAliasComplex(ConcurrentCreateRoutedAliasTest.java:133)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.TestTlogReplica.testCreateDelete {seed=[79DE20C20FACE14D:D6986D72CD9ED251]}

Error Message:
Could not load collection from ZK: tlog_replica_test_create_delete

Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: tlog_replica_test_create_delete
        at __randomizedtesting.SeedInfo.seed([79DE20C20FACE14D:D6986D72CD9ED251]:0)
        at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1232)
        at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:670)
        at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:148)
        at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:131)
        at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:117)
        at org.apache.solr.cloud.SolrCloudTestCase.getCollectionState(SolrCloudTestCase.java:247)
        at org.apache.solr.cloud.TestTlogReplica.testCreateDelete(TestTlogReplica.java:189)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/tlog_replica_test_create_delete/state.json
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
        at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
        at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)
        at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
        at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)
        at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1244)
        at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1230)
        ... 45 more




Build Log:
[...truncated 895 lines...]
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterWithThreads
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestIndexWriterWithThreads -Dtests.method=testCloseWithThreads -Dtests.seed=F74EDD534544EE8 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=el -Dtests.timezone=America/Tortola -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.64s J0 | TestIndexWriterWithThreads.testCloseWithThreads <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: thread failed before indexing a single document
   [junit4]    > at __randomizedtesting.SeedInfo.seed([F74EDD534544EE8:12B0C4A257A1544E]:0)
   [junit4]    > at org.apache.lucene.index.TestIndexWriterWithThreads.testCloseWithThreads(TestIndexWriterWithThreads.java:223)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   1> org.apache.lucene.store.LockObtainFailedException: lock instance already obtained: (dir=RAMDirectory@64acc9e2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@609351af, lockName=write.lock)
   [junit4]   1> at org.apache.lucene.store.SingleInstanceLockFactory.obtainLock(SingleInstanceLockFactory.java:44)
   [junit4]   1> at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45)
   [junit4]   1> at org.apache.lucene.store.FilterDirectory.obtainLock(FilterDirectory.java:104)
   [junit4]   1> at org.apache.lucene.store.MockDirectoryWrapper.obtainLock(MockDirectoryWrapper.java:1049)
   [junit4]   1> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:948)
   [junit4]   1> at org.apache.lucene.index.TestIndexWriterWithThreads$DelayedIndexAndCloseRunnable.run(TestIndexWriterWithThreads.java:564)
   [junit4] IGNOR/A 0.19s J0 | TestIndexWriterWithThreads.testOpenTwoIndexWritersOnDifferentThreads
   [junit4]    > Assumption #1: aborting test: timeout obtaining lock
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {date=PostingsFormat(name=LuceneFixedGap), field=PostingsFormat(name=LuceneFixedGap), docid=FSTOrd50, titleTokenized=PostingsFormat(name=Asserting), body=PostingsFormat(name=LuceneFixedGap), title=PostingsFormat(name=Direct)}, docValues:{dv=DocValuesFormat(name=Memory), docid_intDV=DocValuesFormat(name=Asserting), field=DocValuesFormat(name=Memory), titleDV=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1513, maxMBSortInHeap=6.658256311042419, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@43890498), locale=el, timezone=America/Tortola
   [junit4]   2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=45376944,total=264765440
   [junit4]   2> NOTE: All tests run in this JVM: [TestSpansEnum, TestOmitNorms, TestTermQuery, TestIndependenceSaturated, TestSloppyPhraseQuery, TestNoMergeScheduler, TestAutomatonQuery, TestSpanNotQuery, TestSort, TestPostingsOffsets, TestSimpleSearchEquivalence, TestFuzzyQuery, TestReadOnlyIndex, TestDocIdsWriter, TestTragicIndexWriterDeadlock, TestPerSegmentDeletes, TestIndexInput, TestNoDeletionPolicy, TestIntsRef, TestPhrasePrefixQuery, TestQueryBuilder, TestDocIdSetBuilder, TestAxiomaticF2EXP, TestLogMergePolicy, TestNormsFieldExistsQuery, TestMutableValues, TestWANDScorer, TestPrefixRandom, Test2BSortedDocValuesFixedSorted, TestUTF32ToUTF8, TestStandardAnalyzer, TestIntBlockPool, TestFrequencyTrackingRingBuffer, TestMSBRadixSorter, TestSortedSetSelector, TestLucene70NormsFormat, TestCodecs, TestFlex, TestSnapshotDeletionPolicy, TestRegExp, TestIOUtils, TestNotDocIdSet, TestCharArrayMap, TestBasicModelIne, TestDocInverterPerFieldErrorInfo, TestBlockPostingsFormat, TestDocsAndPositions, TestAxiomaticF3LOG, TestBooleanRewrites, TestSpanMultiTermQueryWrapper, TestDocValuesIndexing, TestSimpleExplanationsWithFillerDocs, TestDelegatingAnalyzerWrapper, TestPersistentSnapshotDeletionPolicy, TestDoubleRangeFieldQueries, Test2BBinaryDocValues, TestFilterLeafReader, TestForUtil, TestBagOfPositions, TestIndexWriterConfig, TestGrowableByteArrayDataOutput, TestTwoPhaseCommitTool, TestTotalHitCountCollector, TestNeverDelete, TestPerFieldPostingsFormat, TestSegmentMerger, TestSparseFixedBitSet, TestIndexCommit, Test2BPositions, TestSizeBoundedForceMerge, TestLMJelinekMercerSimilarity, TestBooleanMinShouldMatch, TestFilterDirectoryReader, LimitedFiniteStringsIteratorTest, TestSpanFirstQuery, TestFieldInvertState, TestParallelTermEnum, TestDemo, TestConstantScoreQuery, TestIndexWriterThreadsToSegments, TestIndexReaderClose, TestIndexWriter, TestIndexWriterMerging, TestFSTs, TestBytesStore, TestIndexWriterWithThreads]
   [junit4] Completed [196/486 (1!)] on J0 in 7.92s, 12 tests, 1 failure, 1 skipped <<< FAILURES!

[...truncated 968 lines...]
   [junit4] JVM J0: stdout was not empty, see: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/temp/junit4-J0-20180313_103440_5258883702234451565957.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] codec: FastCompressingStoredFields, pf: FST50, dvf: Memory
   [junit4] <<< JVM J0: EOF ----

[...truncated 10379 lines...]
   [junit4] Suite: org.apache.solr.cloud.ConcurrentCreateRoutedAliasTest
   [junit4]   2> 450363 INFO  (SUITE-ConcurrentCreateRoutedAliasTest-seed#[79DE20C20FACE14D]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/init-core-data-001
   [junit4]   2> 450364 WARN  (SUITE-ConcurrentCreateRoutedAliasTest-seed#[79DE20C20FACE14D]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=12 numCloses=12
   [junit4]   2> 450391 INFO  (SUITE-ConcurrentCreateRoutedAliasTest-seed#[79DE20C20FACE14D]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 450393 INFO  (SUITE-ConcurrentCreateRoutedAliasTest-seed#[79DE20C20FACE14D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 450395 INFO  (TEST-ConcurrentCreateRoutedAliasTest.testConcurrentCreateRoutedAliasMinimal-seed#[79DE20C20FACE14D]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testConcurrentCreateRoutedAliasMinimal
   [junit4]   2> 450395 INFO  (TEST-ConcurrentCreateRoutedAliasTest.testConcurrentCreateRoutedAliasMinimal-seed#[79DE20C20FACE14D]) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001
   [junit4]   2> 450415 INFO  (TEST-ConcurrentCreateRoutedAliasTest.testConcurrentCreateRoutedAliasMinimal-seed#[79DE20C20FACE14D]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 450416 INFO  (Thread-847) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 450416 INFO  (Thread-847) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 450417 ERROR (Thread-847) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 450517 INFO  (TEST-ConcurrentCreateRoutedAliasTest.testConcurrentCreateRoutedAliasMinimal-seed#[79DE20C20FACE14D]) [    ] o.a.s.c.ZkTestServer start zk server on port:44208
   [junit4]   2> 450538 INFO  (zkConnectionManagerCallback-413-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 450611 INFO  (jetty-launcher-410-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 450612 INFO  (jetty-launcher-410-thread-2) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 450613 INFO  (jetty-launcher-410-thread-3) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 450631 INFO  (jetty-launcher-410-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 450631 INFO  (jetty-launcher-410-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 450631 INFO  (jetty-launcher-410-thread-1) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 450632 INFO  (jetty-launcher-410-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@17d23b57{/solr,null,AVAILABLE}
   [junit4]   2> 450651 INFO  (jetty-launcher-410-thread-4) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 450684 INFO  (jetty-launcher-410-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 450684 INFO  (jetty-launcher-410-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 450684 INFO  (jetty-launcher-410-thread-2) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 450684 INFO  (jetty-launcher-410-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1c1084af{/solr,null,AVAILABLE}
   [junit4]   2> 450685 INFO  (jetty-launcher-410-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@55688d03{HTTP/1.1,[http/1.1]}{127.0.0.1:38034}
   [junit4]   2> 450685 INFO  (jetty-launcher-410-thread-2) [    ] o.e.j.s.Server Started @460888ms
   [junit4]   2> 450685 INFO  (jetty-launcher-410-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38034}
   [junit4]   2> 450685 ERROR (jetty-launcher-410-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 450685 INFO  (jetty-launcher-410-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 450685 INFO  (jetty-launcher-410-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 450685 INFO  (jetty-launcher-410-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 450685 INFO  (jetty-launcher-410-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-13T12:07:26.782Z
   [junit4]   2> 450737 INFO  (jetty-launcher-410-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 450737 INFO  (jetty-launcher-410-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 450737 INFO  (jetty-launcher-410-thread-3) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 450737 INFO  (jetty-launcher-410-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7c224374{/solr,null,AVAILABLE}
   [junit4]   2> 450738 INFO  (jetty-launcher-410-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@40aa3808{HTTP/1.1,[http/1.1]}{127.0.0.1:37093}
   [junit4]   2> 450738 INFO  (jetty-launcher-410-thread-3) [    ] o.e.j.s.Server Started @460941ms
   [junit4]   2> 450738 INFO  (jetty-launcher-410-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37093}
   [junit4]   2> 450738 ERROR (jetty-launcher-410-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 450738 INFO  (jetty-launcher-410-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 450738 INFO  (jetty-launcher-410-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 450738 INFO  (jetty-launcher-410-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 450738 INFO  (jetty-launcher-410-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-13T12:07:26.835Z
   [junit4]   2> 450814 INFO  (jetty-launcher-410-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@501787a{HTTP/1.1,[http/1.1]}{127.0.0.1:42215}
   [junit4]   2> 450814 INFO  (jetty-launcher-410-thread-1) [    ] o.e.j.s.Server Started @461018ms
   [junit4]   2> 450814 INFO  (jetty-launcher-410-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42215}
   [junit4]   2> 450814 ERROR (jetty-launcher-410-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 450814 INFO  (jetty-launcher-410-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 450814 INFO  (jetty-launcher-410-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 450814 INFO  (jetty-launcher-410-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 450815 INFO  (jetty-launcher-410-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-13T12:07:26.911Z
   [junit4]   2> 450987 INFO  (zkConnectionManagerCallback-415-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 451026 INFO  (zkConnectionManagerCallback-417-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 451041 INFO  (jetty-launcher-410-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 451041 INFO  (jetty-launcher-410-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 451042 INFO  (jetty-launcher-410-thread-4) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 451042 INFO  (jetty-launcher-410-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@509842{/solr,null,AVAILABLE}
   [junit4]   2> 451058 INFO  (zkConnectionManagerCallback-419-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 451058 INFO  (jetty-launcher-410-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 451058 INFO  (jetty-launcher-410-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 451079 INFO  (jetty-launcher-410-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1119426a{HTTP/1.1,[http/1.1]}{127.0.0.1:33472}
   [junit4]   2> 451079 INFO  (jetty-launcher-410-thread-4) [    ] o.e.j.s.Server Started @461282ms
   [junit4]   2> 451079 INFO  (jetty-launcher-410-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33472}
   [junit4]   2> 451079 ERROR (jetty-launcher-410-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 451079 INFO  (jetty-launcher-410-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 451079 INFO  (jetty-launcher-410-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 451079 INFO  (jetty-launcher-410-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 451079 INFO  (jetty-launcher-410-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-13T12:07:27.176Z
   [junit4]   2> 451108 INFO  (jetty-launcher-410-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 451126 INFO  (zkConnectionManagerCallback-421-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 451144 INFO  (jetty-launcher-410-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 451144 INFO  (jetty-launcher-410-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44208/solr
   [junit4]   2> 451173 INFO  (jetty-launcher-410-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44208/solr
   [junit4]   2> 451204 INFO  (zkConnectionManagerCallback-426-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 451207 INFO  (zkConnectionManagerCallback-429-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 451208 INFO  (jetty-launcher-410-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44208/solr
   [junit4]   2> 451229 INFO  (jetty-launcher-410-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44208/solr
   [junit4]   2> 451230 INFO  (zkConnectionManagerCallback-434-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 451255 INFO  (zkConnectionManagerCallback-437-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 451255 INFO  (zkConnectionManagerCallback-439-thread-1-processing-n:127.0.0.1:37093_solr) [n:127.0.0.1:37093_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 451256 INFO  (zkConnectionManagerCallback-441-thread-1-processing-n:127.0.0.1:38034_solr) [n:127.0.0.1:38034_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 451270 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100c3849a9b0007, likely client has closed socket
   [junit4]   2> 451270 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100c3849a9b0008, likely client has closed socket
   [junit4]   2> 451314 INFO  (zkConnectionManagerCallback-445-thread-1-processing-n:127.0.0.1:42215_solr) [n:127.0.0.1:42215_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 451314 INFO  (zkConnectionManagerCallback-443-thread-1-processing-n:127.0.0.1:33472_solr) [n:127.0.0.1:33472_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 451783 INFO  (jetty-launcher-410-thread-3) [n:127.0.0.1:37093_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 451784 INFO  (jetty-launcher-410-thread-3) [n:127.0.0.1:37093_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37093_solr
   [junit4]   2> 451798 INFO  (jetty-launcher-410-thread-2) [n:127.0.0.1:38034_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 451799 INFO  (jetty-launcher-410-thread-3) [n:127.0.0.1:37093_solr    ] o.a.s.c.Overseer Overseer (id=72272568334876681-127.0.0.1:37093_solr-n_0000000000) starting
   [junit4]   2> 451800 INFO  (jetty-launcher-410-thread-2) [n:127.0.0.1:38034_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38034_solr
   [junit4]   2> 451954 INFO  (zkCallback-438-thread-1-processing-n:127.0.0.1:37093_solr) [n:127.0.0.1:37093_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 451955 INFO  (zkCallback-440-thread-1-processing-n:127.0.0.1:38034_solr) [n:127.0.0.1:38034_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 452045 INFO  (zkConnectionManagerCallback-453-thread-1-processing-n:127.0.0.1:37093_solr) [n:127.0.0.1:37093_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 452090 INFO  (jetty-launcher-410-thread-3) [n:127.0.0.1:37093_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 452091 INFO  (jetty-launcher-410-thread-3) [n:127.0.0.1:37093_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44208/solr ready
   [junit4]   2> 452112 INFO  (jetty-launcher-410-thread-3) [n:127.0.0.1:37093_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37093_solr
   [junit4]   2> 452155 INFO  (zkCallback-440-thread-1-processing-n:127.0.0.1:38034_solr) [n:127.0.0.1:38034_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 452195 INFO  (jetty-launcher-410-thread-2) [n:127.0.0.1:38034_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38034.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e5ebd63
   [junit4]   2> 452216 INFO  (zkCallback-452-thread-1-processing-n:127.0.0.1:37093_solr) [n:127.0.0.1:37093_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 452217 INFO  (zkCallback-438-thread-1-processing-n:127.0.0.1:37093_solr) [n:127.0.0.1:37093_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 452304 INFO  (jetty-launcher-410-thread-2) [n:127.0.0.1:38034_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38034.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e5ebd63
   [junit4]   2> 452305 INFO  (jetty-launcher-410-thread-2) [n:127.0.0.1:38034_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38034.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e5ebd63
   [junit4]   2> 452338 INFO  (jetty-launcher-410-thread-2) [n:127.0.0.1:38034_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node3/.
   [junit4]   2> 452553 INFO  (zkConnectionManagerCallback-457-thread-1-processing-n:127.0.0.1:38034_solr) [n:127.0.0.1:38034_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 452569 INFO  (jetty-launcher-410-thread-3) [n:127.0.0.1:37093_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37093.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e5ebd63
   [junit4]   2> 452662 INFO  (jetty-launcher-410-thread-3) [n:127.0.0.1:37093_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37093.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e5ebd63
   [junit4]   2> 452662 INFO  (jetty-launcher-410-thread-3) [n:127.0.0.1:37093_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37093.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e5ebd63
   [junit4]   2> 452676 INFO  (jetty-launcher-410-thread-3) [n:127.0.0.1:37093_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/.
   [junit4]   2> 452704 INFO  (jetty-launcher-410-thread-1) [n:127.0.0.1:42215_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 452705 INFO  (jetty-launcher-410-thread-2) [n:127.0.0.1:38034_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 452706 INFO  (jetty-launcher-410-thread-2) [n:127.0.0.1:38034_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44208/solr ready
   [junit4]   2> 452721 INFO  (jetty-launcher-410-thread-1) [n:127.0.0.1:42215_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 452723 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:33472_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 452736 INFO  (jetty-launcher-410-thread-1) [n:127.0.0.1:42215_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42215_solr
   [junit4]   2> 452801 INFO  (zkCallback-440-thread-1-processing-n:127.0.0.1:38034_solr) [n:127.0.0.1:38034_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 452809 INFO  (zkCallback-438-thread-1-processing-n:127.0.0.1:37093_solr) [n:127.0.0.1:37093_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 452863 INFO  (zkCallback-444-thread-1-processing-n:127.0.0.1:42215_solr) [n:127.0.0.1:42215_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 452863 INFO  (zkCallback-452-thread-1-processing-n:127.0.0.1:37093_solr) [n:127.0.0.1:37093_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 452864 INFO  (zkCallback-456-thread-1-processing-n:127.0.0.1:38034_solr) [n:127.0.0.1:38034_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 452864 INFO  (zkCallback-442-thread-1-processing-n:127.0.0.1:33472_solr) [n:127.0.0.1:33472_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 452865 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:33472_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 452866 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:33472_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33472_solr
   [junit4]   2> 452870 INFO  (zkCallback-456-thread-1-processing-n:127.0.0.1:38034_solr) [n:127.0.0.1:38034_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 452870 INFO  (zkCallback-444-thread-1-processing-n:127.0.0.1:42215_solr) [n:127.0.0.1:42215_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 452870 INFO  (zkCallback-452-thread-1-processing-n:127.0.0.1:37093_solr) [n:127.0.0.1:37093_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 452898 INFO  (zkCallback-440-thread-1-processing-n:127.0.0.1:38034_solr) [n:127.0.0.1:38034_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 452907 INFO  (zkCallback-442-thread-1-processing-n:127.0.0.1:33472_solr) [n:127.0.0.1:33472_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 452907 INFO  (zkCallback-438-thread-2-processing-n:127.0.0.1:37093_solr) [n:127.0.0.1:37093_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 453650 INFO  (jetty-launcher-410-thread-1) [n:127.0.0.1:42215_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42215.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e5ebd63
   [junit4]   2> 453686 INFO  (jetty-launcher-410-thread-1) [n:127.0.0.1:42215_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42215.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e5ebd63
   [junit4]   2> 453686 INFO  (jetty-launcher-410-thread-1) [n:127.0.0.1:42215_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42215.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e5ebd63
   [junit4]   2> 453687 INFO  (jetty-launcher-410-thread-1) [n:127.0.0.1:42215_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/.
   [junit4]   2> 453737 INFO  (zkConnectionManagerCallback-461-thread-1-processing-n:127.0.0.1:42215_solr) [n:127.0.0.1:42215_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 453738 INFO  (jetty-launcher-410-thread-1) [n:127.0.0.1:42215_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 453739 INFO  (jetty-launcher-410-thread-1) [n:127.0.0.1:42215_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44208/solr ready
   [junit4]   2> 453812 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:33472_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33472.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e5ebd63
   [junit4]   2> 453839 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:33472_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33472.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e5ebd63
   [junit4]   2> 453839 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:33472_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33472.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e5ebd63
   [junit4]   2> 453840 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:33472_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node1/.
   [junit4]   2> 453858 INFO  (zkConnectionManagerCallback-465-thread-1-processing-n:127.0.0.1:33472_solr) [n:127.0.0.1:33472_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 453871 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:33472_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 453872 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:33472_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44208/solr ready
   [junit4]   2> 454028 INFO  (zkConnectionManagerCallback-467-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 454071 INFO  (qtp1019147787-2178) [n:127.0.0.1:38034_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias2&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
   [junit4]   2> 454071 INFO  (qtp1019147787-2180) [n:127.0.0.1:38034_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias1&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
   [junit4]   2> 454073 INFO  (qtp1019147787-2182) [n:127.0.0.1:38034_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias3&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
   [junit4]   2> 454086 INFO  (qtp1019147787-2181) [n:127.0.0.1:38034_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias0&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
   [junit4]   2> 454165 INFO  (OverseerThreadFactory-733-thread-2-processing-n:127.0.0.1:37093_solr) [n:127.0.0.1:37093_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias3_2018-03-13_12
   [junit4]   2> 454165 INFO  (OverseerThreadFactory-733-thread-3-processing-n:127.0.0.1:37093_solr) [n:127.0.0.1:37093_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias2_2018-03-13_12
   [junit4]   2> 454165 INFO  (OverseerThreadFactory-733-thread-1-processing-n:127.0.0.1:37093_solr) [n:127.0.0.1:37093_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias1_2018-03-13_12
   [junit4]   2> 454183 INFO  (OverseerThreadFactory-733-thread-4-processing-n:127.0.0.1:37093_solr) [n:127.0.0.1:37093_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias0_2018-03-13_12
   [junit4]   2> 454567 INFO  (OverseerStateUpdate-72272568334876681-127.0.0.1:37093_solr-n_0000000000) [n:127.0.0.1:37093_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testAlias1_2018-03-13_12",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testAlias1_2018-03-13_12_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:42215/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 454675 INFO  (OverseerStateUpdate-72272568334876681-127.0.0.1:37093_solr-n_0000000000) [n:127.0.0.1:37093_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testAlias2_2018-03-13_12",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testAlias2_2018-03-13_12_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:42215/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 454913 INFO  (OverseerStateUpdate-72272568334876681-127.0.0.1:37093_solr-n_0000000000) [n:127.0.0.1:37093_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testAlias0_2018-03-13_12",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testAlias0_2018-03-13_12_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:37093/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 454914 INFO  (OverseerStateUpdate-72272568334876681-127.0.0.1:37093_solr-n_0000000000) [n:127.0.0.1:37093_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testAlias3_2018-03-13_12",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testAlias3_2018-03-13_12_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:42215/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 455015 INFO  (qtp1664186031-2170) [n:127.0.0.1:42215_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=_default&newCollection=true&collection=testAlias1_2018-03-13_12&version=2&replicaType=NRT&property.routedAliasName=testAlias1&coreNodeName=core_node2&name=testAlias1_2018-03-13_12_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin
   [junit4]   2> 455016 INFO  (qtp1664186031-2170) [n:127.0.0.1:42215_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 455029 INFO  (qtp1664186031-2171) [n:127.0.0.1:42215_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=_default&newCollection=true&collection=testAlias2_2018-03-13_12&version=2&replicaType=NRT&property.routedAliasName=testAlias2&coreNodeName=core_node2&name=testAlias2_2018-03-13_12_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin
   [junit4]   2> 455114 INFO  (qtp1664186031-2176) [n:127.0.0.1:42215_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=_default&newCollection=true&collection=testAlias3_2018-03-13_12&version=2&replicaType=NRT&property.routedAliasName=testAlias3&coreNodeName=core_node2&name=testAlias3_2018-03-13_12_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin
   [junit4]   2> 455115 INFO  (qtp2101575173-2188) [n:127.0.0.1:37093_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=_default&newCollection=true&collection=testAlias0_2018-03-13_12&version=2&replicaType=NRT&property.routedAliasName=testAlias0&coreNodeName=core_node2&name=testAlias0_2018-03-13_12_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin
   [junit4]   2> 455116 INFO  (qtp2101575173-2188) [n:127.0.0.1:37093_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 455244 INFO  (zkCallback-444-thread-1-processing-n:127.0.0.1:42215_solr) [n:127.0.0.1:42215_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testAlias2_2018-03-13_12/state.json] for collection [testAlias2_2018-03-13_12] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 455245 INFO  (zkCallback-444-thread-2-processing-n:127.0.0.1:42215_solr) [n:127.0.0.1:42215_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testAlias2_2018-03-13_12/state.json] for collection [testAlias2_2018-03-13_12] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 455246 INFO  (zkCallback-444-thread-1-processing-n:127.0.0.1:42215_solr) [n:127.0.0.1:42215_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testAlias1_2018-03-13_12/state.json] for collection [testAlias1_2018-03-13_12] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 455246 INFO  (zkCallback-444-thread-2-processing-n:127.0.0.1:42215_solr) [n:127.0.0.1:42215_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testAlias1_2018-03-13_12/state.json] for collection [testAlias1_2018-03-13_12] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 455246 INFO  (zkCallback-444-thread-2-processing-n:127.0.0.1:42215_solr) [n:127.0.0.1:42215_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testAlias3_2018-03-13_12/state.json] for collection [testAlias3_2018-03-13_12] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 455250 INFO  (zkCallback-438-thread-1-processing-n:127.0.0.1:37093_solr) [n:127.0.0.1:37093_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testAlias0_2018-03-13_12/state.json] for collection [testAlias0_2018-03-13_12] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 455250 INFO  (zkCallback-438-thread-2-processing-n:127.0.0.1:37093_solr) [n:127.0.0.1:37093_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testAlias0_2018-03-13_12/state.json] for collection [testAlias0_2018-03-13_12] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 455246 INFO  (zkCallback-444-thread-1-processing-n:127.0.0.1:42215_solr) [n:127.0.0.1:42215_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testAlias3_2018-03-13_12/state.json] for collection [testAlias3_2018-03-13_12] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 456119 INFO  (qtp1664186031-2171) [n:127.0.0.1:42215_solr c:testAlias2_2018-03-13_12 s:shard1 r:core_node2 x:testAlias2_2018-03-13_12_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 1
   [junit4]   2> 456151 INFO  (qtp1664186031-2171) [n:127.0.0.1:42215_solr c:testAlias2_2018-03-13_12 s:shard1 r:core_node2 x:testAlias2_2018-03-13_12_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 1
   [junit4]   2> 456152 WARN  (qtp1664186031-2171) [n:127.0.0.1:42215_solr c:testAlias2_2018-03-13_12 s:shard1 r:core_node2 x:testAlias2_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias2_2018-03-13_12_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias2_2018-03-13_12_shard1_replica_n1/../../../../contrib/extraction/lib
   [junit4]   2> 456166 WARN  (qtp1664186031-2171) [n:127.0.0.1:42215_solr c:testAlias2_2018-03-13_12 s:shard1 r:core_node2 x:testAlias2_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias2_2018-03-13_12_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias2_2018-03-13_12_shard1_replica_n1/../../../../dist
   [junit4]   2> 456167 WARN  (qtp1664186031-2171) [n:127.0.0.1:42215_solr c:testAlias2_2018-03-13_12 s:shard1 r:core_node2 x:testAlias2_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias2_2018-03-13_12_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias2_2018-03-13_12_shard1_replica_n1/../../../../contrib/clustering/lib
   [junit4]   2> 456167 WARN  (qtp1664186031-2171) [n:127.0.0.1:42215_solr c:testAlias2_2018-03-13_12 s:shard1 r:core_node2 x:testAlias2_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias2_2018-03-13_12_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias2_2018-03-13_12_shard1_replica_n1/../../../../dist
   [junit4]   2> 456167 WARN  (qtp1664186031-2171) [n:127.0.0.1:42215_solr c:testAlias2_2018-03-13_12 s:shard1 r:core_node2 x:testAlias2_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias2_2018-03-13_12_shard1_replica_n1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias2_2018-03-13_12_shard1_replica_n1/../../../../contrib/langid/lib
   [junit4]   2> 456167 WARN  (qtp1664186031-2171) [n:127.0.0.1:42215_solr c:testAlias2_2018-03-13_12 s:shard1 r:core_node2 x:testAlias2_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias2_2018-03-13_12_shard1_replica_n1/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias2_2018-03-13_12_shard1_replica_n1/../../../../dist
   [junit4]   2> 456167 WARN  (qtp1664186031-2171) [n:127.0.0.1:42215_solr c:testAlias2_2018-03-13_12 s:shard1 r:core_node2 x:testAlias2_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias2_2018-03-13_12_shard1_replica_n1/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias2_2018-03-13_12_shard1_replica_n1/../../../../contrib/velocity/lib
   [junit4]   2> 456168 WARN  (qtp1664186031-2171) [n:127.0.0.1:42215_solr c:testAlias2_2018-03-13_12 s:shard1 r:core_node2 x:testAlias2_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias2_2018-03-13_12_shard1_replica_n1/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias2_2018-03-13_12_shard1_replica_n1/../../../../dist
   [junit4]   2> 456208 INFO  (qtp1664186031-2171) [n:127.0.0.1:42215_solr c:testAlias2_2018-03-13_12 s:shard1 r:core_node2 x:testAlias2_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 456193 INFO  (qtp1664186031-2170) [n:127.0.0.1:42215_solr c:testAlias1_2018-03-13_12 s:shard1 r:core_node2 x:testAlias1_2018-03-13_12_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 1
   [junit4]   2> 456280 INFO  (qtp1664186031-2170) [n:127.0.0.1:42215_solr c:testAlias1_2018-03-13_12 s:shard1 r:core_node2 x:testAlias1_2018-03-13_12_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 1
   [junit4]   2> 456281 WARN  (qtp1664186031-2170) [n:127.0.0.1:42215_solr c:testAlias1_2018-03-13_12 s:shard1 r:core_node2 x:testAlias1_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias1_2018-03-13_12_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias1_2018-03-13_12_shard1_replica_n1/../../../../contrib/extraction/lib
   [junit4]   2> 456315 WARN  (qtp1664186031-2170) [n:127.0.0.1:42215_solr c:testAlias1_2018-03-13_12 s:shard1 r:core_node2 x:testAlias1_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias1_2018-03-13_12_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias1_2018-03-13_12_shard1_replica_n1/../../../../dist
   [junit4]   2> 456315 WARN  (qtp1664186031-2170) [n:127.0.0.1:42215_solr c:testAlias1_2018-03-13_12 s:shard1 r:core_node2 x:testAlias1_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias1_2018-03-13_12_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias1_2018-03-13_12_shard1_replica_n1/../../../../contrib/clustering/lib
   [junit4]   2> 456315 WARN  (qtp1664186031-2170) [n:127.0.0.1:42215_solr c:testAlias1_2018-03-13_12 s:shard1 r:core_node2 x:testAlias1_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias1_2018-03-13_12_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias1_2018-03-13_12_shard1_replica_n1/../../../../dist
   [junit4]   2> 456315 WARN  (qtp1664186031-2170) [n:127.0.0.1:42215_solr c:testAlias1_2018-03-13_12 s:shard1 r:core_node2 x:testAlias1_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias1_2018-03-13_12_shard1_replica_n1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias1_2018-03-13_12_shard1_replica_n1/../../../../contrib/langid/lib
   [junit4]   2> 456316 WARN  (qtp1664186031-2170) [n:127.0.0.1:42215_solr c:testAlias1_2018-03-13_12 s:shard1 r:core_node2 x:testAlias1_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias1_2018-03-13_12_shard1_replica_n1/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias1_2018-03-13_12_shard1_replica_n1/../../../../dist
   [junit4]   2> 456316 WARN  (qtp1664186031-2170) [n:127.0.0.1:42215_solr c:testAlias1_2018-03-13_12 s:shard1 r:core_node2 x:testAlias1_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias1_2018-03-13_12_shard1_replica_n1/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias1_2018-03-13_12_shard1_replica_n1/../../../../contrib/velocity/lib
   [junit4]   2> 456316 WARN  (qtp1664186031-2170) [n:127.0.0.1:42215_solr c:testAlias1_2018-03-13_12 s:shard1 r:core_node2 x:testAlias1_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias1_2018-03-13_12_shard1_replica_n1/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias1_2018-03-13_12_shard1_replica_n1/../../../../dist
   [junit4]   2> 456351 INFO  (qtp1664186031-2170) [n:127.0.0.1:42215_solr c:testAlias1_2018-03-13_12 s:shard1 r:core_node2 x:testAlias1_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 456385 INFO  (qtp2101575173-2188) [n:127.0.0.1:37093_solr c:testAlias0_2018-03-13_12 s:shard1 r:core_node2 x:testAlias0_2018-03-13_12_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 1
   [junit4]   2> 456385 INFO  (qtp2101575173-2188) [n:127.0.0.1:37093_solr c:testAlias0_2018-03-13_12 s:shard1 r:core_node2 x:testAlias0_2018-03-13_12_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 1
   [junit4]   2> 457005 WARN  (qtp2101575173-2188) [n:127.0.0.1:37093_solr c:testAlias0_2018-03-13_12 s:shard1 r:core_node2 x:testAlias0_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/testAlias0_2018-03-13_12_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/testAlias0_2018-03-13_12_shard1_replica_n1/../../../../contrib/extraction/lib
   [junit4]   2> 457005 WARN  (qtp2101575173-2188) [n:127.0.0.1:37093_solr c:testAlias0_2018-03-13_12 s:shard1 r:core_node2 x:testAlias0_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/testAlias0_2018-03-13_12_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/testAlias0_2018-03-13_12_shard1_replica_n1/../../../../dist
   [junit4]   2> 457005 WARN  (qtp2101575173-2188) [n:127.0.0.1:37093_solr c:testAlias0_2018-03-13_12 s:shard1 r:core_node2 x:testAlias0_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/testAlias0_2018-03-13_12_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/testAlias0_2018-03-13_12_shard1_replica_n1/../../../../contrib/clustering/lib
   [junit4]   2> 457005 WARN  (qtp2101575173-2188) [n:127.0.0.1:37093_solr c:testAlias0_2018-03-13_12 s:shard1 r:core_node2 x:testAlias0_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/testAlias0_2018-03-13_12_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/testAlias0_2018-03-13_12_shard1_replica_n1/../../../../dist
   [junit4]   2> 457005 WARN  (qtp2101575173-2188) [n:127.0.0.1:37093_solr c:testAlias0_2018-03-13_12 s:shard1 r:core_node2 x:testAlias0_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/testAlias0_2018-03-13_12_shard1_replica_n1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/testAlias0_2018-03-13_12_shard1_replica_n1/../../../../contrib/langid/lib
   [junit4]   2> 457006 WARN  (qtp2101575173-2188) [n:127.0.0.1:37093_solr c:testAlias0_2018-03-13_12 s:shard1 r:core_node2 x:testAlias0_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/testAlias0_2018-03-13_12_shard1_replica_n1/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/testAlias0_2018-03-13_12_shard1_replica_n1/../../../../dist
   [junit4]   2> 457006 WARN  (qtp2101575173-2188) [n:127.0.0.1:37093_solr c:testAlias0_2018-03-13_12 s:shard1 r:core_node2 x:testAlias0_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/testAlias0_2018-03-13_12_shard1_replica_n1/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/testAlias0_2018-03-13_12_shard1_replica_n1/../../../../contrib/velocity/lib
   [junit4]   2> 457006 WARN  (qtp2101575173-2188) [n:127.0.0.1:37093_solr c:testAlias0_2018-03-13_12 s:shard1 r:core_node2 x:testAlias0_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/testAlias0_2018-03-13_12_shard1_replica_n1/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/testAlias0_2018-03-13_12_shard1_replica_n1/../../../../dist
   [junit4]   2> 457021 INFO  (qtp1664186031-2176) [n:127.0.0.1:42215_solr c:testAlias3_2018-03-13_12 s:shard1 r:core_node2 x:testAlias3_2018-03-13_12_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 1
   [junit4]   2> 457021 INFO  (qtp1664186031-2176) [n:127.0.0.1:42215_solr c:testAlias3_2018-03-13_12 s:shard1 r:core_node2 x:testAlias3_2018-03-13_12_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 1
   [junit4]   2> 457021 WARN  (qtp1664186031-2176) [n:127.0.0.1:42215_solr c:testAlias3_2018-03-13_12 s:shard1 r:core_node2 x:testAlias3_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias3_2018-03-13_12_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias3_2018-03-13_12_shard1_replica_n1/../../../../contrib/extraction/lib
   [junit4]   2> 457021 WARN  (qtp1664186031-2176) [n:127.0.0.1:42215_solr c:testAlias3_2018-03-13_12 s:shard1 r:core_node2 x:testAlias3_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias3_2018-03-13_12_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias3_2018-03-13_12_shard1_replica_n1/../../../../dist
   [junit4]   2> 457022 WARN  (qtp1664186031-2176) [n:127.0.0.1:42215_solr c:testAlias3_2018-03-13_12 s:shard1 r:core_node2 x:testAlias3_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias3_2018-03-13_12_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias3_2018-03-13_12_shard1_replica_n1/../../../../contrib/clustering/lib
   [junit4]   2> 457022 WARN  (qtp1664186031-2176) [n:127.0.0.1:42215_solr c:testAlias3_2018-03-13_12 s:shard1 r:core_node2 x:testAlias3_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias3_2018-03-13_12_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias3_2018-03-13_12_shard1_replica_n1/../../../../dist
   [junit4]   2> 457022 WARN  (qtp1664186031-2176) [n:127.0.0.1:42215_solr c:testAlias3_2018-03-13_12 s:shard1 r:core_node2 x:testAlias3_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias3_2018-03-13_12_shard1_replica_n1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias3_2018-03-13_12_shard1_replica_n1/../../../../contrib/langid/lib
   [junit4]   2> 457022 WARN  (qtp1664186031-2176) [n:127.0.0.1:42215_solr c:testAlias3_2018-03-13_12 s:shard1 r:core_node2 x:testAlias3_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias3_2018-03-13_12_shard1_replica_n1/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias3_2018-03-13_12_shard1_replica_n1/../../../../dist
   [junit4]   2> 457023 WARN  (qtp1664186031-2176) [n:127.0.0.1:42215_solr c:testAlias3_2018-03-13_12 s:shard1 r:core_node2 x:testAlias3_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias3_2018-03-13_12_shard1_replica_n1/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias3_2018-03-13_12_shard1_replica_n1/../../../../contrib/velocity/lib
   [junit4]   2> 457023 WARN  (qtp1664186031-2176) [n:127.0.0.1:42215_solr c:testAlias3_2018-03-13_12 s:shard1 r:core_node2 x:testAlias3_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias3_2018-03-13_12_shard1_replica_n1/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias3_2018-03-13_12_shard1_replica_n1/../../../../dist
   [junit4]   2> 457046 INFO  (qtp2101575173-2188) [n:127.0.0.1:37093_solr c:testAlias0_2018-03-13_12 s:shard1 r:core_node2 x:testAlias0_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 457057 INFO  (qtp1664186031-2176) [n:127.0.0.1:42215_solr c:testAlias3_2018-03-13_12 s:shard1 r:core_node2 x:testAlias3_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 457096 INFO  (qtp1664186031-2171) [n:127.0.0.1:42215_solr c:testAlias2_2018-03-13_12 s:shard1 r:core_node2 x:testAlias2_2018-03-13_12_shard1_replica_n1] o.a.s.s.IndexSchema [testAlias2_2018-03-13_12_shard1_replica_n1] Schema name=default-config
   [junit4]   2> 457204 INFO  (qtp1664186031-2176) [n:127.0.0.1:42215_solr c:testAlias3_2018-03-13_12 s:shard1 r:core_node2 x:testAlias3_2018-03-13_12_shard1_replica_n1] o.a.s.s.IndexSchema [testAlias3_2018-03-13_12_shard1_replica_n1] Schema name=default-config
   [junit4]   2> 457215 INFO  (qtp2101575173-2188) [n:127.0.0.1:37093_solr c:testAlias0_2018-03-13_12 s:shard1 r:core_node2 x:testAlias0_2018-03-13_12_shard1_replica_n1] o.a.s.s.IndexSchema [testAlias0_2018-03-13_12_shard1_replica_n1] Schema name=default-config
   [junit4]   2> 457229 INFO  (qtp1664186031-2170) [n:127.0.0.1:42215_solr c:testAlias1_2018-03-13_12 s:shard1 r:core_node2 x:testAlias1_2018-03-13_12_shard1_replica_n1] o.a.s.s.IndexSchema [testAlias1_2018-03-13_12_shard1_replica_n1] Schema name=default-config
   [junit4]   2> 462006 INFO  (qtp2101575173-2188) [n:127.0.0.1:37093_solr c:testAlias0_2018-03-13_12 s:shard1 r:core_node2 x:testAlias0_2018-03-13_12_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
   [junit4]   2> 462097 INFO  (qtp1664186031-2176) [n:127.0.0.1:42215_solr c:testAlias3_2018-03-13_12 s:shard1 r:core_node2 x:testAlias3_2018-03-13_12_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
   [junit4]   2> 462097 INFO  (qtp1664186031-2170) [n:127.0.0.1:42215_solr c:testAlias1_2018-03-13_12 s:shard1 r:core_node2 x:testAlias1_2018-03-13_12_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
   [junit4]   2> 462097 INFO  (qtp1664186031-2171) [n:127.0.0.1:42215_solr c:testAlias2_2018-03-13_12 s:shard1 r:core_node2 x:testAlias2_2018-03-13_12_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
   [junit4]   2> 462953 INFO  (qtp2101575173-2188) [n:127.0.0.1:37093_solr c:testAlias0_2018-03-13_12 s:shard1 r:core_node2 x:testAlias0_2018-03-13_12_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testAlias0_2018-03-13_12_shard1_replica_n1' using configuration from collection testAlias0_2018-03-13_12, trusted=true
   [junit4]   2> 462954 INFO  (qtp2101575173-2188) [n:127.0.0.1:37093_solr c:testAlias0_2018-03-13_12 s:shard1 r:core_node2 x:testAlias0_2018-03-13_12_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37093.solr.core.testAlias0_2018-03-13_12.shard1.replica_n1' (registry 'solr.core.testAlias0_2018-03-13_12.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e5ebd63
   [junit4]   2> 462954 INFO  (qtp2101575173-2188) [n:127.0.0.1:37093_solr c:testAlias0_2018-03-13_12 s:shard1 r:core_node2 x:testAlias0_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 462954 INFO  (qtp2101575173-2188) [n:127.0.0.1:37093_solr c:testAlias0_2018-03-13_12 s:shard1 r:core_node2 x:testAlias0_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrCore [[testAlias0_2018-03-13_12_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/testAlias0_2018-03-13_12_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node4/./testAlias0_2018-03-13_12_shard1_replica_n1/data/]
   [junit4]   2> 463063 INFO  (qtp1664186031-2170) [n:127.0.0.1:42215_solr c:testAlias1_2018-03-13_12 s:shard1 r:core_node2 x:testAlias1_2018-03-13_12_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testAlias1_2018-03-13_12_shard1_replica_n1' using configuration from collection testAlias1_2018-03-13_12, trusted=true
   [junit4]   2> 463117 INFO  (qtp1664186031-2170) [n:127.0.0.1:42215_solr c:testAlias1_2018-03-13_12 s:shard1 r:core_node2 x:testAlias1_2018-03-13_12_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42215.solr.core.testAlias1_2018-03-13_12.shard1.replica_n1' (registry 'solr.core.testAlias1_2018-03-13_12.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e5ebd63
   [junit4]   2> 463117 INFO  (qtp1664186031-2170) [n:127.0.0.1:42215_solr c:testAlias1_2018-03-13_12 s:shard1 r:core_node2 x:testAlias1_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 463117 INFO  (qtp1664186031-2170) [n:127.0.0.1:42215_solr c:testAlias1_2018-03-13_12 s:shard1 r:core_node2 x:testAlias1_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrCore [[testAlias1_2018-03-13_12_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias1_2018-03-13_12_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/./testAlias1_2018-03-13_12_shard1_replica_n1/data/]
   [junit4]   2> 463205 INFO  (qtp1664186031-2171) [n:127.0.0.1:42215_solr c:testAlias2_2018-03-13_12 s:shard1 r:core_node2 x:testAlias2_2018-03-13_12_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testAlias2_2018-03-13_12_shard1_replica_n1' using configuration from collection testAlias2_2018-03-13_12, trusted=true
   [junit4]   2> 463206 INFO  (qtp1664186031-2171) [n:127.0.0.1:42215_solr c:testAlias2_2018-03-13_12 s:shard1 r:core_node2 x:testAlias2_2018-03-13_12_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42215.solr.core.testAlias2_2018-03-13_12.shard1.replica_n1' (registry 'solr.core.testAlias2_2018-03-13_12.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e5ebd63
   [junit4]   2> 463206 INFO  (qtp1664186031-2171) [n:127.0.0.1:42215_solr c:testAlias2_2018-03-13_12 s:shard1 r:core_node2 x:testAlias2_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 463206 INFO  (qtp1664186031-2171) [n:127.0.0.1:42215_solr c:testAlias2_2018-03-13_12 s:shard1 r:core_node2 x:testAlias2_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrCore [[testAlias2_2018-03-13_12_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias2_2018-03-13_12_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/./testAlias2_2018-03-13_12_shard1_replica_n1/data/]
   [junit4]   2> 463281 INFO  (qtp1664186031-2176) [n:127.0.0.1:42215_solr c:testAlias3_2018-03-13_12 s:shard1 r:core_node2 x:testAlias3_2018-03-13_12_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testAlias3_2018-03-13_12_shard1_replica_n1' using configuration from collection testAlias3_2018-03-13_12, trusted=true
   [junit4]   2> 463282 INFO  (qtp1664186031-2176) [n:127.0.0.1:42215_solr c:testAlias3_2018-03-13_12 s:shard1 r:core_node2 x:testAlias3_2018-03-13_12_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42215.solr.core.testAlias3_2018-03-13_12.shard1.replica_n1' (registry 'solr.core.testAlias3_2018-03-13_12.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e5ebd63
   [junit4]   2> 463282 INFO  (qtp1664186031-2176) [n:127.0.0.1:42215_solr c:testAlias3_2018-03-13_12 s:shard1 r:core_node2 x:testAlias3_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 463282 INFO  (qtp1664186031-2176) [n:127.0.0.1:42215_solr c:testAlias3_2018-03-13_12 s:shard1 r:core_node2 x:testAlias3_2018-03-13_12_shard1_replica_n1] o.a.s.c.SolrCore [[testAlias3_2018-03-13_12_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/testAlias3_2018-03-13_12_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_79DE20C20FACE14D-001/tempDir-001/node2/./testAlias3_2018-03-13_12_shard1_replica_n1/data/]
   [junit4]   2> 463355 INFO  (qtp1664186031-2171) [n:127.0.0.1:4

[...truncated too long message...]

1
   [junit4]   2> 16 /solr/aliases.json
   [junit4]   2> 16 /solr/collections/tlog_replica_test_real_time_get/terms/shard1
   [junit4]   2> 16 /solr/collections/tlog_replica_test_kill_tlog_replica/terms/shard1
   [junit4]   2> 14 /solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/terms/shard1
   [junit4]   2> 12 /solr/collections/tlog_replica_test_add_remove_tlog_replica/terms/shard1
   [junit4]   2> 12 /solr/collections/tlog_replica_test_add_remove_tlog_replica/terms/shard2
   [junit4]   2> 10 /solr/collections/tlog_replica_test_add_docs/terms/shard1
   [junit4]   2> 10 /solr/collections/tlog_replica_test_delete_by_id/terms/shard1
   [junit4]   2> 6 /solr/security.json
   [junit4]   2> 5 /solr/clusterprops.json
   [junit4]   2> 4 /solr/collections/tlog_replica_test_create_delete/collectionprops.json
   [junit4]   2> 3 /solr/collections/tlog_replica_test_kill_tlog_replica/collectionprops.json
   [junit4]   2> 3 /solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/collectionprops.json
   [junit4]   2> 3 /solr/collections/tlog_replica_test_basic_leader_election/collectionprops.json
   [junit4]   2> 3 /solr/collections/tlog_replica_test_kill_leader/collectionprops.json
   [junit4]   2> 3 /solr/collections/tlog_replica_test_remove_leader/collectionprops.json
   [junit4]   2> 2 /solr/collections/tlog_replica_test_delete_by_id/collectionprops.json
   [junit4]   2> 2 /solr/collections/tlog_replica_test_real_time_get/collectionprops.json
   [junit4]   2> 2 /solr/collections/tlog_replica_test_add_remove_tlog_replica/collectionprops.json
   [junit4]   2> 2 /solr/collections/tlog_replica_test_add_docs/collectionprops.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 40 /solr/collections/tlog_replica_test_create_delete/state.json
   [junit4]   2> 31 /solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/state.json
   [junit4]   2> 28 /solr/collections/tlog_replica_test_kill_leader/state.json
   [junit4]   2> 26 /solr/collections/tlog_replica_test_remove_leader/state.json
   [junit4]   2> 26 /solr/collections/tlog_replica_test_add_remove_tlog_replica/state.json
   [junit4]   2> 25 /solr/collections/tlog_replica_test_basic_leader_election/state.json
   [junit4]   2> 22 /solr/collections/tlog_replica_test_kill_tlog_replica/state.json
   [junit4]   2> 18 /solr/collections/tlog_replica_test_real_time_get/state.json
   [junit4]   2> 16 /solr/clusterprops.json
   [junit4]   2> 16 /solr/clusterstate.json
   [junit4]   2> 15 /solr/collections/tlog_replica_test_add_docs/state.json
   [junit4]   2> 13 /solr/collections/tlog_replica_test_delete_by_id/state.json
   [junit4]   2> 4 /solr/overseer_elect/election/72272792241373189-127.0.0.1:33753_solr-n_0000000001
   [junit4]   2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard2/election/72272792241373190-core_node15-n_0000000001
   [junit4]   2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard2/election/72272792241373189-core_node14-n_0000000000
   [junit4]   2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard2/election/72272792241373190-core_node15-n_0000000000
   [junit4]   2> 2 /solr/autoscaling.json
   [junit4]   2> 2 /solr/collections/tlog_replica_test_real_time_get/leader_elect/shard1/election/72272792241373212-core_node3-n_0000000001
   [junit4]   2> 2 /solr/collections/tlog_replica_test_kill_tlog_replica/leader_elect/shard1/election/72272792241373189-core_node4-n_0000000000
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 16 /solr/live_nodes
   [junit4]   2> 16 /solr/collections
   [junit4]   2> 2 /solr/overseer/queue
   [junit4]   2> 2 /solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 2 /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 2 /solr/overseer/collection-queue-work
   [junit4]   2> 2 /solr/overseer/queue-work
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_79DE20C20FACE14D-001
   [junit4]   2> Mar 13, 2018 1:06:14 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {foo=PostingsFormat(name=LuceneVarGapDocFreqInterval), title_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=Lucene50(blocksize=128), foo_s=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{_version_=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Direct), inplace_updatable_int=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=358, maxMBSortInHeap=6.169631307036646, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@55b3c6b0), locale=ar-AE, timezone=Asia/Aden
   [junit4]   2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=135967048,total=517996544
   [junit4]   2> NOTE: All tests run in this JVM: [SubstringBytesRefFilterTest, DOMUtilTest, PrimUtilsTest, TestFieldCacheVsDocValues, TestSolrCoreParser, DataDrivenBlockJoinTest, TestLRUCache, TestOnReconnectListenerSupport, TestGraphTermsQParserPlugin, SoftAutoCommitTest, OverseerModifyCollectionTest, SSLMigrationTest, SolrCoreTest, AnalyticsMergeStrategyTest, ScriptEngineTest, TestGraphMLResponseWriter, PeerSyncTest, TestUniqueKeyFieldResource, TestCollectionAPIs, OverseerTest, IndexSchemaTest, QueryEqualityTest, TestTolerantUpdateProcessorRandomCloud, VersionInfoTest, TestSolrCloudWithHadoopAuthPlugin, DistributedQueueTest, ShardSplitTest, JsonLoaderTest, TestCloudSchemaless, GraphQueryTest, SolrJmxReporterCloudTest, TestDistribDocBasedVersion, TestAuthenticationFramework, TestPostingsSolrHighlighter, TestSystemIdResolver, TestInPlaceUpdatesDistrib, SuggesterFSTTest, TestSimDistributedQueue, TestSurroundQueryParser, TimeRoutedAliasUpdateProcessorTest, TestReplicationHandlerBackup, SolrMetricsIntegrationTest, SolrCLIZkUtilsTest, SystemInfoHandlerTest, HighlighterTest, TestTestInjection, TestJsonFacets, ZkControllerTest, TestLuceneIndexBackCompat, ActionThrottleTest, DistributedMLTComponentTest, RankQueryTest, ExecutePlanActionTest, SyncSliceTest, TestLeaderInitiatedRecoveryThread, QueryElevationComponentTest, ForceLeaderTest, UpdateRequestProcessorFactoryTest, TestNRTOpen, DistributedIntervalFacetingTest, BlockDirectoryTest, TestRecovery, TestSha256AuthenticationProvider, CloudMLTQParserTest, DirectSolrConnectionTest, LeaderVoteWaitTimeoutTest, TestAnalyzeInfixSuggestions, MissingSegmentRecoveryTest, TestSweetSpotSimilarityFactory, ChaosMonkeySafeLeaderWithPullReplicasTest, TestFunctionQuery, TestSolrCloudWithSecureImpersonation, TestNumericRangeQuery32, TestScoreJoinQPNoScore, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestExactSharedStatsCache, SchemaVersionSpecificBehaviorTest, TestQueryUtils, AddBlockUpdateTest, ResponseHeaderTest, TestConfigSetsAPIZkFailure, SortByFunctionTest, TestHttpShardHandlerFactory, TestSchemaSimilarityResource, SuggesterTest, TestStressVersions, DistributedQueryComponentCustomSortTest, TestFoldingMultitermQuery, TestCollapseQParserPlugin, TestXmlQParserPlugin, DocValuesMissingTest, OutOfBoxZkACLAndCredentialsProvidersTest, BasicZkTest, TestManagedSynonymGraphFilterFactory, TestOrdValues, TestSimpleQParserPlugin, HLLUtilTest, TestStressRecovery, TestFreeTextSuggestions, BasicAuthIntegrationTest, TriLevelCompositeIdRoutingTest, TestStressUserVersions, TestNumericTerms32, DistanceUnitsTest, CSVRequestHandlerTest, CdcrBidirectionalTest, TestSimpleTextCodec, TestCustomSort, TestReplicaProperties, TestSolrCloudSnapshots, DeleteReplicaTest, ScheduledMaintenanceTriggerTest, SuggestComponentTest, TestMultiWordSynonyms, HDFSCollectionsAPITest, DistributedFacetPivotSmallAdvancedTest, TestSolrIndexConfig, AtomicUpdatesTest, JSONWriterTest, OverseerTaskQueueTest, TestCloudInspectUtil, TestSortableTextField, TestEmbeddedSolrServerSchemaAPI, TestSimpleTrackingShardHandler, DocumentBuilderTest, TestConfigSetProperties, XsltUpdateRequestHandlerTest, ConfigureRecoveryStrategyTest, TestCursorMarkWithoutUniqueKey, OpenCloseCoreStressTest, ShardRoutingTest, FullSolrCloudDistribCmdsTest, ClusterStateUpdateTest, TestRandomFaceting, TestDistributedGrouping, TestStressReorder, TestJoin, TestRangeQuery, TestWriterPerf, FileBasedSpellCheckerTest, PathHierarchyTokenizerFactoryTest, TermVectorComponentTest, RequiredFieldsTest, FieldAnalysisRequestHandlerTest, FastVectorHighlighterTest, TestCollationField, MBeansHandlerTest, MultiTermTest, TestDocSet, NumericFieldsTest, TestElisionMultitermQuery, OutputWriterTest, TestFuzzyAnalyzedSuggestions, NotRequiredUniqueKeyTest, SpellPossibilityIteratorTest, TestXIncludeConfig, TestPerFieldSimilarity, TestBM25SimilarityFactory, ResourceLoaderTest, OpenExchangeRatesOrgProviderTest, TestFastLRUCache, DateFieldTest, SpellingQueryConverterTest, RAMDirectoryFactoryTest, ClusterStateTest, TestDocumentBuilder, SliceStateTest, UUIDFieldTest, CircularListTest, CursorPagingTest, AssignBackwardCompatibilityTest, CloudExitableDirectoryReaderTest, CollectionPropsTest, ConfigSetsAPITest, ConnectionManagerTest, DeleteInactiveReplicaTest, DeleteShardTest, DocValuesNotIndexedTest, HttpPartitionTest, LIROnShardRestartTest, LeaderElectionContextKeyTest, LeaderFailoverAfterPartitionTest, LeaderFailureAfterFreshStartTest, RecoveryAfterSoftCommitTest, TestCloudJSONFacetJoinDomain, TestCloudSearcherWarming, TestDistributedMap, TestExclusionRuleCollectionAccess, TestLockTree, TestPrepRecovery, TestPullReplica, TestStressCloudBlindAtomicUpdates, TestTlogReplica]
   [junit4] Completed [582/785 (2!)] on J1 in 112.24s, 13 tests, 1 error, 2 skipped <<< FAILURES!

[...truncated 50146 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [JENKINS] Lucene-Solr-Tests-master - Build # 2422 - Unstable

Dawid Weiss-2
Doesn't reproduce for me, but looks suspicious?

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterWithThreads
   [junit4]   2> NOTE: reproduce with: ant test
-Dtestcase=TestIndexWriterWithThreads
-Dtests.method=testCloseWithThreads -Dtests.seed=F74EDD534544EE8
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=el
-Dtests.timezone=America/Tortola -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.64s J0 |
TestIndexWriterWithThreads.testCloseWithThreads <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: thread failed
before indexing a single document
   [junit4]    > at
__randomizedtesting.SeedInfo.seed([F74EDD534544EE8:12B0C4A257A1544E]:0)
   [junit4]    > at
org.apache.lucene.index.TestIndexWriterWithThreads.testCloseWithThreads(TestIndexWriterWithThreads.java:223)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   1> org.apache.lucene.store.LockObtainFailedException:
lock instance already obtained: (dir=RAMDirectory@64acc9e2
lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@609351af,
lockName=write.lock)
   [junit4]   1> at
org.apache.lucene.store.SingleInstanceLockFactory.obtainLock(SingleInstanceLockFactory.java:44)
   [junit4]   1> at
org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45)
   [junit4]   1> at
org.apache.lucene.store.FilterDirectory.obtainLock(FilterDirectory.java:104)
   [junit4]   1> at
org.apache.lucene.store.MockDirectoryWrapper.obtainLock(MockDirectoryWrapper.java:1049)
   [junit4]   1> at
org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:948)
   [junit4]   1> at
org.apache.lucene.index.TestIndexWriterWithThreads$DelayedIndexAndCloseRunnable.run(TestIndexWriterWithThreads.java:564)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [JENKINS] Lucene-Solr-Tests-master - Build # 2422 - Unstable

Dawid Weiss-2
There is code in that test that relies on wall time.

      final long stopTime = System.currentTimeMillis() + 200;

I don't think this is a good idea, it will trigger odd exceptions on
heavily loaded test machines.

Dawid

On Tue, Mar 13, 2018 at 9:28 PM, Dawid Weiss <[hidden email]> wrote:

> Doesn't reproduce for me, but looks suspicious?
>
>    [junit4] Suite: org.apache.lucene.index.TestIndexWriterWithThreads
>    [junit4]   2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexWriterWithThreads
> -Dtests.method=testCloseWithThreads -Dtests.seed=F74EDD534544EE8
> -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=el
> -Dtests.timezone=America/Tortola -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
>    [junit4] FAILURE 0.64s J0 |
> TestIndexWriterWithThreads.testCloseWithThreads <<<
>    [junit4]    > Throwable #1: java.lang.AssertionError: thread failed
> before indexing a single document
>    [junit4]    > at
> __randomizedtesting.SeedInfo.seed([F74EDD534544EE8:12B0C4A257A1544E]:0)
>    [junit4]    > at
> org.apache.lucene.index.TestIndexWriterWithThreads.testCloseWithThreads(TestIndexWriterWithThreads.java:223)
>    [junit4]    > at java.lang.Thread.run(Thread.java:748)
>    [junit4]   1> org.apache.lucene.store.LockObtainFailedException:
> lock instance already obtained: (dir=RAMDirectory@64acc9e2
> lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@609351af,
> lockName=write.lock)
>    [junit4]   1> at
> org.apache.lucene.store.SingleInstanceLockFactory.obtainLock(SingleInstanceLockFactory.java:44)
>    [junit4]   1> at
> org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45)
>    [junit4]   1> at
> org.apache.lucene.store.FilterDirectory.obtainLock(FilterDirectory.java:104)
>    [junit4]   1> at
> org.apache.lucene.store.MockDirectoryWrapper.obtainLock(MockDirectoryWrapper.java:1049)
>    [junit4]   1> at
> org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:948)
>    [junit4]   1> at
> org.apache.lucene.index.TestIndexWriterWithThreads$DelayedIndexAndCloseRunnable.run(TestIndexWriterWithThreads.java:564)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2423 - Still Unstable

Apache Jenkins Server-2
In reply to this post by Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2423/

1 tests failed.
FAILED:  org.apache.solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest.testAsyncIdBackCompat

Error Message:
KeeperErrorCode = Session expired for /overseer

Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer
        at __randomizedtesting.SeedInfo.seed([995CC4B4A4EAA767:476C571808064337]:0)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
        at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1105)
        at org.apache.solr.common.cloud.SolrZkClient.lambda$exists$2(SolrZkClient.java:304)
        at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
        at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:304)
        at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:483)
        at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:386)
        at org.apache.solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest.testAsyncIdBackCompat(CollectionsAPIAsyncDistributedZkTest.java:252)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 1828 lines...]
   [junit4] JVM J0: stdout was not empty, see: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/temp/junit4-J0-20180314_121640_9184179940412265839343.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] codec: CheapBastard, pf: Lucene50, dvf: Direct
   [junit4] <<< JVM J0: EOF ----

[...truncated 10677 lines...]
   [junit4] Suite: org.apache.solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest
   [junit4]   2> 829257 INFO  (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[995CC4B4A4EAA767]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_995CC4B4A4EAA767-001/init-core-data-001
   [junit4]   2> 829257 WARN  (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[995CC4B4A4EAA767]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=3 numCloses=3
   [junit4]   2> 829274 INFO  (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[995CC4B4A4EAA767]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 829275 INFO  (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[995CC4B4A4EAA767]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 829278 INFO  (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[995CC4B4A4EAA767]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_995CC4B4A4EAA767-001/tempDir-001
   [junit4]   2> 829278 INFO  (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[995CC4B4A4EAA767]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 829280 INFO  (Thread-761) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 829280 INFO  (Thread-761) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 829310 ERROR (Thread-761) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 829380 INFO  (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[995CC4B4A4EAA767]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:35951
   [junit4]   2> 829458 INFO  (zkConnectionManagerCallback-622-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 829507 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100c8e923bd0000, likely client has closed socket
   [junit4]   2> 829509 INFO  (jetty-launcher-619-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 829510 INFO  (jetty-launcher-619-thread-2) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 829538 INFO  (jetty-launcher-619-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 829538 INFO  (jetty-launcher-619-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 829538 INFO  (jetty-launcher-619-thread-2) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 829538 INFO  (jetty-launcher-619-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@37ad796e{/solr,null,AVAILABLE}
   [junit4]   2> 829540 INFO  (jetty-launcher-619-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@740de624{SSL,[ssl, http/1.1]}{127.0.0.1:36078}
   [junit4]   2> 829540 INFO  (jetty-launcher-619-thread-2) [    ] o.e.j.s.Server Started @840054ms
   [junit4]   2> 829540 INFO  (jetty-launcher-619-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36078}
   [junit4]   2> 829540 ERROR (jetty-launcher-619-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 829541 INFO  (jetty-launcher-619-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 829541 INFO  (jetty-launcher-619-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 829541 INFO  (jetty-launcher-619-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 829541 INFO  (jetty-launcher-619-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-14T13:15:21.541Z
   [junit4]   2> 829586 INFO  (zkConnectionManagerCallback-624-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 829587 INFO  (jetty-launcher-619-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 829587 INFO  (jetty-launcher-619-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 829587 INFO  (jetty-launcher-619-thread-1) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 829588 INFO  (jetty-launcher-619-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@60c83ebc{/solr,null,AVAILABLE}
   [junit4]   2> 829588 INFO  (jetty-launcher-619-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@714a0236{SSL,[ssl, http/1.1]}{127.0.0.1:33807}
   [junit4]   2> 829588 INFO  (jetty-launcher-619-thread-1) [    ] o.e.j.s.Server Started @840102ms
   [junit4]   2> 829588 INFO  (jetty-launcher-619-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33807}
   [junit4]   2> 829589 ERROR (jetty-launcher-619-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 829589 INFO  (jetty-launcher-619-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 829589 INFO  (jetty-launcher-619-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 829589 INFO  (jetty-launcher-619-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 829589 INFO  (jetty-launcher-619-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-14T13:15:21.589Z
   [junit4]   2> 829591 INFO  (jetty-launcher-619-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 829732 INFO  (zkConnectionManagerCallback-626-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 829733 INFO  (jetty-launcher-619-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 829834 INFO  (jetty-launcher-619-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35951/solr
   [junit4]   2> 829859 INFO  (jetty-launcher-619-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35951/solr
   [junit4]   2> 829862 INFO  (zkConnectionManagerCallback-631-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 829865 INFO  (zkConnectionManagerCallback-634-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 829952 INFO  (zkConnectionManagerCallback-636-thread-1-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 829953 INFO  (zkConnectionManagerCallback-638-thread-1-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 830455 INFO  (jetty-launcher-619-thread-1) [n:127.0.0.1:33807_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 830507 INFO  (jetty-launcher-619-thread-1) [n:127.0.0.1:33807_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33807_solr
   [junit4]   2> 830508 INFO  (jetty-launcher-619-thread-1) [n:127.0.0.1:33807_solr    ] o.a.s.c.Overseer Overseer (id=72278497690451973-127.0.0.1:33807_solr-n_0000000000) starting
   [junit4]   2> 830574 INFO  (zkConnectionManagerCallback-644-thread-1-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 830720 INFO  (jetty-launcher-619-thread-1) [n:127.0.0.1:33807_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35951/solr ready
   [junit4]   2> 830813 INFO  (jetty-launcher-619-thread-1) [n:127.0.0.1:33807_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33807_solr
   [junit4]   2> 830911 INFO  (OverseerStateUpdate-72278497690451973-127.0.0.1:33807_solr-n_0000000000) [n:127.0.0.1:33807_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 831040 INFO  (zkCallback-643-thread-1-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 831234 INFO  (jetty-launcher-619-thread-2) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 831236 INFO  (jetty-launcher-619-thread-2) [n:127.0.0.1:36078_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 831254 INFO  (jetty-launcher-619-thread-1) [n:127.0.0.1:33807_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33807.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d9075ae
   [junit4]   2> 831266 INFO  (jetty-launcher-619-thread-2) [n:127.0.0.1:36078_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36078_solr
   [junit4]   2> 831272 INFO  (zkCallback-635-thread-1-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 831281 INFO  (jetty-launcher-619-thread-1) [n:127.0.0.1:33807_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33807.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d9075ae
   [junit4]   2> 831281 INFO  (jetty-launcher-619-thread-1) [n:127.0.0.1:33807_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33807.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d9075ae
   [junit4]   2> 831282 INFO  (jetty-launcher-619-thread-1) [n:127.0.0.1:33807_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_995CC4B4A4EAA767-001/tempDir-001/node1/.
   [junit4]   2> 831316 INFO  (zkCallback-637-thread-1-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 831326 INFO  (zkCallback-643-thread-1-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 831585 INFO  (jetty-launcher-619-thread-2) [n:127.0.0.1:36078_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36078.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d9075ae
   [junit4]   2> 831616 INFO  (jetty-launcher-619-thread-2) [n:127.0.0.1:36078_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36078.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d9075ae
   [junit4]   2> 831616 INFO  (jetty-launcher-619-thread-2) [n:127.0.0.1:36078_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36078.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d9075ae
   [junit4]   2> 831618 INFO  (jetty-launcher-619-thread-2) [n:127.0.0.1:36078_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_995CC4B4A4EAA767-001/tempDir-001/node2/.
   [junit4]   2> 831670 INFO  (zkConnectionManagerCallback-648-thread-1-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 831670 INFO  (jetty-launcher-619-thread-2) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 831680 INFO  (jetty-launcher-619-thread-2) [n:127.0.0.1:36078_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35951/solr ready
   [junit4]   2> 831774 INFO  (zkConnectionManagerCallback-650-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 831779 INFO  (zkConnectionManagerCallback-654-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 831780 INFO  (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[995CC4B4A4EAA767]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 831793 INFO  (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[995CC4B4A4EAA767]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35951/solr ready
   [junit4]   2> 832231 INFO  (TEST-CollectionsAPIAsyncDistributedZkTest.testSolrJAPICalls-seed#[995CC4B4A4EAA767]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSolrJAPICalls
   [junit4]   2> 832383 INFO  (qtp173942539-3456) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=824d4996-bfd0-483d-acd0-22f60156e543&replicationFactor=1&collection.configName=conf1&name=testasynccollectioncreation&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 832671 INFO  (qtp173942539-3456) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=824d4996-bfd0-483d-acd0-22f60156e543&replicationFactor=1&collection.configName=conf1&name=testasynccollectioncreation&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=288
   [junit4]   2> 832683 INFO  (OverseerThreadFactory-819-thread-1-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection testasynccollectioncreation
   [junit4]   2> 832698 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=824d4996-bfd0-483d-acd0-22f60156e543&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 832731 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=824d4996-bfd0-483d-acd0-22f60156e543&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=32
   [junit4]   2> 832856 INFO  (OverseerStateUpdate-72278497690451973-127.0.0.1:33807_solr-n_0000000000) [n:127.0.0.1:33807_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testasynccollectioncreation",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testasynccollectioncreation_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:36078/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 833254 INFO  (qtp272600882-3445) [n:127.0.0.1:36078_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=testasynccollectioncreation&version=2&replicaType=NRT&async=824d4996-bfd0-483d-acd0-22f60156e5433370725806574853&coreNodeName=core_node2&name=testasynccollectioncreation_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin} status=0 QTime=0
   [junit4]   2> 833255 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=testasynccollectioncreation&version=2&replicaType=NRT&async=824d4996-bfd0-483d-acd0-22f60156e5433370725806574853&coreNodeName=core_node2&name=testasynccollectioncreation_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin
   [junit4]   2> 833255 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 833330 INFO  (qtp272600882-3443) [n:127.0.0.1:36078_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853
   [junit4]   2> 833331 INFO  (qtp272600882-3443) [n:127.0.0.1:36078_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=824d4996-bfd0-483d-acd0-22f60156e5433370725806574853&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 833430 INFO  (zkCallback-637-thread-1-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 833431 INFO  (zkCallback-637-thread-2-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 833807 INFO  (qtp173942539-3456) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=824d4996-bfd0-483d-acd0-22f60156e543&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 833824 INFO  (qtp173942539-3456) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=824d4996-bfd0-483d-acd0-22f60156e543&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=17
   [junit4]   2> 834348 INFO  (qtp272600882-3445) [n:127.0.0.1:36078_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853
   [junit4]   2> 834348 INFO  (qtp272600882-3445) [n:127.0.0.1:36078_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=824d4996-bfd0-483d-acd0-22f60156e5433370725806574853&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 834743 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 834856 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=824d4996-bfd0-483d-acd0-22f60156e543&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 834857 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=824d4996-bfd0-483d-acd0-22f60156e543&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 834869 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.s.IndexSchema [testasynccollectioncreation_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 834884 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 834885 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testasynccollectioncreation_shard1_replica_n1' using configuration from collection testasynccollectioncreation, trusted=true
   [junit4]   2> 834958 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36078.solr.core.testasynccollectioncreation.shard1.replica_n1' (registry 'solr.core.testasynccollectioncreation.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d9075ae
   [junit4]   2> 834958 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 834958 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.c.SolrCore [[testasynccollectioncreation_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_995CC4B4A4EAA767-001/tempDir-001/node2/testasynccollectioncreation_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_995CC4B4A4EAA767-001/tempDir-001/node2/./testasynccollectioncreation_shard1_replica_n1/data/]
   [junit4]   2> 836354 INFO  (qtp173942539-3456) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=824d4996-bfd0-483d-acd0-22f60156e543&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 836367 INFO  (qtp173942539-3456) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=824d4996-bfd0-483d-acd0-22f60156e543&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=13
   [junit4]   2> 836391 INFO  (qtp272600882-3443) [n:127.0.0.1:36078_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853
   [junit4]   2> 836391 INFO  (qtp272600882-3443) [n:127.0.0.1:36078_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=824d4996-bfd0-483d-acd0-22f60156e5433370725806574853&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 836494 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 836494 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 836495 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 836495 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 836513 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@69cb6a50[testasynccollectioncreation_shard1_replica_n1] main]
   [junit4]   2> 836514 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 836514 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 836514 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 836559 INFO  (searcherExecutor-824-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n1 r:core_node2 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE n:127.0.0.1:36078_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.c.SolrCore [testasynccollectioncreation_shard1_replica_n1] Registered new searcher Searcher@69cb6a50[testasynccollectioncreation_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 836559 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594919043527081984
   [junit4]   2> 836564 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update terms at /collections/testasynccollectioncreation/terms/shard1 to Terms{values={core_node2=0}, version=0}
   [junit4]   2> 836567 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 836567 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 836567 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:36078/solr/testasynccollectioncreation_shard1_replica_n1/
   [junit4]   2> 836567 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 836567 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:36078/solr/testasynccollectioncreation_shard1_replica_n1/ has no replicas
   [junit4]   2> 836567 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 836582 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:36078/solr/testasynccollectioncreation_shard1_replica_n1/ shard1
   [junit4]   2> 836583 INFO  (zkCallback-637-thread-1-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 836583 INFO  (zkCallback-637-thread-2-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 836633 INFO  (parallelCoreAdminExecutor-822-thread-1-processing-n:127.0.0.1:36078_solr 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 836745 INFO  (zkCallback-637-thread-2-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 836745 INFO  (zkCallback-637-thread-1-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 837369 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=824d4996-bfd0-483d-acd0-22f60156e543&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 837372 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=824d4996-bfd0-483d-acd0-22f60156e543&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 837393 INFO  (qtp272600882-3445) [n:127.0.0.1:36078_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 824d4996-bfd0-483d-acd0-22f60156e5433370725806574853
   [junit4]   2> 837393 INFO  (qtp272600882-3445) [n:127.0.0.1:36078_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=824d4996-bfd0-483d-acd0-22f60156e5433370725806574853&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 837416 INFO  (OverseerThreadFactory-819-thread-1-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 838379 INFO  (qtp173942539-3456) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=824d4996-bfd0-483d-acd0-22f60156e543&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 838380 INFO  (qtp173942539-3456) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=824d4996-bfd0-483d-acd0-22f60156e543&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 838402 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=824d4996-bfd0-483d-acd0-22f60156e543&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 838403 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=824d4996-bfd0-483d-acd0-22f60156e543&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 838404 INFO  (qtp173942539-3456) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=9ba4e45c-c197-4a3e-9b17-7b5fd2cfe9e6&replicationFactor=1&collection.configName=conf1&name=testasynccollectioncreation&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 838406 INFO  (qtp173942539-3456) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=9ba4e45c-c197-4a3e-9b17-7b5fd2cfe9e6&replicationFactor=1&collection.configName=conf1&name=testasynccollectioncreation&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 838407 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=9ba4e45c-c197-4a3e-9b17-7b5fd2cfe9e6&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 838427 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=9ba4e45c-c197-4a3e-9b17-7b5fd2cfe9e6&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=19
   [junit4]   2> 838428 INFO  (OverseerThreadFactory-819-thread-2-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection testasynccollectioncreation
   [junit4]   2> 838428 INFO  (OverseerCollectionConfigSetProcessor-72278497690451973-127.0.0.1:33807_solr-n_0000000000) [n:127.0.0.1:33807_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 838480 ERROR (OverseerThreadFactory-819-thread-2-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.a.c.OverseerCollectionMessageHandler Collection: testasynccollectioncreation operation: create failed:org.apache.solr.common.SolrException: collection already exists: testasynccollectioncreation
   [junit4]   2> at org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:106)
   [junit4]   2> at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:252)
   [junit4]   2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:469)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>
   [junit4]   2> 838679 INFO  (OverseerThreadFactory-819-thread-2-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000001 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 839434 INFO  (qtp173942539-3456) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=9ba4e45c-c197-4a3e-9b17-7b5fd2cfe9e6&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 839488 INFO  (qtp173942539-3456) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=9ba4e45c-c197-4a3e-9b17-7b5fd2cfe9e6&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=53
   [junit4]   2> 839489 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=9ba4e45c-c197-4a3e-9b17-7b5fd2cfe9e6&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 839559 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=9ba4e45c-c197-4a3e-9b17-7b5fd2cfe9e6&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=70
   [junit4]   2> 839560 INFO  (qtp173942539-3456) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params async=d8491741-511c-4bc7-84dc-10c720dc720d&action=ADDREPLICA&collection=testasynccollectioncreation&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 839598 INFO  (qtp173942539-3456) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=d8491741-511c-4bc7-84dc-10c720dc720d&action=ADDREPLICA&collection=testasynccollectioncreation&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=37
   [junit4]   2> 839599 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=d8491741-511c-4bc7-84dc-10c720dc720d&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 839637 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=d8491741-511c-4bc7-84dc-10c720dc720d&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=38
   [junit4]   2> 839690 INFO  (OverseerCollectionConfigSetProcessor-72278497690451973-127.0.0.1:33807_solr-n_0000000000) [n:127.0.0.1:33807_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000001 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 839692 INFO  (OverseerThreadFactory-819-thread-3-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:33807_solr for creating new replica
   [junit4]   2> 839763 INFO  (OverseerStateUpdate-72278497690451973-127.0.0.1:33807_solr-n_0000000000) [n:127.0.0.1:33807_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"addreplica",
   [junit4]   2>   "collection":"testasynccollectioncreation",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testasynccollectioncreation_shard1_replica_n3",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:33807/solr",
   [junit4]   2>   "node_name":"127.0.0.1:33807_solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 839765 INFO  (zkCallback-637-thread-1-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 839765 INFO  (zkCallback-637-thread-2-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 839973 INFO  (qtp173942539-3457) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789&qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf1&name=testasynccollectioncreation_shard1_replica_n3&action=CREATE&collection=testasynccollectioncreation&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=0
   [junit4]   2> 839973 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789&qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf1&name=testasynccollectioncreation_shard1_replica_n3&action=CREATE&collection=testasynccollectioncreation&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 839977 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789
   [junit4]   2> 839996 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=19
   [junit4]   2> 840015 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 840113 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 840168 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.s.IndexSchema [testasynccollectioncreation_shard1_replica_n3] Schema name=minimal
   [junit4]   2> 840173 INFO  (zkCallback-637-thread-2-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 840174 INFO  (zkCallback-637-thread-1-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 840174 INFO  (zkCallback-635-thread-1-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 840178 INFO  (zkCallback-635-thread-2-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 840203 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 840203 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'testasynccollectioncreation_shard1_replica_n3' using configuration from collection testasynccollectioncreation, trusted=true
   [junit4]   2> 840204 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33807.solr.core.testasynccollectioncreation.shard1.replica_n3' (registry 'solr.core.testasynccollectioncreation.shard1.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d9075ae
   [junit4]   2> 840204 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 840204 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.SolrCore [[testasynccollectioncreation_shard1_replica_n3] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_995CC4B4A4EAA767-001/tempDir-001/node1/testasynccollectioncreation_shard1_replica_n3], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_995CC4B4A4EAA767-001/tempDir-001/node1/./testasynccollectioncreation_shard1_replica_n3/data/]
   [junit4]   2> 840430 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 840430 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 840431 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 840431 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 840433 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@4b081c6b[testasynccollectioncreation_shard1_replica_n3] main]
   [junit4]   2> 840434 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 840434 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 840435 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 840462 INFO  (searcherExecutor-829-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.SolrCore [testasynccollectioncreation_shard1_replica_n3] Registered new searcher Searcher@4b081c6b[testasynccollectioncreation_shard1_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 840462 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594919047619674112
   [junit4]   2> 840474 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.ZkShardTerms Successful update terms at /collections/testasynccollectioncreation/terms/shard1 to Terms{values={core_node2=0, core_node4=0}, version=1}
   [junit4]   2> 840476 INFO  (parallelCoreAdminExecutor-820-thread-1-processing-n:127.0.0.1:33807_solr d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.ZkController Core needs to recover:testasynccollectioncreation_shard1_replica_n3
   [junit4]   2> 840526 INFO  (updateExecutor-627-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 840527 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 840528 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 840751 INFO  (qtp272600882-3446) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.c.S.Request [testasynccollectioncreation_shard1_replica_n1]  webapp=/solr path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=57
   [junit4]   2> 840751 INFO  (qtp272600882-3446) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.c.S.Request [testasynccollectioncreation_shard1_replica_n1]  webapp=/solr path=/admin/ping params={wt=javabin&version=2} status=0 QTime=57
   [junit4]   2> 840752 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[testasynccollectioncreation_shard1_replica_n3]
   [junit4]   2> 840752 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 840752 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.RecoveryStrategy Publishing state of core [testasynccollectioncreation_shard1_replica_n3] as recovering, leader is [https://127.0.0.1:36078/solr/testasynccollectioncreation_shard1_replica_n1/] and I am [https://127.0.0.1:33807/solr/testasynccollectioncreation_shard1_replica_n3/]
   [junit4]   2> 840789 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.ZkShardTerms Successful update terms at /collections/testasynccollectioncreation/terms/shard1 to Terms{values={core_node2=0, core_node4_recovering=0, core_node4=0}, version=2}
   [junit4]   2> 840819 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.RecoveryStrategy Sending prep recovery command to [https://127.0.0.1:36078/solr]; [WaitForState: action=PREPRECOVERY&core=testasynccollectioncreation_shard1_replica_n1&nodeName=127.0.0.1:33807_solr&coreNodeName=core_node4&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 840827 INFO  (qtp173942539-3456) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=d8491741-511c-4bc7-84dc-10c720dc720d&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 840870 INFO  (qtp173942539-3456) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=d8491741-511c-4bc7-84dc-10c720dc720d&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=42
   [junit4]   2> 840936 INFO  (qtp272600882-3443) [n:127.0.0.1:36078_solr    ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
   [junit4]   2> 840954 INFO  (zkCallback-637-thread-2-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 840954 INFO  (zkCallback-637-thread-1-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 840954 INFO  (zkCallback-635-thread-2-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 840954 INFO  (zkCallback-635-thread-1-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 841008 INFO  (qtp173942539-3457) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789
   [junit4]   2> 841008 INFO  (qtp173942539-3457) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 841046 INFO  (qtp272600882-3443) [n:127.0.0.1:36078_solr    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=testasynccollectioncreation, shard=shard1, thisCore=testasynccollectioncreation_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:33807_solr, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{"core":"testasynccollectioncreation_shard1_replica_n3","base_url":"https://127.0.0.1:33807/solr","node_name":"127.0.0.1:33807_solr","state":"down","type":"NRT"}
   [junit4]   2> 841062 INFO  (OverseerThreadFactory-819-thread-3-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 841702 INFO  (OverseerCollectionConfigSetProcessor-72278497690451973-127.0.0.1:33807_solr-n_0000000000) [n:127.0.0.1:33807_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 841885 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=d8491741-511c-4bc7-84dc-10c720dc720d&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 841892 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=d8491741-511c-4bc7-84dc-10c720dc720d&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=7
   [junit4]   2> 841909 INFO  (qtp173942539-3457) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=d8491741-511c-4bc7-84dc-10c720dc720d&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 841910 INFO  (qtp173942539-3457) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=d8491741-511c-4bc7-84dc-10c720dc720d&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 841912 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :splitshard with params async=74188428-f754-4b14-a1c6-0d4e345d27a8&action=SPLITSHARD&collection=testasynccollectioncreation&shard=shard1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 841923 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=74188428-f754-4b14-a1c6-0d4e345d27a8&action=SPLITSHARD&collection=testasynccollectioncreation&shard=shard1&wt=javabin&version=2} status=0 QTime=10
   [junit4]   2> 841924 INFO  (qtp173942539-3457) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=74188428-f754-4b14-a1c6-0d4e345d27a8&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 841933 INFO  (qtp173942539-3457) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=74188428-f754-4b14-a1c6-0d4e345d27a8&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=9
   [junit4]   2> 841945 INFO  (OverseerThreadFactory-819-thread-4-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.a.c.SplitShardCmd Split shard invoked
   [junit4]   2> 841986 INFO  (OverseerThreadFactory-819-thread-4-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.a.c.SplitShardCmd Creating slice shard1_0 of collection testasynccollectioncreation on 127.0.0.1:36078_solr
   [junit4]   2> 842047 INFO  (qtp272600882-3443) [n:127.0.0.1:36078_solr    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=testasynccollectioncreation, shard=shard1, thisCore=testasynccollectioncreation_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:33807_solr, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{"core":"testasynccollectioncreation_shard1_replica_n3","base_url":"https://127.0.0.1:33807/solr","node_name":"127.0.0.1:33807_solr","state":"recovering","type":"NRT"}
   [junit4]   2> 842047 INFO  (qtp272600882-3443) [n:127.0.0.1:36078_solr    ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
   [junit4]   2> 842047 INFO  (qtp272600882-3443) [n:127.0.0.1:36078_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:33807_solr&onlyIfLeaderActive=true&core=testasynccollectioncreation_shard1_replica_n1&coreNodeName=core_node4&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1110
   [junit4]   2> 842125 INFO  (zkCallback-635-thread-1-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 842125 INFO  (zkCallback-635-thread-2-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 842160 INFO  (zkCallback-637-thread-2-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 842160 INFO  (zkCallback-637-thread-1-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 842642 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [https://127.0.0.1:36078/solr/testasynccollectioncreation_shard1_replica_n1/] - recoveringAfterStartup=[true]
   [junit4]   2> 842642 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.u.PeerSync PeerSync: core=testasynccollectioncreation_shard1_replica_n3 url=https://127.0.0.1:33807/solr START replicas=[https://127.0.0.1:36078/solr/testasynccollectioncreation_shard1_replica_n1/] nUpdates=100
   [junit4]   2> 842997 INFO  (OverseerThreadFactory-819-thread-4-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.a.c.SplitShardCmd Adding replica testasynccollectioncreation_shard1_0_replica_n5 as part of slice shard1_0 of collection testasynccollectioncreation on 127.0.0.1:36078_solr
   [junit4]   2> 843017 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=74188428-f754-4b14-a1c6-0d4e345d27a8&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 843028 INFO  (qtp173942539-3454) [n:127.0.0.1:33807_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=74188428-f754-4b14-a1c6-0d4e345d27a8&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=11
   [junit4]   2> 843048 INFO  (qtp272600882-3445) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.u.IndexFingerprint IndexFingerprint millis:4.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
   [junit4]   2> 843048 INFO  (qtp272600882-3445) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1 r:core_node2 x:testasynccollectioncreation_shard1_replica_n1] o.a.s.c.S.Request [testasynccollectioncreation_shard1_replica_n1]  webapp=/solr path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=5
   [junit4]   2> 843122 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.u.IndexFingerprint IndexFingerprint millis:58.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
   [junit4]   2> 843122 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
   [junit4]   2> 843122 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 843122 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 843157 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 843180 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
   [junit4]   2> 843180 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
   [junit4]   2> 843180 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.RecoveryStrategy No replay needed.
   [junit4]   2> 843180 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
   [junit4]   2> 843181 INFO  (recoveryExecutor-629-thread-1-processing-x:testasynccollectioncreation_shard1_replica_n3 r:core_node4 d8491741-511c-4bc7-84dc-10c720dc720d3370732806404789 CREATE n:127.0.0.1:33807_solr s:shard1 c:testasynccollectioncreation) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.c.ZkShardTerms Successful update terms at /collections/testasynccollectioncreation/terms/shard1 to Terms{values={core_node2=0, core_node4=0}, version=3}
   [junit4]   2> 843209 INFO  (OverseerThreadFactory-819-thread-4-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:36078_solr for creating new replica
   [junit4]   2> 843210 INFO  (OverseerStateUpdate-72278497690451973-127.0.0.1:33807_solr-n_0000000000) [n:127.0.0.1:33807_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"addreplica",
   [junit4]   2>   "collection":"testasynccollectioncreation",
   [junit4]   2>   "shard":"shard1_0",
   [junit4]   2>   "core":"testasynccollectioncreation_shard1_0_replica_n5",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:36078/solr",
   [junit4]   2>   "node_name":"127.0.0.1:36078_solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 843318 INFO  (zkCallback-635-thread-2-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 843319 INFO  (zkCallback-635-thread-1-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 843326 INFO  (zkCallback-637-thread-1-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 843326 INFO  (zkCallback-637-thread-2-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 843525 INFO  (qtp272600882-3446) [n:127.0.0.1:36078_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=74188428-f754-4b14-a1c6-0d4e345d27a83370736373050819&qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&name=testasynccollectioncreation_shard1_0_replica_n5&action=CREATE&collection=testasynccollectioncreation&shard=shard1_0&wt=javabin&version=2&replicaType=NRT} status=0 QTime=71
   [junit4]   2> 843527 INFO  (parallelCoreAdminExecutor-822-thread-2-processing-n:127.0.0.1:36078_solr 74188428-f754-4b14-a1c6-0d4e345d27a83370736373050819 CREATE) [n:127.0.0.1:36078_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=74188428-f754-4b14-a1c6-0d4e345d27a83370736373050819&qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&name=testasynccollectioncreation_shard1_0_replica_n5&action=CREATE&collection=testasynccollectioncreation&shard=shard1_0&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 843529 INFO  (qtp272600882-3445) [n:127.0.0.1:36078_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 74188428-f754-4b14-a1c6-0d4e345d27a83370736373050819
   [junit4]   2> 843529 INFO  (qtp272600882-3445) [n:127.0.0.1:36078_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=74188428-f754-4b14-a1c6-0d4e345d27a83370736373050819&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 843619 INFO  (parallelCoreAdminExecutor-822-thread-2-processing-n:127.0.0.1:36078_solr 74188428-f754-4b14-a1c6-0d4e345d27a83370736373050819 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1_0 r:core_node7 x:testasynccollectioncreation_shard1_0_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 843638 INFO  (parallelCoreAdminExecutor-822-thread-2-processing-n:127.0.0.1:36078_solr 74188428-f754-4b14-a1c6-0d4e345d27a83370736373050819 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1_0 r:core_node7 x:testasynccollectioncreation_shard1_0_replica_n5] o.a.s.s.IndexSchema [testasynccollectioncreation_shard1_0_replica_n5] Schema name=minimal
   [junit4]   2> 843653 INFO  (parallelCoreAdminExecutor-822-thread-2-processing-n:127.0.0.1:36078_solr 74188428-f754-4b14-a1c6-0d4e345d27a83370736373050819 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1_0 r:core_node7 x:testasynccollectioncreation_shard1_0_replica_n5] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 843653 INFO  (parallelCoreAdminExecutor-822-thread-2-processing-n:127.0.0.1:36078_solr 74188428-f754-4b14-a1c6-0d4e345d27a83370736373050819 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1_0 r:core_node7 x:testasynccollectioncreation_shard1_0_replica_n5] o.a.s.c.CoreContainer Creating SolrCore 'testasynccollectioncreation_shard1_0_replica_n5' using configuration from collection testasynccollectioncreation, trusted=true
   [junit4]   2> 843654 INFO  (parallelCoreAdminExecutor-822-thread-2-processing-n:127.0.0.1:36078_solr 74188428-f754-4b14-a1c6-0d4e345d27a83370736373050819 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1_0 r:core_node7 x:testasynccollectioncreation_shard1_0_replica_n5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36078.solr.core.testasynccollectioncreation.shard1_0.replica_n5' (registry 'solr.core.testasynccollectioncreation.shard1_0.replica_n5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d9075ae
   [junit4]   2> 843654 INFO  (parallelCoreAdminExecutor-822-thread-2-processing-n:127.0.0.1:36078_solr 74188428-f754-4b14-a1c6-0d4e345d27a83370736373050819 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1_0 r:core_node7 x:testasynccollectioncreation_shard1_0_replica_n5] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 843654 INFO  (parallelCoreAdminExecutor-822-thread-2-processing-n:127.0.0.1:36078_solr 74188428-f754-4b14-a1c6-0d4e345d27a83370736373050819 CREATE) [n:127.0.0.1:36078_solr c:testasynccollectioncreation s:shard1_0 r:core_node7 x:testasynccollectioncreation_shard1_0_replica_n5] o.a.s.c.SolrCore [[testasynccollectioncreation_shard1_0_replica_n5] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_995CC4B4A4EAA767-001/tempDir-001/node2/testasynccollectioncreation_shard1_0_replica_n5], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_995CC4B4A4EAA767-001/tempDir-001/node2/./testasynccollectioncreation_shard1_0_replica_n5/data/]
   [junit4]   2> 843716 INFO  (zkCallback-635-thread-3-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 843716 INFO  (zkCallback-635-thread-2-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 843720 INFO  (zkCallback-637-thread-1-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 843720 INFO  (zkCallback-637-thread-2-processing-n:127.0.0.1:36078_solr) [n:127.0.0.1:36078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 843956 INFO  (parallelCoreAdminExecutor-822-thread-2-processing-n:127.0.0.1:36078_solr 74188428-f754-4b14-a1

[...truncated too long message...]

tasynccollectioncreation_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testasynccollectioncreation.shard1.replica_n3, tag=1475359117
   [junit4]   2> 902226 INFO  (coreCloseExecutor-892-thread-1) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@78f024a: rootName = solr_33807, domain = solr.core.testasynccollectioncreation.shard1.replica_n3, service url = null, agent id = null] for registry solr.core.testasynccollectioncreation.shard1.replica_n3 / com.codahale.metrics.MetricRegistry@228a2aa
   [junit4]   2> 902242 INFO  (coreCloseExecutor-892-thread-2) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1_0 r:core_node9 x:testasynccollectioncreation_shard1_0_replica0] o.a.s.c.SolrCore [testasynccollectioncreation_shard1_0_replica0]  CLOSING SolrCore org.apache.solr.core.SolrCore@2332e462
   [junit4]   2> 902261 INFO  (coreCloseExecutor-892-thread-3) [n:127.0.0.1:33807_solr c:testAsyncIdRaceCondition s:shard1 r:core_node2 x:testAsyncIdRaceCondition_shard1_replica_n1] o.a.s.c.SolrCore [testAsyncIdRaceCondition_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@67af9436
   [junit4]   2> 902266 INFO  (coreCloseExecutor-892-thread-1) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1 r:core_node4 x:testasynccollectioncreation_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testasynccollectioncreation.shard1.leader, tag=1475359117
   [junit4]   2> 902266 INFO  (coreCloseExecutor-892-thread-2) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1_0 r:core_node9 x:testasynccollectioncreation_shard1_0_replica0] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testasynccollectioncreation.shard1_0.replica0, tag=590537826
   [junit4]   2> 902266 INFO  (coreCloseExecutor-892-thread-2) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1_0 r:core_node9 x:testasynccollectioncreation_shard1_0_replica0] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6e82df21: rootName = solr_33807, domain = solr.core.testasynccollectioncreation.shard1_0.replica0, service url = null, agent id = null] for registry solr.core.testasynccollectioncreation.shard1_0.replica0 / com.codahale.metrics.MetricRegistry@7fd4e10e
   [junit4]   2> 902319 INFO  (coreCloseExecutor-892-thread-2) [n:127.0.0.1:33807_solr c:testasynccollectioncreation s:shard1_0 r:core_node9 x:testasynccollectioncreation_shard1_0_replica0] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testasynccollectioncreation.shard1_0.leader, tag=590537826
   [junit4]   2> 902457 INFO  (coreCloseExecutor-892-thread-3) [n:127.0.0.1:33807_solr c:testAsyncIdRaceCondition s:shard1 r:core_node2 x:testAsyncIdRaceCondition_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testAsyncIdRaceCondition.shard1.replica_n1, tag=1739559990
   [junit4]   2> 902457 INFO  (coreCloseExecutor-892-thread-3) [n:127.0.0.1:33807_solr c:testAsyncIdRaceCondition s:shard1 r:core_node2 x:testAsyncIdRaceCondition_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@57f33d4f: rootName = solr_33807, domain = solr.core.testAsyncIdRaceCondition.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.testAsyncIdRaceCondition.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@3bff2119
   [junit4]   2> 902459 INFO  (coreCloseExecutor-892-thread-3) [n:127.0.0.1:33807_solr c:testAsyncIdRaceCondition s:shard1 r:core_node2 x:testAsyncIdRaceCondition_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testAsyncIdRaceCondition.shard1.leader, tag=1739559990
   [junit4]   2> 902820 INFO  (zkCallback-635-thread-4-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [0])
   [junit4]   2> 902821 INFO  (zkCallback-635-thread-3-processing-n:127.0.0.1:33807_solr) [n:127.0.0.1:33807_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testasynccollectioncreation/state.json] for collection [testasynccollectioncreation] has occurred - updating... (live nodes size: [0])
   [junit4]   2> 902880 INFO  (jetty-closer-620-thread-1) [    ] o.a.s.c.Overseer Overseer (id=72278497690451973-127.0.0.1:33807_solr-n_0000000000) closing
   [junit4]   2> 902892 INFO  (OverseerStateUpdate-72278497690451973-127.0.0.1:33807_solr-n_0000000000) [n:127.0.0.1:33807_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:33807_solr
   [junit4]   2> 902917 WARN  (OverseerAutoScalingTriggerThread-72278497690451973-127.0.0.1:33807_solr-n_0000000000) [n:127.0.0.1:33807_solr    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 903022 INFO  (jetty-closer-620-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@60c83ebc{/solr,null,UNAVAILABLE}
   [junit4]   2> 903115 INFO  (jetty-closer-620-thread-1) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 903567 INFO  (jetty-closer-620-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@37ad796e{/solr,null,UNAVAILABLE}
   [junit4]   2> 903660 INFO  (jetty-closer-620-thread-2) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 903666 ERROR (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[995CC4B4A4EAA767]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 903666 INFO  (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[995CC4B4A4EAA767]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35951 35951
   [junit4]   2> 903843 INFO  (Thread-761) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35951 35951
   [junit4]   2> 903887 WARN  (Thread-761) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 19 /solr/collections/testAsyncOperations/terms/shard1
   [junit4]   2> 10 /solr/collections/testasynccollectioncreation/terms/shard1_0
   [junit4]   2> 9 /solr/collections/testasynccollectioncreation/terms/shard1
   [junit4]   2> 8 /solr/aliases.json
   [junit4]   2> 6 /solr/collections/testasynccollectioncreation/terms/shard1_1
   [junit4]   2> 4 /solr/collections/testAsyncOperations/terms/shard2
   [junit4]   2> 3 /solr/collections/testAsyncOperations/collectionprops.json
   [junit4]   2> 2 /solr/security.json
   [junit4]   2> 2 /solr/configs/conf1
   [junit4]   2> 2 /solr/collections/testasynccollectioncreation/collectionprops.json
   [junit4]   2> 2 /solr/collections/testAsyncIdRaceCondition/terms/shard1
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 51 /solr/collections/testasynccollectioncreation/state.json
   [junit4]   2> 35 /solr/collections/testAsyncOperations/state.json
   [junit4]   2> 8 /solr/clusterprops.json
   [junit4]   2> 8 /solr/clusterstate.json
   [junit4]   2> 6 /solr/collections/testAsyncIdRaceCondition/state.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 8 /solr/live_nodes
   [junit4]   2> 8 /solr/collections
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest_995CC4B4A4EAA767-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {id=FSTOrd50, _route_=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))}, docValues:{_version_=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=1631, maxMBSortInHeap=6.227140034484268, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@6a071679), locale=is, timezone=Europe/Dublin
   [junit4]   2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=267966064,total=521142272
   [junit4]   2> NOTE: All tests run in this JVM: [TestHdfsCloudBackupRestore, TestGenericDistributedQueue, TestJsonFacetsWithNestedObjects, HdfsChaosMonkeySafeLeaderTest, TestNodeLostTrigger, TestFastWriter, CoreSorterTest, CurrencyFieldTypeTest, TestRandomDVFaceting, TestNonDefinedSimilarityFactory, TestConfigOverlay, TestFieldResource, TestSchemaVersionResource, SparseHLLTest, CdcrRequestHandlerTest, AssignTest, TestBadConfig, TestReloadAndDeleteDocs, TestSolrCloudWithDelegationTokens, WordBreakSolrSpellCheckerTest, SolrCloudExampleTest, InfixSuggestersTest, DistanceFunctionTest, HdfsRecoveryZkTest, TestMaxScoreQueryParser, TestExportWriter, SolrCmdDistributorTest, TestJettySolrRunner, TestComplexPhraseLeadingWildcard, SpellPossibilityIteratorTest, TestCloudNestedDocsSort, TestLegacyField, TestSQLHandlerNonCloud, TestZkAclsWithHadoopAuth, ClusterStateUpdateTest, BigEndianAscendingWordSerializerTest, PeerSyncWithIndexFingerprintCachingTest, TestFaceting, AutoscalingHistoryHandlerTest, VMParamsZkACLAndCredentialsProvidersTest, NoCacheHeaderTest, TestSolrDeletionPolicy2, SuggesterTSTTest, FullHLLTest, TestCoreBackup, CollectionsAPIAsyncDistributedZkTest]
   [junit4] Completed [117/785 (1!)] on J2 in 74.91s, 4 tests, 1 error <<< FAILURES!

[...truncated 51771 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2424 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2424/

1 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeWithMultipleReplicasLost

Error Message:
The operations computed by ComputePlanAction should not be null SolrClientNodeStateProvider.DEBUG{AFTER_ACTION=[compute_plan, null], BEFORE_ACTION=[compute_plan, null]}

Stack Trace:
java.lang.AssertionError: The operations computed by ComputePlanAction should not be null SolrClientNodeStateProvider.DEBUG{AFTER_ACTION=[compute_plan, null], BEFORE_ACTION=[compute_plan, null]}
        at __randomizedtesting.SeedInfo.seed([DB3957B6117D9155:EBF9B634990F7009]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNotNull(Assert.java:526)
        at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeWithMultipleReplicasLost(ComputePlanActionTest.java:291)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 1834 lines...]
   [junit4] JVM J0: stdout was not empty, see: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/temp/junit4-J0-20180314_162046_28122566093485749154.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] codec: SimpleText, pf: BloomFilter, dvf: Asserting
   [junit4] <<< JVM J0: EOF ----

[...truncated 11881 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.ComputePlanActionTest
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.ComputePlanActionTest_DB3957B6117D9155-001/init-core-data-001
   [junit4]   2> 2300946 WARN  (SUITE-ComputePlanActionTest-seed#[DB3957B6117D9155]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=11 numCloses=11
   [junit4]   2> 2300946 INFO  (SUITE-ComputePlanActionTest-seed#[DB3957B6117D9155]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 2300959 INFO  (SUITE-ComputePlanActionTest-seed#[DB3957B6117D9155]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 2300960 INFO  (SUITE-ComputePlanActionTest-seed#[DB3957B6117D9155]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 2300961 INFO  (SUITE-ComputePlanActionTest-seed#[DB3957B6117D9155]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.ComputePlanActionTest_DB3957B6117D9155-001/tempDir-001
   [junit4]   2> 2300961 INFO  (SUITE-ComputePlanActionTest-seed#[DB3957B6117D9155]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2300975 INFO  (Thread-2620) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2300975 INFO  (Thread-2620) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2301012 ERROR (Thread-2620) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2301075 INFO  (SUITE-ComputePlanActionTest-seed#[DB3957B6117D9155]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:41341
   [junit4]   2> 2301137 INFO  (zkConnectionManagerCallback-2022-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2301199 INFO  (jetty-launcher-2019-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 2301254 INFO  (jetty-launcher-2019-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2301254 INFO  (jetty-launcher-2019-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2301254 INFO  (jetty-launcher-2019-thread-1) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 2301255 INFO  (jetty-launcher-2019-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e49f98{/solr,null,AVAILABLE}
   [junit4]   2> 2301256 INFO  (jetty-launcher-2019-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@60b2afff{SSL,[ssl, http/1.1]}{127.0.0.1:45294}
   [junit4]   2> 2301256 INFO  (jetty-launcher-2019-thread-1) [    ] o.e.j.s.Server Started @2303659ms
   [junit4]   2> 2301256 INFO  (jetty-launcher-2019-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45294}
   [junit4]   2> 2301256 ERROR (jetty-launcher-2019-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2301256 INFO  (jetty-launcher-2019-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 2301256 INFO  (jetty-launcher-2019-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2301256 INFO  (jetty-launcher-2019-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2301256 INFO  (jetty-launcher-2019-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-14T17:21:28.917Z
   [junit4]   2> 2301283 INFO  (zkConnectionManagerCallback-2024-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2301288 INFO  (jetty-launcher-2019-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2301340 INFO  (jetty-launcher-2019-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41341/solr
   [junit4]   2> 2301364 INFO  (zkConnectionManagerCallback-2028-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2301421 INFO  (zkConnectionManagerCallback-2030-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2301842 INFO  (jetty-launcher-2019-thread-1) [n:127.0.0.1:45294_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2301842 INFO  (jetty-launcher-2019-thread-1) [n:127.0.0.1:45294_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45294_solr
   [junit4]   2> 2301859 INFO  (jetty-launcher-2019-thread-1) [n:127.0.0.1:45294_solr    ] o.a.s.c.Overseer Overseer (id=72279465483698179-127.0.0.1:45294_solr-n_0000000000) starting
   [junit4]   2> 2301877 INFO  (zkConnectionManagerCallback-2035-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2301878 INFO  (jetty-launcher-2019-thread-1) [n:127.0.0.1:45294_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41341/solr ready
   [junit4]   2> 2301878 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2301969 INFO  (jetty-launcher-2019-thread-1) [n:127.0.0.1:45294_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45294_solr
   [junit4]   2> 2301969 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 2301971 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 2301972 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: []
   [junit4]   2> 2301972 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 2301972 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 2302022 INFO  (zkCallback-2029-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2302042 INFO  (zkCallback-2034-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2302237 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2302290 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 2302290 DEBUG (ScheduledTrigger-4021-thread-1) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 2302399 INFO  (jetty-launcher-2019-thread-1) [n:127.0.0.1:45294_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45294.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7367bc20
   [junit4]   2> 2302418 INFO  (jetty-launcher-2019-thread-1) [n:127.0.0.1:45294_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45294.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7367bc20
   [junit4]   2> 2302418 INFO  (jetty-launcher-2019-thread-1) [n:127.0.0.1:45294_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45294.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7367bc20
   [junit4]   2> 2302472 INFO  (jetty-launcher-2019-thread-1) [n:127.0.0.1:45294_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.ComputePlanActionTest_DB3957B6117D9155-001/tempDir-001/node1/.
   [junit4]   2> 2302564 INFO  (zkConnectionManagerCallback-2037-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2302584 INFO  (zkConnectionManagerCallback-2041-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2302584 INFO  (SUITE-ComputePlanActionTest-seed#[DB3957B6117D9155]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2302585 INFO  (SUITE-ComputePlanActionTest-seed#[DB3957B6117D9155]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41341/solr ready
   [junit4] IGNOR/A 0.00s J0 | ComputePlanActionTest.testSelectedCollections
   [junit4]    > Assumption #1: 'badapple' test group is disabled (@BadApple(bugUrl=https://issues.apache.org/jira/browse/SOLR-12028))
   [junit4]   2> 2302655 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAdded
   [junit4]   2> 2302656 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2302671 DEBUG (zkCallback-2034-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 2302671 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 2302746 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 2302746 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2302747 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 2302839 INFO  (qtp163944847-13324) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2302839 DEBUG (qtp163944847-13325) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 2302840 INFO  (qtp163944847-13325) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=58
   [junit4]   2> 2302841 DEBUG (zkCallback-2034-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 2302841 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 2302841 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 2302842 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2302854 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 2302855 INFO  (qtp163944847-13331) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2302890 DEBUG (qtp163944847-13327) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 2302895 INFO  (qtp163944847-13327) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=54
   [junit4]   2> 2302896 DEBUG (zkCallback-2034-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 2302896 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 2302897 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 2302915 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2302918 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 2302918 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2302931 DEBUG (qtp163944847-13324) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 2302934 DEBUG (zkCallback-2034-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 2302950 INFO  (qtp163944847-13324) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=54
   [junit4]   2> 2302970 DEBUG (zkCallback-2034-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:45294_solr]
   [junit4]   2> 2302970 DEBUG (zkCallback-2034-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=compute_plan, class=solr.ComputePlanAction}, {name=test, class=org.apache.solr.cloud.autoscaling.ComputePlanActionTest$AssertingTriggerAction}], enabled=true}
   [junit4]   2> 2302970 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 2302985 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 2302985 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 2302986 DEBUG (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 2302986 DEBUG (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 2302987 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2303002 DEBUG (qtp163944847-13324) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 2303003 DEBUG (zkCallback-2034-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 2303003 DEBUG (zkCallback-2034-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:45294_solr]
   [junit4]   2> 2303003 DEBUG (zkCallback-2034-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=compute_plan, class=solr.ComputePlanAction}, {name=test, class=org.apache.solr.cloud.autoscaling.ComputePlanActionTest$AssertingTriggerAction}], enabled=true}
   [junit4]   2> 2303003 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 2303003 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 2303003 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
   [junit4]   2> 2303004 INFO  (qtp163944847-13324) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=52
   [junit4]   2> 2303024 INFO  (qtp163944847-13327) [n:127.0.0.1:45294_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&name=testNodeAdded&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2303074 INFO  (OverseerThreadFactory-4023-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection testNodeAdded
   [junit4]   2> 2303074 WARN  (OverseerThreadFactory-4023-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.c.CreateCollectionCmd Specified number of replicas of 2 on collection testNodeAdded is higher than the number of Solr instances currently live or live and part of your createNodeSet(1). It's unusual to run two replica of the same slice on the same Solr-instance.
   [junit4]   2> 2303094 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2303094 INFO  (OverseerThreadFactory-4023-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 3385491692 sessionWrapper.createTime 3385491692555886, this.sessionWrapper.createTime 3385491692555886
   [junit4]   2> 2303109 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "name":"testNodeAdded",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "collection.configName":"conf",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"} current state version: 0
   [junit4]   2> 2303110 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ClusterStateMutator building a new cName: testNodeAdded
   [junit4]   2> 2303126 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/testNodeAdded/state.json
   [junit4]   2> 2303200 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 2303200 INFO  (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 2303201 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{}}
   [junit4]   2> 2303201 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 2303202 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 2303202 INFO  (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 2303220 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 2303220 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>       "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>       "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 2303320 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testNodeAdded/state.json version: 0
   [junit4]   2> 2303416 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testNodeAdded_shard1_replica_n1&action=CREATE&numShards=1&collection=testNodeAdded&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2303416 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2303456 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testNodeAdded_shard1_replica_n2&action=CREATE&numShards=1&collection=testNodeAdded&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2303489 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node4",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>   "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2303489 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node4",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>   "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2303490 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node4:{"core":"testNodeAdded_shard1_replica_n2","base_url":"https://127.0.0.1:45294/solr","node_name":"127.0.0.1:45294_solr","state":"down","type":"NRT"}
   [junit4]   2> 2303490 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(testNodeAdded//collections/testNodeAdded/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>           "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 2303490 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>   "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2303490 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>   "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2303490 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node3:{"core":"testNodeAdded_shard1_replica_n1","base_url":"https://127.0.0.1:45294/solr","node_name":"127.0.0.1:45294_solr","state":"down","type":"NRT"}
   [junit4]   2> 2303490 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(testNodeAdded//collections/testNodeAdded/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>           "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>           "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 2303597 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testNodeAdded/state.json version: 1
   [junit4]   2> 2303598 INFO  (zkCallback-2029-thread-2-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2303598 INFO  (zkCallback-2029-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2303990 DEBUG (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 2303990 DEBUG (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 2304531 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2304698 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.s.IndexSchema [testNodeAdded_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 2304699 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2304713 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2304713 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testNodeAdded_shard1_replica_n1' using configuration from collection testNodeAdded, trusted=true
   [junit4]   2> 2304714 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45294.solr.core.testNodeAdded.shard1.replica_n1' (registry 'solr.core.testNodeAdded.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7367bc20
   [junit4]   2> 2304714 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2304714 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.SolrCore [[testNodeAdded_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.ComputePlanActionTest_DB3957B6117D9155-001/tempDir-001/node1/testNodeAdded_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.ComputePlanActionTest_DB3957B6117D9155-001/tempDir-001/node1/./testNodeAdded_shard1_replica_n1/data/]
   [junit4]   2> 2304731 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.s.IndexSchema [testNodeAdded_shard1_replica_n2] Schema name=minimal
   [junit4]   2> 2304733 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2304733 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testNodeAdded_shard1_replica_n2' using configuration from collection testNodeAdded, trusted=true
   [junit4]   2> 2304733 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45294.solr.core.testNodeAdded.shard1.replica_n2' (registry 'solr.core.testNodeAdded.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7367bc20
   [junit4]   2> 2304733 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2304734 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.SolrCore [[testNodeAdded_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.ComputePlanActionTest_DB3957B6117D9155-001/tempDir-001/node1/testNodeAdded_shard1_replica_n2], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.ComputePlanActionTest_DB3957B6117D9155-001/tempDir-001/node1/./testNodeAdded_shard1_replica_n2/data/]
   [junit4]   2> 2304989 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2304989 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2304990 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2304990 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2304992 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@34d82063[testNodeAdded_shard1_replica_n1] main]
   [junit4]   2> 2304997 DEBUG (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 2304997 DEBUG (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 2305003 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2305003 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2305004 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2305004 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2305022 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@5d1368d6[testNodeAdded_shard1_replica_n2] main]
   [junit4]   2> 2305030 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2305030 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2305031 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2305032 INFO  (searcherExecutor-4026-thread-1-processing-n:127.0.0.1:45294_solr x:testNodeAdded_shard1_replica_n1 s:shard1 c:testNodeAdded r:core_node3) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.SolrCore [testNodeAdded_shard1_replica_n1] Registered new searcher Searcher@34d82063[testNodeAdded_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2305032 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594934524843655168
   [junit4]   2> 2305036 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update terms at /collections/testNodeAdded/terms/shard1 to Terms{values={core_node3=0}, version=0}
   [junit4]   2> 2305056 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 2305056 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded"} current state version: 0
   [junit4]   2> 2305058 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2305058 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2305059 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2305059 INFO  (searcherExecutor-4027-thread-1-processing-n:127.0.0.1:45294_solr x:testNodeAdded_shard1_replica_n2 s:shard1 c:testNodeAdded r:core_node4) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.SolrCore [testNodeAdded_shard1_replica_n2] Registered new searcher Searcher@5d1368d6[testNodeAdded_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2305059 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594934524871966720
   [junit4]   2> 2305079 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update terms at /collections/testNodeAdded/terms/shard1 to Terms{values={core_node3=0, core_node4=0}, version=1}
   [junit4]   2> 2305157 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testNodeAdded/state.json version: 2
   [junit4]   2> 2305163 INFO  (zkCallback-2029-thread-2-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2305171 INFO  (zkCallback-2029-thread-3-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2305557 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2305557 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2305557 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:45294/solr/testNodeAdded_shard1_replica_n1/
   [junit4]   2> 2305558 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=testNodeAdded_shard1_replica_n1 url=https://127.0.0.1:45294/solr START replicas=[https://127.0.0.1:45294/solr/testNodeAdded_shard1_replica_n2/] nUpdates=100
   [junit4]   2> 2305560 INFO  (qtp163944847-13331) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.S.Request [testNodeAdded_shard1_replica_n2]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2305869 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=testNodeAdded_shard1_replica_n1 url=https://127.0.0.1:45294/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 2305869 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 2305869 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 2305869 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2305897 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:45294/solr/testNodeAdded_shard1_replica_n1/ shard1
   [junit4]   2> 2305906 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>   "state":"active"} current state version: 0
   [junit4]   2> 2305998 DEBUG (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 2305998 DEBUG (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 2306019 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testNodeAdded/state.json version: 3
   [junit4]   2> 2306022 INFO  (zkCallback-2029-thread-3-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2306022 INFO  (zkCallback-2029-thread-2-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2306076 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2306077 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>   "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2306077 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>   "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2306078 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node3:{"core":"testNodeAdded_shard1_replica_n1","base_url":"https://127.0.0.1:45294/solr","node_name":"127.0.0.1:45294_solr","state":"active","type":"NRT","leader":"true"}
   [junit4]   2> 2306078 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(testNodeAdded//collections/testNodeAdded/state.json/4)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>           "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "leader":"true"},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>           "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 2306078 INFO  (qtp163944847-13328) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testNodeAdded_shard1_replica_n1&action=CREATE&numShards=1&collection=testNodeAdded&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2662
   [junit4]   2> 2306104 INFO  (qtp163944847-13329) [n:127.0.0.1:45294_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testNodeAdded_shard1_replica_n2&action=CREATE&numShards=1&collection=testNodeAdded&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2648
   [junit4]   2> 2306104 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node4",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>   "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2306104 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node4",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>   "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"testNodeAdded",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2306104 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node4:{"core":"testNodeAdded_shard1_replica_n2","base_url":"https://127.0.0.1:45294/solr","node_name":"127.0.0.1:45294_solr","state":"active","type":"NRT"}
   [junit4]   2> 2306105 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(testNodeAdded//collections/testNodeAdded/state.json/4)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>           "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "leader":"true"},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>           "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 2306106 INFO  (qtp163944847-13327) [n:127.0.0.1:45294_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 2306205 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testNodeAdded/state.json version: 4
   [junit4]   2> 2306206 INFO  (zkCallback-2029-thread-3-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2306206 INFO  (zkCallback-2029-thread-2-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2307005 DEBUG (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 2307005 DEBUG (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 2307086 INFO  (OverseerCollectionConfigSetProcessor-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2307112 INFO  (qtp163944847-13327) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&name=testNodeAdded&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=4087
   [junit4]   2> 2307188 INFO  (qtp163944847-13330) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.testNodeAdded.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.testNodeAdded.shard1.replica_n2:INDEX.sizeInBytes} status=0 QTime=0
   [junit4]   2> 2307190 INFO  (qtp163944847-13331) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
   [junit4]   2> 2307218 DEBUG (qtp163944847-13381) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 2307232 INFO  (qtp163944847-13381) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=84
   [junit4]   2> 2307242 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 2307257 DEBUG (zkCallback-2034-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
   [junit4]   2> 2307277 DEBUG (zkCallback-2034-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:45294_solr]
   [junit4]   2> 2307277 DEBUG (zkCallback-2034-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=compute_plan, class=solr.ComputePlanAction}, {name=test, class=org.apache.solr.cloud.autoscaling.ComputePlanActionTest$AssertingTriggerAction}], enabled=true}
   [junit4]   2> 2307277 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
   [junit4]   2> 2307277 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2307278 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2307278 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 2307278 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6bcf8062{/solr,null,AVAILABLE}
   [junit4]   2> 2307279 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@66791d66{SSL,[ssl, http/1.1]}{127.0.0.1:38800}
   [junit4]   2> 2307279 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.e.j.s.Server Started @2309682ms
   [junit4]   2> 2307279 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38800}
   [junit4]   2> 2307279 ERROR (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2307279 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 2307279 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2307279 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2307279 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-14T17:21:34.940Z
   [junit4]   2> 2307293 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 2307309 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
   [junit4]   2> 2307314 INFO  (zkConnectionManagerCallback-2044-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2307330 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2307419 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41341/solr
   [junit4]   2> 2307425 INFO  (zkConnectionManagerCallback-2048-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2307440 INFO  (zkConnectionManagerCallback-2050-thread-1-processing-n:127.0.0.1:38800_solr) [n:127.0.0.1:38800_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2307458 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [n:127.0.0.1:38800_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2307476 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [n:127.0.0.1:38800_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2307510 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [n:127.0.0.1:38800_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38800_solr
   [junit4]   2> 2307510 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:38800_solr"} current state version: 0
   [junit4]   2> 2307511 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:38800_solr
   [junit4]   2> 2307525 INFO  (zkCallback-2029-thread-2-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2307525 INFO  (zkCallback-2034-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2307525 INFO  (zkCallback-2040-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2307525 INFO  (zkCallback-2040-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2307598 INFO  (zkCallback-2049-thread-1-processing-n:127.0.0.1:38800_solr) [n:127.0.0.1:38800_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2307797 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [n:127.0.0.1:38800_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38800.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7367bc20
   [junit4]   2> 2307831 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [n:127.0.0.1:38800_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38800.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7367bc20
   [junit4]   2> 2307831 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [n:127.0.0.1:38800_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38800.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7367bc20
   [junit4]   2> 2307849 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [n:127.0.0.1:38800_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.ComputePlanActionTest_DB3957B6117D9155-001/tempDir-001/node2/.
   [junit4]   2> 2307876 INFO  (zkConnectionManagerCallback-2055-thread-1-processing-n:127.0.0.1:38800_solr) [n:127.0.0.1:38800_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2307877 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [n:127.0.0.1:38800_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2307886 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [n:127.0.0.1:38800_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41341/solr ready
   [junit4]   2> 2308011 DEBUG (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 2308011 DEBUG (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 2308012 DEBUG (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:38800_solr at time 3385496610064086
   [junit4]   2> 2309013 DEBUG (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 2309013 DEBUG (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 2309014 DEBUG (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:38800_solr] added at times [3385496610064086], now=3385497612079627
   [junit4]   2> 2309014 DEBUG (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"c07175ad4c6d6T1k4sp01ec6gc35zv21wpeaiqu",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":3385496610064086,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[3385496610064086],
   [junit4]   2>     "nodeNames":["127.0.0.1:38800_solr"]}}
   [junit4]   2> 2309014 DEBUG (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_trigger]
   [junit4]   2> 2309015 INFO  (ScheduledTrigger-4021-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
   [junit4]   2> 2309016 DEBUG (AutoscalingActionExecutor-4022-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"c07175ad4c6d6T1k4sp01ec6gc35zv21wpeaiqu",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":3385496610064086,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[3385496610064086],
   [junit4]   2>     "_enqueue_time_":3385497612989813,
   [junit4]   2>     "nodeNames":["127.0.0.1:38800_solr"]}}
   [junit4]   2> 2309016 DEBUG (AutoscalingActionExecutor-4022-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"c07175ad4c6d6T1k4sp01ec6gc35zv21wpeaiqu",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":3385496610064086,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[3385496610064086],
   [junit4]   2>     "_enqueue_time_":3385497612989813,
   [junit4]   2>     "nodeNames":["127.0.0.1:38800_solr"]}} with context properties: {BEFORE_ACTION=[compute_plan]}
   [junit4]   2> 2309152 INFO  (qtp1307586901-13390) [n:127.0.0.1:38800_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2309153 INFO  (qtp163944847-13330) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.testNodeAdded.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.testNodeAdded.shard1.replica_n2:INDEX.sizeInBytes} status=0 QTime=0
   [junit4]   2> 2309156 INFO  (qtp163944847-13381) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2309158 INFO  (AutoscalingActionExecutor-4022-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:38800_solr&inPlaceMove=true&replica=core_node3
   [junit4]   2> 2309159 INFO  (AutoscalingActionExecutor-4022-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 3385497757 sessionWrapper.createTime 3385497755120549, this.sessionWrapper.createTime 3385497755120549
   [junit4]   2> 2309168 DEBUG (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.a.s.c.a.ComputePlanActionTest -------------_ FINAL STATE --------------
   [junit4]   2> 2309204 INFO  (qtp1307586901-13388) [n:127.0.0.1:38800_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2309205 DEBUG (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.a.s.c.a.ComputePlanActionTest * Node values: 127.0.0.1:38800_solr
   [junit4]   2> {
   [junit4]   2>   "node":"127.0.0.1:38800_solr",
   [junit4]   2>   "cores":0,
   [junit4]   2>   "port":"38800",
   [junit4]   2>   "host":"127.0.0.1",
   [junit4]   2>   "ip_1":"1",
   [junit4]   2>   "ip_3":"0",
   [junit4]   2>   "ip_2":"0",
   [junit4]   2>   "ip_4":"127",
   [junit4]   2>   "freedisk":32.569313049316406}
   [junit4]   2> 2309209 DEBUG (AutoscalingActionExecutor-4022-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 5000ms
   [junit4]   2> 2309209 DEBUG (AutoscalingActionExecutor-4022-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.ScheduledTriggers -- processing took 193 ms for event id=c07175ad4c6d6T1k4sp01ec6gc35zv21wpeaiqu
   [junit4]   2> 2309211 INFO  (qtp163944847-13330) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2309212 DEBUG (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.a.s.c.a.ComputePlanActionTest * Node values: 127.0.0.1:45294_solr
   [junit4]   2> {
   [junit4]   2>   "node":"127.0.0.1:45294_solr",
   [junit4]   2>   "cores":2,
   [junit4]   2>   "port":"45294",
   [junit4]   2>   "host":"127.0.0.1",
   [junit4]   2>   "ip_1":"1",
   [junit4]   2>   "ip_3":"0",
   [junit4]   2>   "ip_2":"0",
   [junit4]   2>   "ip_4":"127",
   [junit4]   2>   "freedisk":32.569313049316406}
   [junit4]   2> 2309212 DEBUG (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.a.s.c.a.ComputePlanActionTest * Live nodes: [127.0.0.1:38800_solr, 127.0.0.1:45294_solr]
   [junit4]   2> 2309228 DEBUG (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.a.s.c.a.ComputePlanActionTest * Collection testNodeAdded state: DocCollection(testNodeAdded//collections/testNodeAdded/state.json/5)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n1",
   [junit4]   2>           "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>           "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "leader":"true"},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"testNodeAdded_shard1_replica_n2",
   [junit4]   2>           "base_url":"https://127.0.0.1:45294/solr",
   [junit4]   2>           "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT"}}}},
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0"}
   [junit4]   2> 2309228 INFO  (TEST-ComputePlanActionTest.testNodeAdded-seed#[DB3957B6117D9155]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAdded
   [junit4]   2> 2309478 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[DB3957B6117D9155]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLost
   [junit4]   2> 2309492 DEBUG (zkCallback-2034-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
   [junit4]   2> 2309493 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
   [junit4]   2> 2309494 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 2309494 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2309495 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[DB3957B6117D9155]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@60b2afff{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 2309495 DEBUG (OverseerAutoScalingTriggerThread-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
   [junit4]   2> 2309495 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[DB3957B6117D9155]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1254909344
   [junit4]   2> 2309496 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[DB3957B6117D9155]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 2309496 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[DB3957B6117D9155]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1cf8e8de: rootName = solr_45294, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@23d1ed89
   [junit4]   2> 2309581 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[DB3957B6117D9155]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 2309581 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[DB3957B6117D9155]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@75143f04: rootName = solr_45294, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@161950c6
   [junit4]   2> 2309601 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[DB3957B6117D9155]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 2309601 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[DB3957B6117D9155]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@35463e70: rootName = solr_45294, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@13b95e77
   [junit4]   2> 2309615 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[DB3957B6117D9155]) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:45294_solr
   [junit4]   2> 2309616 INFO  (zkCallback-2034-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 2309616 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:45294_solr"} current state version: 0
   [junit4]   2> 2309616 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:45294_solr
   [junit4]   2> 2309616 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.NodeMutator Update replica state for core_node3:{"core":"testNodeAdded_shard1_replica_n1","base_url":"https://127.0.0.1:45294/solr","node_name":"127.0.0.1:45294_solr","state":"active","type":"NRT","leader":"true"} to down
   [junit4]   2> 2309616 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.NodeMutator Update replica state for core_node4:{"core":"testNodeAdded_shard1_replica_n2","base_url":"https://127.0.0.1:45294/solr","node_name":"127.0.0.1:45294_solr","state":"active","type":"NRT"} to down
   [junit4]   2> 2309617 INFO  (TEST-ComputePlanActionTest.testNodeLost-seed#[DB3957B6117D9155]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 2309617 DEBUG (OverseerStateUpdate-72279465483698179-127.0.0.1:45294_solr-n_0000000000) [n:127.0.0.1:45294_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testNodeAdded/state.json version: 5
   [junit4]   2> 2309617 INFO  (zkCallback-2029-thread-2-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 2309617 INFO  (zkCallback-2029-thread-3-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2309617 INFO  (zkCallback-2029-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])

[...truncated too long message...]

icasLost.shard2.replica_n8 / com.codahale.metrics.MetricRegistry@3fee5cb5
   [junit4]   2> 2350310 INFO  (zkCallback-2120-thread-1-processing-n:127.0.0.1:35909_solr) [n:127.0.0.1:35909_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35909_solr
   [junit4]   2> 2350311 INFO  (jetty-closer-2020-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4869775e{/solr,null,UNAVAILABLE}
   [junit4]   2> 2350311 INFO  (jetty-closer-2020-thread-2) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 2350333 INFO  (coreCloseExecutor-4128-thread-2) [n:127.0.0.1:35909_solr c:testNodeWithMultipleReplicasLost s:shard2 r:core_node11 x:testNodeWithMultipleReplicasLost_shard2_replica_n8] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testNodeWithMultipleReplicasLost.shard2.leader, tag=971683436
   [junit4]   2> 2350334 INFO  (coreCloseExecutor-4128-thread-1) [n:127.0.0.1:35909_solr c:testNodeWithMultipleReplicasLost s:shard1 r:core_node7 x:testNodeWithMultipleReplicasLost_shard1_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testNodeWithMultipleReplicasLost.shard1.leader, tag=162747274
   [junit4]   2> 2350345 INFO  (jetty-closer-2020-thread-3) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3e4cbb3e{/solr,null,UNAVAILABLE}
   [junit4]   2> 2350349 INFO  (jetty-closer-2020-thread-3) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 2352511 WARN  (zkCallback-2094-thread-1-processing-n:127.0.0.1:33402_solr) [n:127.0.0.1:33402_solr c:testNodeWithMultipleReplicasLost s:shard2 r:core_node9 x:testNodeWithMultipleReplicasLost_shard2_replica_n6] o.a.s.c.SyncStrategy Closed, skipping sync up.
   [junit4]   2> 2352511 INFO  (zkCallback-2094-thread-1-processing-n:127.0.0.1:33402_solr) [n:127.0.0.1:33402_solr c:testNodeWithMultipleReplicasLost s:shard2 r:core_node9 x:testNodeWithMultipleReplicasLost_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 2352511 INFO  (zkCallback-2094-thread-1-processing-n:127.0.0.1:33402_solr) [n:127.0.0.1:33402_solr c:testNodeWithMultipleReplicasLost s:shard2 r:core_node9 x:testNodeWithMultipleReplicasLost_shard2_replica_n6] o.a.s.c.SolrCore [testNodeWithMultipleReplicasLost_shard2_replica_n6]  CLOSING SolrCore org.apache.solr.core.SolrCore@3ca575f
   [junit4]   2> 2352511 INFO  (zkCallback-2094-thread-1-processing-n:127.0.0.1:33402_solr) [n:127.0.0.1:33402_solr c:testNodeWithMultipleReplicasLost s:shard2 r:core_node9 x:testNodeWithMultipleReplicasLost_shard2_replica_n6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testNodeWithMultipleReplicasLost.shard2.replica_n6, tag=63592287
   [junit4]   2> 2352511 INFO  (zkCallback-2094-thread-1-processing-n:127.0.0.1:33402_solr) [n:127.0.0.1:33402_solr c:testNodeWithMultipleReplicasLost s:shard2 r:core_node9 x:testNodeWithMultipleReplicasLost_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@25efd08f: rootName = solr_33402, domain = solr.core.testNodeWithMultipleReplicasLost.shard2.replica_n6, service url = null, agent id = null] for registry solr.core.testNodeWithMultipleReplicasLost.shard2.replica_n6 / com.codahale.metrics.MetricRegistry@30732113
   [junit4]   2> 2352519 INFO  (zkCallback-2094-thread-1-processing-n:127.0.0.1:33402_solr) [n:127.0.0.1:33402_solr c:testNodeWithMultipleReplicasLost s:shard2 r:core_node9 x:testNodeWithMultipleReplicasLost_shard2_replica_n6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testNodeWithMultipleReplicasLost.shard2.leader, tag=63592287
   [junit4]   2> 2352520 INFO  (jetty-closer-2020-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@73810620{/solr,null,UNAVAILABLE}
   [junit4]   2> 2352521 INFO  (jetty-closer-2020-thread-1) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 2352521 ERROR (SUITE-ComputePlanActionTest-seed#[DB3957B6117D9155]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2352522 INFO  (SUITE-ComputePlanActionTest-seed#[DB3957B6117D9155]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41341 41341
   [junit4]   2> 2352621 INFO  (Thread-2620) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41341 41341
   [junit4]   2> 2352622 WARN  (Thread-2620) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 21 /solr/aliases.json
   [junit4]   2> 9 /solr/collections/testNodeWithMultipleReplicasLost/terms/shard2
   [junit4]   2> 9 /solr/collections/testNodeWithMultipleReplicasLost/terms/shard1
   [junit4]   2> 7 /solr/security.json
   [junit4]   2> 7 /solr/configs/conf
   [junit4]   2> 6 /solr/collections/testNodeLost/terms/shard1
   [junit4]   2> 4 /solr/collections/testNodeWithMultipleReplicasLost/collectionprops.json
   [junit4]   2> 3 /solr/collections/testNodeAdded/terms/shard1
   [junit4]   2> 2 /solr/collections/testNodeLost/collectionprops.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 24 /solr/collections/testNodeWithMultipleReplicasLost/state.json
   [junit4]   2> 21 /solr/clusterprops.json
   [junit4]   2> 21 /solr/clusterstate.json
   [junit4]   2> 16 /solr/collections/testNodeLost/state.json
   [junit4]   2> 7 /solr/collections/testNodeAdded/state.json
   [junit4]   2> 3 /solr/autoscaling.json
   [junit4]   2> 3 /solr/overseer_elect/election/72279465483698185-127.0.0.1:38800_solr-n_0000000001
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 29 /solr/live_nodes
   [junit4]   2> 28 /solr/collections
   [junit4]   2> 3 /solr/overseer/queue
   [junit4]   2> 3 /solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 3 /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 3 /solr/overseer/collection-queue-work
   [junit4]   2> 3 /solr/overseer/queue-work
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.ComputePlanActionTest_DB3957B6117D9155-001
   [junit4]   2> Mar 14, 2018 5:22:20 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@21b18dd5), locale=sr-ME, timezone=Africa/Bangui
   [junit4]   2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=263448656,total=518520832
   [junit4]   2> NOTE: All tests run in this JVM: [UpdateLogTest, TestManagedSchemaThreadSafety, SolrCoreCheckLockOnStartupTest, TestLocalFSCloudBackupRestore, RequestLoggingTest, CdcrVersionReplicationTest, DocExpirationUpdateProcessorFactoryTest, ClassificationUpdateProcessorIntegrationTest, MergeStrategyTest, AutoAddReplicasIntegrationTest, FullSolrCloudDistribCmdsTest, TestEmbeddedSolrServerConstructors, TestPerFieldSimilarityWithDefaultOverride, TestBinaryField, TestStressLucene, TestHalfAndHalfDocValues, TestComplexPhraseQParserPlugin, DistributedFacetPivotWhiteBoxTest, UUIDFieldTest, TestDeleteCollectionOnDownNodes, DocumentAnalysisRequestHandlerTest, SolrPluginUtilsTest, TestDynamicFieldResource, UpdateParamsTest, TestRandomFaceting, TestApiFramework, SearchRateTriggerTest, TokenizerChainTest, TestBlobHandler, SolrXmlInZkTest, BaseCdcrDistributedZkTest, TestHdfsUpdateLog, AliasIntegrationTest, TestQuerySenderNoQuery, V2ApiIntegrationTest, ConnectionReuseTest, SharedFSAutoReplicaFailoverTest, SolrIndexConfigTest, DistributedFacetPivotLongTailTest, RAMDirectoryFactoryTest, TestLegacyFieldReuse, TestSort, TestRuleBasedAuthorizationPlugin, TestHashPartitioner, HLLSerializationTest, TestMergePolicyConfig, ExternalFileFieldSortTest, TestDocSet, TestBulkSchemaAPI, WrapperMergePolicyFactoryTest, HttpSolrCallGetCoreTest, DocValuesNotIndexedTest, HttpPartitionOnCommitTest, TestStandardQParsers, SuggestComponentContextFilterQueryTest, RegexBytesRefFilterTest, TestInitQParser, TestNestedDocsSort, TestDistributedSearch, TestExpandComponent, TestInitParams, DateRangeFieldTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, NodeMutatorTest, BasicDistributedZkTest, TestRemoteStreaming, TestWriterPerf, TestSerializedLuceneMatchVersion, TestFileDictionaryLookup, MetricsConfigTest, TestAddFieldRealTimeGet, TestStressInPlaceUpdates, CreateRoutedAliasTest, TestFieldCache, TestPullReplica, TestOmitPositions, TestTolerantUpdateProcessorCloud, AtomicUpdateProcessorFactoryTest, CollectionsAPIDistributedZkTest, TestIndexingPerformance, TestPhraseSuggestions, TestCloudPivotFacet, TestLargeCluster, InfoHandlerTest, MigrateRouteKeyTest, TestRandomRequestDistribution, TestSchemalessBufferedUpdates, TestCloudDeleteByQuery, TestMissingGroups, PluginInfoTest, TestGroupingSearch, TestSolrXml, TestConfigSetImmutable, TestUseDocValuesAsStored2, TestTrackingShardHandlerFactory, OpenCloseCoreStressTest, TestManagedResource, HdfsRestartWhileUpdatingTest, NodeLostTriggerTest, TestIndexSearcher, DirectoryFactoryTest, TestMultiValuedNumericRangeQuery, TestDynamicLoading, TestRestoreCore, TestCoreAdminApis, TestPolicyCloud, TestMacros, TestLMDirichletSimilarityFactory, SpellCheckCollatorWithCollapseTest, TestMaxTokenLenTokenizer, ComputePlanActionTest]
   [junit4] Completed [510/785 (1!)] on J0 in 52.58s, 4 tests, 1 failure, 1 skipped <<< FAILURES!

[...truncated 50545 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2425 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2425/

9 tests failed.
FAILED:  org.apache.solr.cloud.ReplaceNodeTest.test

Error Message:


Stack Trace:
java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([BA4CB527817F8441:32188AFD2F83E9B9]:0)
        at org.junit.Assert.fail(Assert.java:92)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertFalse(Assert.java:68)
        at org.junit.Assert.assertFalse(Assert.java:79)
        at org.apache.solr.cloud.ReplaceNodeTest.test(ReplaceNodeTest.java:99)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testNoSsl

Error Message:
Could not find collection:first_collection

Stack Trace:
java.lang.AssertionError: Could not find collection:first_collection
        at __randomizedtesting.SeedInfo.seed([BA4CB527817F8441:89F491A9227FB887]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNotNull(Assert.java:526)
        at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
        at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkCreateCollection(TestMiniSolrCloudClusterSSL.java:202)
        at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkClusterWithCollectionCreations(TestMiniSolrCloudClusterSSL.java:172)
        at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkClusterWithNodeReplacement(TestMiniSolrCloudClusterSSL.java:138)
        at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testNoSsl(TestMiniSolrCloudClusterSSL.java:93)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestMiniSolrCloudClusterSSL

Error Message:
8 threads leaked from SUITE scope at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL:     1) Thread[id=20524, name=zkConnectionManagerCallback-3991-thread-1, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    2) Thread[id=20488, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)    3) Thread[id=20502, name=zkConnectionManagerCallback-3980-thread-1, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    4) Thread[id=20489, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)    5) Thread[id=20500, name=jetty-launcher-3953-thread-1-EventThread, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)    6) Thread[id=20519, name=Thread-4118, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1731)    7) Thread[id=20521, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)    8) Thread[id=20499, name=jetty-launcher-3953-thread-1-SendThread(127.0.0.1:36352), state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 8 threads leaked from SUITE scope at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL:
   1) Thread[id=20524, name=zkConnectionManagerCallback-3991-thread-1, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   2) Thread[id=20488, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
   3) Thread[id=20502, name=zkConnectionManagerCallback-3980-thread-1, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   4) Thread[id=20489, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
   5) Thread[id=20500, name=jetty-launcher-3953-thread-1-EventThread, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
   6) Thread[id=20519, name=Thread-4118, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at org.apache.solr.core.CloserThread.run(CoreContainer.java:1731)
   7) Thread[id=20521, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
   8) Thread[id=20499, name=jetty-launcher-3953-thread-1-SendThread(127.0.0.1:36352), state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)
        at __randomizedtesting.SeedInfo.seed([BA4CB527817F8441]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestMiniSolrCloudClusterSSL

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=20524, name=zkConnectionManagerCallback-3991-thread-1, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    2) Thread[id=20502, name=zkConnectionManagerCallback-3980-thread-1, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    3) Thread[id=20519, name=Thread-4118, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1731)    4) Thread[id=20499, name=jetty-launcher-3953-thread-1-SendThread(127.0.0.1:36352), state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=20524, name=zkConnectionManagerCallback-3991-thread-1, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   2) Thread[id=20502, name=zkConnectionManagerCallback-3980-thread-1, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   3) Thread[id=20519, name=Thread-4118, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at org.apache.solr.core.CloserThread.run(CoreContainer.java:1731)
   4) Thread[id=20499, name=jetty-launcher-3953-thread-1-SendThread(127.0.0.1:36352), state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)
        at __randomizedtesting.SeedInfo.seed([BA4CB527817F8441]:0)


FAILED:  org.apache.solr.cloud.api.collections.CustomCollectionTest.testCustomCollectionsAPI

Error Message:
Error from server at https://127.0.0.1:33757/solr: Error creating counter node in Zookeeper for collection:implicitcoll

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:33757/solr: Error creating counter node in Zookeeper for collection:implicitcoll
        at __randomizedtesting.SeedInfo.seed([BA4CB527817F8441:D0AD3B4CBCE53239]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1105)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:885)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:818)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.api.collections.CustomCollectionTest.testCustomCollectionsAPI(CustomCollectionTest.java:104)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.TestPolicyCloud.testDataProvider

Error Message:
Error from server at http://127.0.0.1:34036/solr: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qnr-0000000032

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:34036/solr: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qnr-0000000032
        at __randomizedtesting.SeedInfo.seed([BA4CB527817F8441:82E71D1233BD8098]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1105)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:885)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:818)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.autoscaling.TestPolicyCloud.testDataProvider(TestPolicyCloud.java:327)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.handler.TestSQLHandler.doTest

Error Message:
--> http://127.0.0.1:36460/collection1_shard2_replica_n41:Failed to execute sqlQuery 'select str_s as myString, count(*) as myCount, sum(field_i) as mySum, min(field_i) as myMin, max(field_i) as myMax, avg(field_i) as myAvg from collection1 where (text='XXXX' AND NOT (text='XXXY')) group by str_s order by str_s desc' against JDBC connection 'jdbc:calcitesolr:'. Error while executing SQL "select str_s as myString, count(*) as myCount, sum(field_i) as mySum, min(field_i) as myMin, max(field_i) as myMax, avg(field_i) as myAvg from collection1 where (text='XXXX' AND NOT (text='XXXY')) group by str_s order by str_s desc": java.io.IOException: org.apache.solr.common.SolrException: Could not load collection from ZK: control_collection

Stack Trace:
java.io.IOException: --> http://127.0.0.1:36460/collection1_shard2_replica_n41:Failed to execute sqlQuery 'select str_s as myString, count(*) as myCount, sum(field_i) as mySum, min(field_i) as myMin, max(field_i) as myMax, avg(field_i) as myAvg from collection1 where (text='XXXX' AND NOT (text='XXXY')) group by str_s order by str_s desc' against JDBC connection 'jdbc:calcitesolr:'.
Error while executing SQL "select str_s as myString, count(*) as myCount, sum(field_i) as mySum, min(field_i) as myMin, max(field_i) as myMax, avg(field_i) as myAvg from collection1 where (text='XXXX' AND NOT (text='XXXY')) group by str_s order by str_s desc": java.io.IOException: org.apache.solr.common.SolrException: Could not load collection from ZK: control_collection
        at __randomizedtesting.SeedInfo.seed([BA4CB527817F8441:1D080D83ECC497F8]:0)
        at org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:222)
        at org.apache.solr.handler.TestSQLHandler.getTuples(TestSQLHandler.java:2522)
        at org.apache.solr.handler.TestSQLHandler.testBasicGrouping(TestSQLHandler.java:788)
        at org.apache.solr.handler.TestSQLHandler.doTest(TestSQLHandler.java:85)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestSQLHandler

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.handler.TestSQLHandler:     1) Thread[id=14502, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestSQLHandler]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestSQLHandler:
   1) Thread[id=14502, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestSQLHandler]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([BA4CB527817F8441]:0)


FAILED:  org.apache.solr.security.TestPKIAuthenticationPlugin.test

Error Message:


Stack Trace:
java.lang.NullPointerException
        at __randomizedtesting.SeedInfo.seed([BA4CB527817F8441:32188AFD2F83E9B9]:0)
        at org.apache.solr.security.TestPKIAuthenticationPlugin.test(TestPKIAuthenticationPlugin.java:102)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 1924 lines...]
   [junit4] JVM J0: stdout was not empty, see: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/temp/junit4-J0-20180314_212244_8677924689177308492262.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] codec: DummyCompressingStoredFields, pf: Memory, dvf: Asserting
   [junit4] <<< JVM J0: EOF ----

[...truncated 10450 lines...]
   [junit4] Suite: org.apache.solr.cloud.api.collections.CustomCollectionTest
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.CustomCollectionTest_BA4CB527817F8441-001/init-core-data-001
   [junit4]   2> 501418 WARN  (SUITE-CustomCollectionTest-seed#[BA4CB527817F8441]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=21 numCloses=21
   [junit4]   2> 501419 INFO  (SUITE-CustomCollectionTest-seed#[BA4CB527817F8441]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 501420 INFO  (SUITE-CustomCollectionTest-seed#[BA4CB527817F8441]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 501438 INFO  (SUITE-CustomCollectionTest-seed#[BA4CB527817F8441]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 501438 INFO  (SUITE-CustomCollectionTest-seed#[BA4CB527817F8441]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.CustomCollectionTest_BA4CB527817F8441-001/tempDir-001
   [junit4]   2> 501438 INFO  (SUITE-CustomCollectionTest-seed#[BA4CB527817F8441]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 501466 INFO  (Thread-378) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 501466 INFO  (Thread-378) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 501545 ERROR (Thread-378) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 501580 INFO  (SUITE-CustomCollectionTest-seed#[BA4CB527817F8441]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:34330
   [junit4]   2> 501661 INFO  (zkConnectionManagerCallback-413-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 501744 INFO  (jetty-launcher-410-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:57:37+05:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 501780 INFO  (jetty-launcher-410-thread-2) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:57:37+05:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 501817 INFO  (jetty-launcher-410-thread-3) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:57:37+05:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 501855 INFO  (jetty-launcher-410-thread-4) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:57:37+05:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 501889 INFO  (jetty-launcher-410-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 501889 INFO  (jetty-launcher-410-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 501889 INFO  (jetty-launcher-410-thread-3) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 501889 INFO  (jetty-launcher-410-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5bc8430d{/solr,null,AVAILABLE}
   [junit4]   2> 501892 INFO  (jetty-launcher-410-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4f6e9381{SSL,[ssl, http/1.1]}{127.0.0.1:40714}
   [junit4]   2> 501892 INFO  (jetty-launcher-410-thread-3) [    ] o.e.j.s.Server Started @523411ms
   [junit4]   2> 501892 INFO  (jetty-launcher-410-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40714}
   [junit4]   2> 501892 ERROR (jetty-launcher-410-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 501892 INFO  (jetty-launcher-410-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 501892 INFO  (jetty-launcher-410-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 501892 INFO  (jetty-launcher-410-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 501905 INFO  (jetty-launcher-410-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-15T00:08:20.020Z
   [junit4]   2> 502015 INFO  (jetty-launcher-410-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 502015 INFO  (jetty-launcher-410-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 502015 INFO  (jetty-launcher-410-thread-1) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 502043 INFO  (jetty-launcher-410-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@55b6bfaa{/solr,null,AVAILABLE}
   [junit4]   2> 502044 INFO  (jetty-launcher-410-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5dfce57{SSL,[ssl, http/1.1]}{127.0.0.1:40580}
   [junit4]   2> 502044 INFO  (jetty-launcher-410-thread-1) [    ] o.e.j.s.Server Started @523563ms
   [junit4]   2> 502044 INFO  (jetty-launcher-410-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40580}
   [junit4]   2> 502044 ERROR (jetty-launcher-410-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 502044 INFO  (jetty-launcher-410-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 502044 INFO  (jetty-launcher-410-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 502044 INFO  (jetty-launcher-410-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 502044 INFO  (jetty-launcher-410-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-15T00:08:20.172Z
   [junit4]   2> 502104 INFO  (jetty-launcher-410-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 502104 INFO  (jetty-launcher-410-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 502104 INFO  (jetty-launcher-410-thread-4) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 502105 INFO  (jetty-launcher-410-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@227c78cf{/solr,null,AVAILABLE}
   [junit4]   2> 502105 INFO  (jetty-launcher-410-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2f5ea5e6{SSL,[ssl, http/1.1]}{127.0.0.1:45917}
   [junit4]   2> 502105 INFO  (jetty-launcher-410-thread-4) [    ] o.e.j.s.Server Started @523624ms
   [junit4]   2> 502105 INFO  (jetty-launcher-410-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45917}
   [junit4]   2> 502106 ERROR (jetty-launcher-410-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 502106 INFO  (jetty-launcher-410-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 502106 INFO  (jetty-launcher-410-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 502106 INFO  (jetty-launcher-410-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 502106 INFO  (jetty-launcher-410-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-15T00:08:20.234Z
   [junit4]   2> 502117 INFO  (zkConnectionManagerCallback-417-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 502118 INFO  (jetty-launcher-410-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 502118 INFO  (jetty-launcher-410-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 502118 INFO  (jetty-launcher-410-thread-2) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 502118 INFO  (jetty-launcher-410-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@70b01aac{/solr,null,AVAILABLE}
   [junit4]   2> 502141 INFO  (zkConnectionManagerCallback-415-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 502161 INFO  (jetty-launcher-410-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 502161 INFO  (zkConnectionManagerCallback-419-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 502172 INFO  (jetty-launcher-410-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7c2c368e{SSL,[ssl, http/1.1]}{127.0.0.1:33757}
   [junit4]   2> 502172 INFO  (jetty-launcher-410-thread-2) [    ] o.e.j.s.Server Started @523691ms
   [junit4]   2> 502172 INFO  (jetty-launcher-410-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33757}
   [junit4]   2> 502172 ERROR (jetty-launcher-410-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 502172 INFO  (jetty-launcher-410-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 502172 INFO  (jetty-launcher-410-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 502172 INFO  (jetty-launcher-410-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 502172 INFO  (jetty-launcher-410-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-15T00:08:20.300Z
   [junit4]   2> 502177 INFO  (jetty-launcher-410-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 502177 INFO  (jetty-launcher-410-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 502214 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100cb3ef4080002, likely client has closed socket
   [junit4]   2> 502237 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100cb3ef4080001, likely client has closed socket
   [junit4]   2> 502237 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100cb3ef4080003, likely client has closed socket
   [junit4]   2> 502263 INFO  (jetty-launcher-410-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34330/solr
   [junit4]   2> 502320 INFO  (jetty-launcher-410-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34330/solr
   [junit4]   2> 502356 INFO  (zkConnectionManagerCallback-425-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 502363 INFO  (zkConnectionManagerCallback-421-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 502466 INFO  (jetty-launcher-410-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34330/solr
   [junit4]   2> 502471 INFO  (jetty-launcher-410-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 502471 INFO  (zkConnectionManagerCallback-429-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 502490 INFO  (zkConnectionManagerCallback-433-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 502492 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100cb3ef4080006, likely client has closed socket
   [junit4]   2> 514543 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100cb3ef4080005, likely client has closed socket
   [junit4]   2> 514544 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100cb3ef4080007, likely client has closed socket
   [junit4]   2> 514624 INFO  (jetty-launcher-410-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34330/solr
   [junit4]   2> 514688 INFO  (zkConnectionManagerCallback-435-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 515086 INFO  (zkConnectionManagerCallback-439-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 515089 INFO  (zkConnectionManagerCallback-437-thread-1-processing-n:127.0.0.1:45917_solr) [n:127.0.0.1:45917_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 515089 INFO  (zkConnectionManagerCallback-443-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 515239 INFO  (zkConnectionManagerCallback-445-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 515750 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:45917_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 515751 INFO  (jetty-launcher-410-thread-3) [n:127.0.0.1:40714_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 515751 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:45917_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45917_solr
   [junit4]   2> 515753 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:45917_solr    ] o.a.s.c.Overseer Overseer (id=72281065280503817-127.0.0.1:45917_solr-n_0000000000) starting
   [junit4]   2> 515753 INFO  (jetty-launcher-410-thread-3) [n:127.0.0.1:40714_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40714_solr
   [junit4]   2> 515872 INFO  (zkCallback-438-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 515913 INFO  (zkCallback-436-thread-1-processing-n:127.0.0.1:45917_solr) [n:127.0.0.1:45917_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 515929 INFO  (zkCallback-434-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 515950 INFO  (jetty-launcher-410-thread-1) [n:127.0.0.1:40580_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 516001 INFO  (jetty-launcher-410-thread-1) [n:127.0.0.1:40580_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40580_solr
   [junit4]   2> 516156 INFO  (zkConnectionManagerCallback-453-thread-1-processing-n:127.0.0.1:45917_solr) [n:127.0.0.1:45917_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 516160 INFO  (zkCallback-438-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 516164 INFO  (zkCallback-434-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 516176 INFO  (zkCallback-436-thread-1-processing-n:127.0.0.1:45917_solr) [n:127.0.0.1:45917_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 516229 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:45917_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 516302 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:45917_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34330/solr ready
   [junit4]   2> 516323 INFO  (jetty-launcher-410-thread-3) [n:127.0.0.1:40714_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40714.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc08bfb
   [junit4]   2> 516361 INFO  (jetty-launcher-410-thread-3) [n:127.0.0.1:40714_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40714.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc08bfb
   [junit4]   2> 516361 INFO  (jetty-launcher-410-thread-3) [n:127.0.0.1:40714_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40714.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc08bfb
   [junit4]   2> 516375 INFO  (jetty-launcher-410-thread-3) [n:127.0.0.1:40714_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.CustomCollectionTest_BA4CB527817F8441-001/tempDir-001/node3/.
   [junit4]   2> 516381 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:45917_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45917_solr
   [junit4]   2> 516400 INFO  (zkCallback-438-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 516403 INFO  (zkCallback-436-thread-1-processing-n:127.0.0.1:45917_solr) [n:127.0.0.1:45917_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 516419 INFO  (zkCallback-434-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 516468 INFO  (zkCallback-452-thread-1-processing-n:127.0.0.1:45917_solr) [n:127.0.0.1:45917_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 516667 INFO  (jetty-launcher-410-thread-1) [n:127.0.0.1:40580_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40580.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc08bfb
   [junit4]   2> 516815 INFO  (zkConnectionManagerCallback-457-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 516903 INFO  (jetty-launcher-410-thread-1) [n:127.0.0.1:40580_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40580.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc08bfb
   [junit4]   2> 516903 INFO  (jetty-launcher-410-thread-1) [n:127.0.0.1:40580_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40580.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc08bfb
   [junit4]   2> 516904 INFO  (jetty-launcher-410-thread-1) [n:127.0.0.1:40580_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.CustomCollectionTest_BA4CB527817F8441-001/tempDir-001/node1/.
   [junit4]   2> 516963 INFO  (jetty-launcher-410-thread-3) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 516976 INFO  (jetty-launcher-410-thread-3) [n:127.0.0.1:40714_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34330/solr ready
   [junit4]   2> 517065 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:45917_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45917.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc08bfb
   [junit4]   2> 517100 INFO  (zkConnectionManagerCallback-461-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 517210 INFO  (jetty-launcher-410-thread-1) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 517211 INFO  (jetty-launcher-410-thread-1) [n:127.0.0.1:40580_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34330/solr ready
   [junit4]   2> 517267 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:45917_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45917.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc08bfb
   [junit4]   2> 517267 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:45917_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45917.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc08bfb
   [junit4]   2> 517269 INFO  (jetty-launcher-410-thread-4) [n:127.0.0.1:45917_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.CustomCollectionTest_BA4CB527817F8441-001/tempDir-001/node4/.
   [junit4]   2> 517813 INFO  (jetty-launcher-410-thread-2) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 517847 INFO  (jetty-launcher-410-thread-2) [n:127.0.0.1:33757_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 517849 INFO  (jetty-launcher-410-thread-2) [n:127.0.0.1:33757_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33757_solr
   [junit4]   2> 517867 INFO  (zkCallback-434-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 517920 INFO  (zkCallback-436-thread-1-processing-n:127.0.0.1:45917_solr) [n:127.0.0.1:45917_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 517920 INFO  (zkCallback-438-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 517941 INFO  (zkCallback-452-thread-1-processing-n:127.0.0.1:45917_solr) [n:127.0.0.1:45917_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 518017 INFO  (zkCallback-444-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 518017 INFO  (zkCallback-460-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 518041 INFO  (zkCallback-456-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 518616 INFO  (jetty-launcher-410-thread-2) [n:127.0.0.1:33757_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33757.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc08bfb
   [junit4]   2> 518957 INFO  (jetty-launcher-410-thread-2) [n:127.0.0.1:33757_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33757.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc08bfb
   [junit4]   2> 518958 INFO  (jetty-launcher-410-thread-2) [n:127.0.0.1:33757_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33757.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc08bfb
   [junit4]   2> 518959 INFO  (jetty-launcher-410-thread-2) [n:127.0.0.1:33757_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.api.collections.CustomCollectionTest_BA4CB527817F8441-001/tempDir-001/node2/.
   [junit4]   2> 519174 INFO  (zkConnectionManagerCallback-465-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 519175 INFO  (jetty-launcher-410-thread-2) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 519189 INFO  (jetty-launcher-410-thread-2) [n:127.0.0.1:33757_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34330/solr ready
   [junit4]   2> 544548 WARN  (jetty-launcher-410-thread-2-SendThread(127.0.0.1:34330)) [n:127.0.0.1:33757_solr    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 25541ms for sessionid 0x100cb3ef408000c
   [junit4]   2> 544556 WARN  (jetty-launcher-410-thread-4-SendThread(127.0.0.1:34330)) [n:127.0.0.1:45917_solr    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 26138ms for sessionid 0x100cb3ef4080009
   [junit4]   2> 544556 WARN  (jetty-launcher-410-thread-3-SendThread(127.0.0.1:34330)) [n:127.0.0.1:40714_solr    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 26637ms for sessionid 0x100cb3ef408000b
   [junit4]   2> 544556 WARN  (jetty-launcher-410-thread-1-SendThread(127.0.0.1:34330)) [n:127.0.0.1:40580_solr    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 26689ms for sessionid 0x100cb3ef4080008
   [junit4]   2> 544565 WARN  (jetty-launcher-410-thread-2-SendThread(127.0.0.1:34330)) [n:127.0.0.1:33757_solr    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 25376ms for sessionid 0x100cb3ef4080010
   [junit4]   2> 544565 WARN  (jetty-launcher-410-thread-4-SendThread(127.0.0.1:34330)) [n:127.0.0.1:45917_solr    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 25713ms for sessionid 0x100cb3ef408000d
   [junit4]   2> 544565 WARN  (jetty-launcher-410-thread-3-SendThread(127.0.0.1:34330)) [n:127.0.0.1:40714_solr    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 26609ms for sessionid 0x100cb3ef408000e
   [junit4]   2> 544565 WARN  (jetty-launcher-410-thread-1-SendThread(127.0.0.1:34330)) [n:127.0.0.1:40580_solr    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 26548ms for sessionid 0x100cb3ef408000f
   [junit4]   2> 544780 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100cb3ef408000c, likely client has closed socket
   [junit4]   2> 544881 WARN  (zkConnectionManagerCallback-445-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2303acd1 name: ZooKeeperConnection Watcher:127.0.0.1:34330/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 544881 WARN  (zkConnectionManagerCallback-445-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 544987 WARN  (zkConnectionManagerCallback-437-thread-1-processing-n:127.0.0.1:45917_solr) [n:127.0.0.1:45917_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2c5a1386 name: ZooKeeperConnection Watcher:127.0.0.1:34330/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 544987 WARN  (zkConnectionManagerCallback-437-thread-1-processing-n:127.0.0.1:45917_solr) [n:127.0.0.1:45917_solr    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 545015 WARN  (zkConnectionManagerCallback-457-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@69c4724a name: ZooKeeperConnection Watcher:127.0.0.1:34330/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 545015 WARN  (zkConnectionManagerCallback-457-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 545018 WARN  (zkConnectionManagerCallback-465-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3ec59ee5 name: ZooKeeperConnection Watcher:127.0.0.1:34330/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 545018 WARN  (zkConnectionManagerCallback-465-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 545023 WARN  (zkConnectionManagerCallback-453-thread-1-processing-n:127.0.0.1:45917_solr) [n:127.0.0.1:45917_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@57dbc5d8 name: ZooKeeperConnection Watcher:127.0.0.1:34330/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 545023 WARN  (zkConnectionManagerCallback-453-thread-1-processing-n:127.0.0.1:45917_solr) [n:127.0.0.1:45917_solr    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 545044 WARN  (zkConnectionManagerCallback-435-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@70f0ca1e name: ZooKeeperConnection Watcher:127.0.0.1:34330/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 545044 WARN  (zkConnectionManagerCallback-435-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 545057 WARN  (zkConnectionManagerCallback-461-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1a739bf2 name: ZooKeeperConnection Watcher:127.0.0.1:34330/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 545057 WARN  (zkConnectionManagerCallback-461-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 545060 WARN  (zkConnectionManagerCallback-439-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@401f84ef name: ZooKeeperConnection Watcher:127.0.0.1:34330/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 545060 WARN  (zkConnectionManagerCallback-439-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 545210 INFO  (zkConnectionManagerCallback-467-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 546030 WARN  (jetty-launcher-410-thread-2-SendThread(127.0.0.1:34330)) [n:127.0.0.1:33757_solr    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100cb3ef4080010 has expired
   [junit4]   2> 546043 WARN  (zkConnectionManagerCallback-465-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3ec59ee5 name: ZooKeeperConnection Watcher:127.0.0.1:34330/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 546043 WARN  (zkConnectionManagerCallback-465-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 546044 WARN  (zkConnectionManagerCallback-465-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 546117 WARN  (jetty-launcher-410-thread-3-SendThread(127.0.0.1:34330)) [n:127.0.0.1:40714_solr    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100cb3ef408000b has expired
   [junit4]   2> 546120 WARN  (zkConnectionManagerCallback-439-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@401f84ef name: ZooKeeperConnection Watcher:127.0.0.1:34330/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 546120 WARN  (zkConnectionManagerCallback-439-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 546121 INFO  (zkConnectionManagerCallback-439-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 546140 INFO  (zkConnectionManagerCallback-465-thread-1-processing-n:127.0.0.1:33757_solr-EventThread) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 546140 INFO  (zkConnectionManagerCallback-465-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 546121 WARN  (zkConnectionManagerCallback-439-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 546153 WARN  (jetty-launcher-410-thread-1-SendThread(127.0.0.1:34330)) [n:127.0.0.1:40580_solr    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100cb3ef4080008 has expired
   [junit4]   2> 546177 WARN  (zkConnectionManagerCallback-435-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@70f0ca1e name: ZooKeeperConnection Watcher:127.0.0.1:34330/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 546177 WARN  (zkConnectionManagerCallback-435-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 546177 INFO  (zkConnectionManagerCallback-435-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 546177 WARN  (zkConnectionManagerCallback-435-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 546228 WARN  (jetty-launcher-410-thread-3-SendThread(127.0.0.1:34330)) [n:127.0.0.1:40714_solr    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100cb3ef408000e has expired
   [junit4]   2> 546234 WARN  (zkConnectionManagerCallback-457-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@69c4724a name: ZooKeeperConnection Watcher:127.0.0.1:34330/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 546234 WARN  (zkConnectionManagerCallback-457-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 546234 WARN  (zkConnectionManagerCallback-457-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 546262 INFO  (zkConnectionManagerCallback-439-thread-1-processing-n:127.0.0.1:40714_solr-EventThread) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 546263 INFO  (zkConnectionManagerCallback-439-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 546263 INFO  (zkConnectionManagerCallback-439-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 546265 INFO  (zkConnectionManagerCallback-457-thread-1-processing-n:127.0.0.1:40714_solr-EventThread) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 546266 INFO  (zkConnectionManagerCallback-457-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 546285 INFO  (zkConnectionManagerCallback-465-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (0)
   [junit4]   2> 546319 WARN  (jetty-launcher-410-thread-1-SendThread(127.0.0.1:34330)) [n:127.0.0.1:40580_solr    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100cb3ef408000f has expired
   [junit4]   2> 546323 INFO  (zkConnectionManagerCallback-435-thread-1-processing-n:127.0.0.1:40580_solr-EventThread) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 546323 INFO  (zkConnectionManagerCallback-435-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 546323 INFO  (zkConnectionManagerCallback-435-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 546327 WARN  (zkConnectionManagerCallback-461-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1a739bf2 name: ZooKeeperConnection Watcher:127.0.0.1:34330/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 546327 WARN  (zkConnectionManagerCallback-461-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 546327 WARN  (zkConnectionManagerCallback-461-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 546392 INFO  (zkConnectionManagerCallback-465-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper
   [junit4]   2> 546392 INFO  (zkConnectionManagerCallback-465-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ConnectionManager zkClient Connected:true
   [junit4]   2> 546393 INFO  (zkConnectionManagerCallback-457-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (0)
   [junit4]   2> 546433 INFO  (zkConnectionManagerCallback-457-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper
   [junit4]   2> 546433 INFO  (zkConnectionManagerCallback-457-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ConnectionManager zkClient Connected:true
   [junit4]   2> 546434 INFO  (zkConnectionManagerCallback-439-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (0)
   [junit4]   2> 546434 INFO  (zkConnectionManagerCallback-461-thread-1-processing-n:127.0.0.1:40580_solr-EventThread) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 546434 INFO  (zkConnectionManagerCallback-461-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 546453 INFO  (zkConnectionManagerCallback-435-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (0)
   [junit4]   2> 546490 WARN  (jetty-launcher-410-thread-2-SendThread(127.0.0.1:34330)) [n:127.0.0.1:33757_solr    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100cb3ef408000c has expired
   [junit4]   2> 546490 WARN  (zkConnectionManagerCallback-445-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2303acd1 name: ZooKeeperConnection Watcher:127.0.0.1:34330/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 546490 WARN  (zkConnectionManagerCallback-445-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 546490 INFO  (zkConnectionManagerCallback-445-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 546490 WARN  (zkConnectionManagerCallback-445-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 546491 INFO  (zkConnectionManagerCallback-461-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (0)
   [junit4]   2> 546527 INFO  (zkConnectionManagerCallback-461-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper
   [junit4]   2> 546527 INFO  (zkConnectionManagerCallback-461-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ConnectionManager zkClient Connected:true
   [junit4]   2> 546545 INFO  (zkConnectionManagerCallback-439-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40714_solr
   [junit4]   2> 546562 INFO  (zkConnectionManagerCallback-445-thread-1-processing-n:127.0.0.1:33757_solr-EventThread) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 546562 INFO  (zkConnectionManagerCallback-445-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 546562 INFO  (zkConnectionManagerCallback-445-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 546600 INFO  (zkConnectionManagerCallback-435-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40580_solr
   [junit4]   2> 546602 INFO  (zkCallback-460-thread-4-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 546618 INFO  (zkCallback-464-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 546618 INFO  (zkCallback-456-thread-2-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 546618 INFO  (zkConnectionManagerCallback-439-thread-1-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.Overseer Overseer (id=72281065280503827-127.0.0.1:40714_solr-n_0000000004) starting
   [junit4]   2> 546618 INFO  (zkCallback-438-thread-5-processing-n:127.0.0.1:40714_solr) [n:127.0.0.1:40714_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 546631 INFO  (zkConnectionManagerCallback-435-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper
   [junit4]   2> 546631 INFO  (zkConnectionManagerCallback-435-thread-1-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ConnectionManager zkClient Connected:true
   [junit4]   2> 546652 INFO  (zkCallback-434-thread-5-processing-n:127.0.0.1:40580_solr) [n:127.0.0.1:40580_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 546652 INFO  (zkConnectionManagerCallback-445-thread-1-processing-n:127.0.0.1:33757_solr) [n:127.0.0.1:33757_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (1)
   [junit4]   2> 546724 WARN  (jetty-launcher-410-thread-4-SendThread(127.0.0.1:34330)) [n:127.0.0.1:45917_solr    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100cb3e

[...truncated too long message...]

stComponentsName, OverseerCollectionConfigSetProcessorTest, RecoveryAfterSoftCommitTest, SpellCheckCollatorWithCollapseTest, V2ApiIntegrationTest, HdfsDirectoryTest, TestOverriddenPrefixQueryForCustomFieldType, TermVectorComponentDistributedTest, TestCloudPivotFacet, TestSchemalessBufferedUpdates, CoreAdminCreateDiscoverTest, TestManagedResource, TestRandomFaceting, TestStandardQParsers, DebugComponentTest, TestRemoteStreaming, OpenCloseCoreStressTest, TestSolrQueryParser, AtomicUpdateProcessorFactoryTest, TestHalfAndHalfDocValues, SolrPluginUtilsTest, TestFieldCache, NodeLostTriggerTest, TestComplexPhraseQParserPlugin, HdfsNNFailoverTest, SharedFSAutoReplicaFailoverTest, CleanupOldIndexTest, CurrencyRangeFacetCloudTest, CdcrVersionReplicationTest, TestLRUStatsCache, MergeStrategyTest, TestLMDirichletSimilarityFactory, SolrIndexSplitterTest, SuggestComponentContextFilterQueryTest, FieldAnalysisRequestHandlerTest, OverseerStatusTest, CollectionStateFormat2Test, TestIndexingPerformance, TestDistributedSearch, SpellingQueryConverterTest, TestAddFieldRealTimeGet, TestConfigSetsAPIExclusivity, ChaosMonkeyNothingIsSafeTest, BasicDistributedZkTest, SolrSlf4jReporterTest, TestMergePolicyConfig, SolrXmlInZkTest, CreateRoutedAliasTest, ConnectionReuseTest, MBeansHandlerTest, DateRangeFieldTest, RollingRestartTest, TriggerIntegrationTest, TestCollectionAPI, SolrRequestParserTest, PreAnalyzedUpdateProcessorTest, TestMultiValuedNumericRangeQuery, SynonymTokenizerTest, TestInitParams, TestFileDictionaryLookup, DistributedQueryElevationComponentTest, BaseCdcrDistributedZkTest, TestPerFieldSimilarityWithDefaultOverride, CdcrReplicationDistributedZkTest, TestLocalFSCloudBackupRestore, StatsComponentTest, TestShardHandlerFactory, TestSchemaResource, TestRuleBasedAuthorizationPlugin, TestSearcherReuse, TestManagedSchemaThreadSafety, NodeMutatorTest, TestUseDocValuesAsStored2, TestNestedDocsSort, TestNamedUpdateProcessors, DistributedExpandComponentTest, TestDocSet, LIRRollingUpdatesTest, TestSort, ClassificationUpdateProcessorIntegrationTest, TestSubQueryTransformer, SecurityConfHandlerTest, TestFieldCacheSortRandom, TestMaxTokenLenTokenizer, TestCloudDeleteByQuery, TestCloudSearcherWarming, TestDistribDocBasedVersion, TestMiniSolrCloudClusterSSL]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestMiniSolrCloudClusterSSL -Dtests.seed=BA4CB527817F8441 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=sr -Dtests.timezone=Europe/Rome -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J0 | TestMiniSolrCloudClusterSSL (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 8 threads leaked from SUITE scope at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL:
   [junit4]    >    1) Thread[id=20524, name=zkConnectionManagerCallback-3991-thread-1, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    2) Thread[id=20488, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    3) Thread[id=20502, name=zkConnectionManagerCallback-3980-thread-1, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    4) Thread[id=20489, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    5) Thread[id=20500, name=jetty-launcher-3953-thread-1-EventThread, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
   [junit4]    >    6) Thread[id=20519, name=Thread-4118, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
   [junit4]    >         at java.lang.Object.wait(Native Method)
   [junit4]    >         at java.lang.Object.wait(Object.java:502)
   [junit4]    >         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1731)
   [junit4]    >    7) Thread[id=20521, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    8) Thread[id=20499, name=jetty-launcher-3953-thread-1-SendThread(127.0.0.1:36352), state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)
   [junit4]    > at __randomizedtesting.SeedInfo.seed([BA4CB527817F8441]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=20524, name=zkConnectionManagerCallback-3991-thread-1, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    2) Thread[id=20502, name=zkConnectionManagerCallback-3980-thread-1, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]    >         at java.lang.Thread.run(Thread.java:748)
   [junit4]    >    3) Thread[id=20519, name=Thread-4118, state=WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
   [junit4]    >         at java.lang.Object.wait(Native Method)
   [junit4]    >         at java.lang.Object.wait(Object.java:502)
   [junit4]    >         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1731)
   [junit4]    >    4) Thread[id=20499, name=jetty-launcher-3953-thread-1-SendThread(127.0.0.1:36352), state=TIMED_WAITING, group=TGRP-TestMiniSolrCloudClusterSSL]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)
   [junit4]    > at __randomizedtesting.SeedInfo.seed([BA4CB527817F8441]:0)
   [junit4] Completed [575/785 (6!)] on J0 in 217.63s, 4 tests, 1 failure, 2 errors <<< FAILURES!

[...truncated 50193 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2426 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2426/

2 tests failed.
FAILED:  org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove

Error Message:
No live SolrServers available to handle this request:[https://127.0.0.1:39887/solr/MoveReplicaHDFSTest_failed_coll_true, https://127.0.0.1:57785/solr/MoveReplicaHDFSTest_failed_coll_true]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:39887/solr/MoveReplicaHDFSTest_failed_coll_true, https://127.0.0.1:57785/solr/MoveReplicaHDFSTest_failed_coll_true]
        at __randomizedtesting.SeedInfo.seed([20FCAB38253F4A3F:8A3178CA92EC9FEF]:0)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1105)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:885)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:992)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:992)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:818)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
        at org.apache.solr.cloud.MoveReplicaTest.testFailedMove(MoveReplicaTest.java:309)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:39887/solr/MoveReplicaHDFSTest_failed_coll_true: no servers hosting shard: shard1
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:436)
        ... 47 more


FAILED:  org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup

Error Message:
should be at least one inactive event

Stack Trace:
java.lang.AssertionError: should be at least one inactive event
        at __randomizedtesting.SeedInfo.seed([20FCAB38253F4A3F:3DD06B4A447C6D34]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup(ScheduledMaintenanceTriggerTest.java:218)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 1844 lines...]
   [junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/temp/junit4-J0-20180315_092814_6277564501970733679558.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] codec: CheapBastard, pf: LuceneVarGapFixedInterval, dvf: Asserting
   [junit4] <<< JVM J0: EOF ----

[...truncated 10263 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_20FCAB38253F4A3F-001/init-core-data-001
   [junit4]   2> 119218 WARN  (SUITE-ScheduledMaintenanceTriggerTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=6 numCloses=6
   [junit4]   2> 119218 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 119228 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 119228 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 119233 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_20FCAB38253F4A3F-001/tempDir-001
   [junit4]   2> 119233 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 119251 INFO  (Thread-1468) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 119251 INFO  (Thread-1468) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 119315 ERROR (Thread-1468) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 119349 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:36724
   [junit4]   2> 119389 INFO  (zkConnectionManagerCallback-158-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 119418 INFO  (jetty-launcher-155-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:27:37-04:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 119506 INFO  (jetty-launcher-155-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 119513 INFO  (jetty-launcher-155-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 119513 INFO  (jetty-launcher-155-thread-1) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 119517 INFO  (jetty-launcher-155-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3953a57{/solr,null,AVAILABLE}
   [junit4]   2> 119526 INFO  (jetty-launcher-155-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@6bb953dc{HTTP/1.1,[http/1.1]}{127.0.0.1:48017}
   [junit4]   2> 119530 INFO  (jetty-launcher-155-thread-1) [    ] o.e.j.s.Server Started @123150ms
   [junit4]   2> 119530 INFO  (jetty-launcher-155-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=48017}
   [junit4]   2> 119531 ERROR (jetty-launcher-155-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 119531 INFO  (jetty-launcher-155-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 119532 INFO  (jetty-launcher-155-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 119532 INFO  (jetty-launcher-155-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 119532 INFO  (jetty-launcher-155-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-15T10:04:47.972Z
   [junit4]   2> 119567 INFO  (zkConnectionManagerCallback-160-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 119567 INFO  (jetty-launcher-155-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 119652 INFO  (jetty-launcher-155-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36724/solr
   [junit4]   2> 119687 INFO  (zkConnectionManagerCallback-164-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 119737 INFO  (zkConnectionManagerCallback-166-thread-1-processing-n:127.0.0.1:48017_solr) [n:127.0.0.1:48017_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 120352 INFO  (jetty-launcher-155-thread-1) [n:127.0.0.1:48017_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 120358 INFO  (jetty-launcher-155-thread-1) [n:127.0.0.1:48017_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:48017_solr
   [junit4]   2> 120361 INFO  (jetty-launcher-155-thread-1) [n:127.0.0.1:48017_solr    ] o.a.s.c.Overseer Overseer (id=73457455469166595-127.0.0.1:48017_solr-n_0000000000) starting
   [junit4]   2> 120403 INFO  (zkConnectionManagerCallback-171-thread-1-processing-n:127.0.0.1:48017_solr) [n:127.0.0.1:48017_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 120420 INFO  (jetty-launcher-155-thread-1) [n:127.0.0.1:48017_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36724/solr ready
   [junit4]   2> 120461 INFO  (jetty-launcher-155-thread-1) [n:127.0.0.1:48017_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48017_solr
   [junit4]   2> 120462 DEBUG (OverseerAutoScalingTriggerThread-73457455469166595-127.0.0.1:48017_solr-n_0000000000) [n:127.0.0.1:48017_solr    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 120467 DEBUG (OverseerAutoScalingTriggerThread-73457455469166595-127.0.0.1:48017_solr-n_0000000000) [n:127.0.0.1:48017_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 120473 DEBUG (OverseerAutoScalingTriggerThread-73457455469166595-127.0.0.1:48017_solr-n_0000000000) [n:127.0.0.1:48017_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: []
   [junit4]   2> 120473 DEBUG (OverseerAutoScalingTriggerThread-73457455469166595-127.0.0.1:48017_solr-n_0000000000) [n:127.0.0.1:48017_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 120473 DEBUG (OverseerAutoScalingTriggerThread-73457455469166595-127.0.0.1:48017_solr-n_0000000000) [n:127.0.0.1:48017_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 120522 INFO  (zkCallback-170-thread-1-processing-n:127.0.0.1:48017_solr) [n:127.0.0.1:48017_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 120531 INFO  (zkCallback-165-thread-1-processing-n:127.0.0.1:48017_solr) [n:127.0.0.1:48017_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 120571 DEBUG (OverseerAutoScalingTriggerThread-73457455469166595-127.0.0.1:48017_solr-n_0000000000) [n:127.0.0.1:48017_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 120591 DEBUG (OverseerAutoScalingTriggerThread-73457455469166595-127.0.0.1:48017_solr-n_0000000000) [n:127.0.0.1:48017_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 120561 DEBUG (ScheduledTrigger-155-thread-1) [n:127.0.0.1:48017_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 121097 INFO  (jetty-launcher-155-thread-1) [n:127.0.0.1:48017_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48017.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@508e8f9e
   [junit4]   2> 121114 INFO  (jetty-launcher-155-thread-1) [n:127.0.0.1:48017_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48017.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@508e8f9e
   [junit4]   2> 121115 INFO  (jetty-launcher-155-thread-1) [n:127.0.0.1:48017_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48017.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@508e8f9e
   [junit4]   2> 121117 INFO  (jetty-launcher-155-thread-1) [n:127.0.0.1:48017_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_20FCAB38253F4A3F-001/tempDir-001/node1/.
   [junit4]   2> 121200 INFO  (zkConnectionManagerCallback-173-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 121234 INFO  (zkConnectionManagerCallback-177-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 121236 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 121238 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36724/solr ready
   [junit4]   2> 121345 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 121346 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 121346 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
   [junit4]   2> 121346 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 121346 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 121378 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 121379 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 121386 INFO  (TEST-ScheduledMaintenanceTriggerTest.testInactiveShardCleanup-seed#[20FCAB38253F4A3F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testInactiveShardCleanup
   [junit4]   2> 121396 DEBUG (ScheduledTrigger-162-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 121428 DEBUG (ScheduledTrigger-162-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 121461 DEBUG (ScheduledTrigger-162-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 121481 DEBUG (ScheduledTrigger-162-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 121501 DEBUG (ScheduledTrigger-162-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 121522 DEBUG (ScheduledTrigger-162-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 121640 DEBUG (ScheduledTrigger-162-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 121656 DEBUG (ScheduledTrigger-155-thread-3) [n:127.0.0.1:48017_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 121661 DEBUG (ScheduledTrigger-162-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 121681 DEBUG (ScheduledTrigger-162-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 121698 DEBUG (simCloudManagerPool-161-thread-4) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for ScheduledMaintenanceTriggerTest_collection1 / shard1: {"core_node1":{
   [junit4]   2>     "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>     "core":"ScheduledMaintenanceTriggerTest_collection1_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"ScheduledMaintenanceTriggerTest_collection1"}}
   [junit4]   2> 121707 DEBUG (ScheduledTrigger-162-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 121707 DEBUG (simCloudManagerPool-161-thread-5) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for ScheduledMaintenanceTriggerTest_collection1 / shard1_1: {"core_node5":{
   [junit4]   2>     "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>     "base_url":"http://127.0.0.1:10000/solr",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "core":"ScheduledMaintenanceTriggerTest_collection1_shard1_1_replica0",
   [junit4]   2>     "shard":"shard1_1",
   [junit4]   2>     "collection":"ScheduledMaintenanceTriggerTest_collection1"}}
   [junit4]   2> 121723 DEBUG (simCloudManagerPool-161-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for ScheduledMaintenanceTriggerTest_collection1 / shard1_0: {"core_node4":{
   [junit4]   2>     "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>     "base_url":"http://127.0.0.1:10000/solr",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInBytes":1,
   [junit4]   2>     "core":"ScheduledMaintenanceTriggerTest_collection1_shard1_0_replica0",
   [junit4]   2>     "shard":"shard1_0",
   [junit4]   2>     "collection":"ScheduledMaintenanceTriggerTest_collection1"}}
   [junit4]   2> 121728 DEBUG (ScheduledTrigger-162-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 121748 DEBUG (ScheduledTrigger-162-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 121768 DEBUG (ScheduledTrigger-162-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 121771 DEBUG (simCloudManagerPool-161-thread-7) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 121771 DEBUG (simCloudManagerPool-161-thread-7) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 121772 DEBUG (simCloudManagerPool-161-thread-7) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
   [junit4]   2> 121772 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 121773 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 121773 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 121788 DEBUG (ScheduledTrigger-162-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 121809 DEBUG (ScheduledTrigger-162-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 121811 DEBUG (simCloudManagerPool-161-thread-8) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 121812 DEBUG (simCloudManagerPool-161-thread-8) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 121812 DEBUG (simCloudManagerPool-161-thread-8) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
   [junit4]   2> 121813 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 121813 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 121813 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 121813 DEBUG (ScheduledTrigger-162-thread-4) [    ] o.a.s.c.a.ScheduledTrigger ScheduledTrigger .scheduled_maintenance firing registered processor for scheduled time 2018-03-15T10:04:56.735Z, now=2018-03-15T10:05:18.109Z
   [junit4]   2> 121816 DEBUG (ScheduledTrigger-162-thread-4) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"4be301f168d53cT3decn53euwaxscd3tf1chimuh",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":21360220737951036,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{"actualEventTime":1521108318109}}
   [junit4]   2> 121817 DEBUG (ScheduledTrigger-162-thread-4) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, .scheduled_maintenance]
   [junit4]   2> 121834 INFO  (ScheduledTrigger-162-thread-4) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=21360221715195686, stage=STARTED, actionName='null', event={
   [junit4]   2>   "id":"4be301f168d53cT3decn53euwaxscd3tf1chimuh",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":21360220737951036,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1521108318109,
   [junit4]   2>     "_enqueue_time_":21360220906205686}}, context={}, config={
   [junit4]   2>   "trigger":".scheduled_maintenance",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":"inactive_shard_plan",
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":"inactive_shard_plan"}, message='null'}
   [junit4]   2> 121837 DEBUG (AutoscalingActionExecutor-163-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"4be301f168d53cT3decn53euwaxscd3tf1chimuh",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":21360220737951036,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1521108318109,
   [junit4]   2>     "_enqueue_time_":21360220906205686}}
   [junit4]   2> 121857 INFO  (AutoscalingActionExecutor-163-thread-1) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=21360222860667286, stage=BEFORE_ACTION, actionName='inactive_shard_plan', event={
   [junit4]   2>   "id":"4be301f168d53cT3decn53euwaxscd3tf1chimuh",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":21360220737951036,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1521108318109,
   [junit4]   2>     "_enqueue_time_":21360220906205686}}, context={properties.BEFORE_ACTION=[inactive_shard_plan], source=.scheduled_maintenance}, config={
   [junit4]   2>   "trigger":".scheduled_maintenance",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":"inactive_shard_plan",
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":"inactive_shard_plan"}, message='null'}
   [junit4]   2> 121911 DEBUG (AutoscalingActionExecutor-163-thread-1) [    ] o.a.s.c.a.InactiveShardPlanAction ScheduledMaintenanceTriggerTest_collection1/shard1: tstamp=1521108310748034500, time=1521108322971937900, delta=12
   [junit4]   2> 121911 DEBUG (AutoscalingActionExecutor-163-thread-1) [    ] o.a.s.c.a.InactiveShardPlanAction -- delete inactive ScheduledMaintenanceTriggerTest_collection1 / shard1
   [junit4]   2> 121924 INFO  (AutoscalingActionExecutor-163-thread-1) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=21360226200717136, stage=AFTER_ACTION, actionName='inactive_shard_plan', event={
   [junit4]   2>   "id":"4be301f168d53cT3decn53euwaxscd3tf1chimuh",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":21360220737951036,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1521108318109,
   [junit4]   2>     "_enqueue_time_":21360220906205686}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$DeleteShard, method=GET, params.action=DELETESHARD, params.collection=ScheduledMaintenanceTriggerTest_collection1, params.shard=shard1}], properties.BEFORE_ACTION=[inactive_shard_plan], source=.scheduled_maintenance, properties.inactive_shard_plan={inactive={ScheduledMaintenanceTriggerTest_collection1=[shard1]}, cleanup={ScheduledMaintenanceTriggerTest_collection1=[shard1]}}, properties.AFTER_ACTION=[inactive_shard_plan]}, config={
   [junit4]   2>   "trigger":".scheduled_maintenance",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":"inactive_shard_plan",
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":"inactive_shard_plan"}, message='null'}
   [junit4]   2> 121924 DEBUG (AutoscalingActionExecutor-163-thread-1) [    ] o.a.s.c.a.ExecutePlanAction -- processing event: {
   [junit4]   2>   "id":"4be301f168d53cT3decn53euwaxscd3tf1chimuh",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":21360220737951036,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1521108318109,
   [junit4]   2>     "_enqueue_time_":21360220906205686}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$DeleteShard@27467e33], inactive_shard_plan={inactive={ScheduledMaintenanceTriggerTest_collection1=[shard1]}, cleanup={ScheduledMaintenanceTriggerTest_collection1=[shard1]}}, AFTER_ACTION=[inactive_shard_plan], BEFORE_ACTION=[inactive_shard_plan, execute_plan]}
   [junit4]   2> 121924 DEBUG (AutoscalingActionExecutor-163-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=DELETESHARD&collection=ScheduledMaintenanceTriggerTest_collection1&shard=shard1
   [junit4]   2> 122055 INFO  (AutoscalingActionExecutor-163-thread-1) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=21360232780840236, stage=SUCCEEDED, actionName='null', event={
   [junit4]   2>   "id":"4be301f168d53cT3decn53euwaxscd3tf1chimuh",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":21360220737951036,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1521108318109,
   [junit4]   2>     "_enqueue_time_":21360220906205686}}, context={}, config={
   [junit4]   2>   "trigger":".scheduled_maintenance",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":"inactive_shard_plan",
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":"inactive_shard_plan"}, message='null'}
   [junit4]   2> 122056 DEBUG (simCloudManagerPool-161-thread-13) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 122056 DEBUG (simCloudManagerPool-161-thread-13) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 122057 DEBUG (AutoscalingActionExecutor-163-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 122058 DEBUG (AutoscalingActionExecutor-163-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .scheduled_maintenance after 100ms
   [junit4]   2> 122058 DEBUG (AutoscalingActionExecutor-163-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 11053 ms for event id=4be301f168d53cT3decn53euwaxscd3tf1chimuh
   [junit4]   2> 122060 DEBUG (simCloudManagerPool-161-thread-13) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
   [junit4]   2> 122060 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 122065 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 122066 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 122078 DEBUG (simCloudManagerPool-161-thread-15) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 122079 DEBUG (simCloudManagerPool-161-thread-15) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 122081 DEBUG (simCloudManagerPool-161-thread-15) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
   [junit4]   2> 122082 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 122082 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 122082 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 122082 INFO  (TEST-ScheduledMaintenanceTriggerTest.testInactiveShardCleanup-seed#[20FCAB38253F4A3F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testInactiveShardCleanup
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ScheduledMaintenanceTriggerTest -Dtests.method=testInactiveShardCleanup -Dtests.seed=20FCAB38253F4A3F -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=de-CH -Dtests.timezone=Europe/Zurich -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 0.75s J1 | ScheduledMaintenanceTriggerTest.testInactiveShardCleanup <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: should be at least one inactive event
   [junit4]    > at __randomizedtesting.SeedInfo.seed([20FCAB38253F4A3F:3DD06B4A447C6D34]:0)
   [junit4]    > at org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup(ScheduledMaintenanceTriggerTest.java:218)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 122118 INFO  (TEST-ScheduledMaintenanceTriggerTest.testTriggerDefaults-seed#[20FCAB38253F4A3F]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerDefaults
   [junit4]   2> 122118 INFO  (TEST-ScheduledMaintenanceTriggerTest.testTriggerDefaults-seed#[20FCAB38253F4A3F]) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest {
   [junit4]   2>   "cluster-preferences":[
   [junit4]   2>     {
   [junit4]   2>       "minimize":"cores",
   [junit4]   2>       "precision":1},
   [junit4]   2>     {"maximize":"freedisk"}],
   [junit4]   2>   "triggers":{
   [junit4]   2>     ".scheduled_maintenance":{
   [junit4]   2>       "startTime":"NOW",
   [junit4]   2>       "event":"scheduled",
   [junit4]   2>       "every":"+1DAY",
   [junit4]   2>       "actions":[
   [junit4]   2>         {
   [junit4]   2>           "name":"inactive_shard_plan",
   [junit4]   2>           "class":"solr.InactiveShardPlanAction"},
   [junit4]   2>         {
   [junit4]   2>           "name":"execute_plan",
   [junit4]   2>           "class":"solr.ExecutePlanAction"}],
   [junit4]   2>       "enabled":true},
   [junit4]   2>     ".auto_add_replicas":{
   [junit4]   2>       "name":".auto_add_replicas",
   [junit4]   2>       "event":"nodeLost",
   [junit4]   2>       "waitFor":120,
   [junit4]   2>       "actions":[
   [junit4]   2>         {
   [junit4]   2>           "name":"auto_add_replicas_plan",
   [junit4]   2>           "class":"solr.AutoAddReplicasPlanAction"},
   [junit4]   2>         {
   [junit4]   2>           "name":"execute_plan",
   [junit4]   2>           "class":"solr.ExecutePlanAction"}],
   [junit4]   2>       "enabled":true}},
   [junit4]   2>   "listeners":{
   [junit4]   2>     ".scheduled_maintenance.system":{
   [junit4]   2>       "trigger":".scheduled_maintenance",
   [junit4]   2>       "afterAction":[],
   [junit4]   2>       "stage":[
   [junit4]   2>         "STARTED",
   [junit4]   2>         "ABORTED",
   [junit4]   2>         "SUCCEEDED",
   [junit4]   2>         "FAILED",
   [junit4]   2>         "BEFORE_ACTION",
   [junit4]   2>         "AFTER_ACTION",
   [junit4]   2>         "IGNORED"],
   [junit4]   2>       "class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
   [junit4]   2>       "beforeAction":[]},
   [junit4]   2>     ".auto_add_replicas.system":{
   [junit4]   2>       "trigger":".auto_add_replicas",
   [junit4]   2>       "afterAction":[],
   [junit4]   2>       "stage":[
   [junit4]   2>         "STARTED",
   [junit4]   2>         "ABORTED",
   [junit4]   2>         "SUCCEEDED",
   [junit4]   2>         "FAILED",
   [junit4]   2>         "BEFORE_ACTION",
   [junit4]   2>         "AFTER_ACTION",
   [junit4]   2>         "IGNORED"],
   [junit4]   2>       "class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
   [junit4]   2>       "beforeAction":[]}},
   [junit4]   2>   "properties":{}}
   [junit4]   2> 122134 INFO  (TEST-ScheduledMaintenanceTriggerTest.testTriggerDefaults-seed#[20FCAB38253F4A3F]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testTriggerDefaults
   [junit4]   2> 122135 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 122135 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 122139 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x104f92aa3580006, likely client has closed socket
   [junit4]   2> 122179 INFO  (jetty-closer-156-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@6bb953dc{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 122179 INFO  (jetty-closer-156-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=995214395
   [junit4]   2> 122180 INFO  (jetty-closer-156-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 122180 INFO  (jetty-closer-156-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@187df27e: rootName = solr_48017, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@54b23271
   [junit4]   2> 122207 INFO  (jetty-closer-156-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 122207 INFO  (jetty-closer-156-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3858db6d: rootName = solr_48017, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@498820e2
   [junit4]   2> 122228 INFO  (jetty-closer-156-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 122228 INFO  (jetty-closer-156-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@28af1b0b: rootName = solr_48017, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@45fc7c1e
   [junit4]   2> 122233 INFO  (jetty-closer-156-thread-1) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:48017_solr
   [junit4]   2> 122235 INFO  (zkCallback-170-thread-1-processing-n:127.0.0.1:48017_solr) [n:127.0.0.1:48017_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 122236 INFO  (zkCallback-165-thread-1-processing-n:127.0.0.1:48017_solr) [n:127.0.0.1:48017_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 122236 INFO  (jetty-closer-156-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 122237 INFO  (jetty-closer-156-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 122237 INFO  (jetty-closer-156-thread-1) [    ] o.a.s.c.Overseer Overseer (id=73457455469166595-127.0.0.1:48017_solr-n_0000000000) closing
   [junit4]   2> 122237 INFO  (OverseerStateUpdate-73457455469166595-127.0.0.1:48017_solr-n_0000000000) [n:127.0.0.1:48017_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:48017_solr
   [junit4]   2> 122238 WARN  (OverseerAutoScalingTriggerThread-73457455469166595-127.0.0.1:48017_solr-n_0000000000) [n:127.0.0.1:48017_solr    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 122241 DEBUG (jetty-closer-156-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 122247 INFO  (jetty-closer-156-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3953a57{/solr,null,UNAVAILABLE}
   [junit4]   2> 122248 INFO  (jetty-closer-156-thread-1) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 122249 ERROR (SUITE-ScheduledMaintenanceTriggerTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 122250 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36724 36724
   [junit4]   2> 122493 INFO  (Thread-1468) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36724 36724
   [junit4]   2> 122494 WARN  (Thread-1468) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/aliases.json
   [junit4]   2> 3 /solr/clusterprops.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/clusterstate.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/live_nodes
   [junit4]   2> 3 /solr/collections
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_20FCAB38253F4A3F-001
   [junit4]   2> Mar 15, 2018 10:04:50 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@76d7f554), locale=de-CH, timezone=Europe/Zurich
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=155236024,total=394788864
   [junit4]   2> NOTE: All tests run in this JVM: [HttpPartitionTest, CheckHdfsIndexTest, XsltUpdateRequestHandlerTest, ScheduledMaintenanceTriggerTest]
   [junit4] Completed [18/785 (1!)] on J1 in 3.87s, 2 tests, 1 failure <<< FAILURES!

[...truncated 1825 lines...]
   [junit4] Suite: org.apache.solr.cloud.MoveReplicaHDFSTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.MoveReplicaHDFSTest_20FCAB38253F4A3F-001/init-core-data-001
   [junit4]   2> 1606720 WARN  (SUITE-MoveReplicaHDFSTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=12 numCloses=12
   [junit4]   2> 1606720 INFO  (SUITE-MoveReplicaHDFSTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1606722 INFO  (SUITE-MoveReplicaHDFSTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 1606722 INFO  (SUITE-MoveReplicaHDFSTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1606723 INFO  (SUITE-MoveReplicaHDFSTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.MoveReplicaHDFSTest_20FCAB38253F4A3F-001/tempDir-001
   [junit4]   2> 1606723 INFO  (SUITE-MoveReplicaHDFSTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1606723 INFO  (Thread-5594) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1606724 INFO  (Thread-5594) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1606731 ERROR (Thread-5594) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1606824 INFO  (SUITE-MoveReplicaHDFSTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:41384
   [junit4]   2> 1606826 INFO  (zkConnectionManagerCallback-5659-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1606854 INFO  (jetty-launcher-5656-thread-4) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:27:37-04:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1606855 INFO  (jetty-launcher-5656-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1606855 INFO  (jetty-launcher-5656-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1606855 INFO  (jetty-launcher-5656-thread-4) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1606856 INFO  (jetty-launcher-5656-thread-2) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:27:37-04:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1606856 INFO  (jetty-launcher-5656-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@769ffe82{/solr,null,AVAILABLE}
   [junit4]   2> 1606857 INFO  (jetty-launcher-5656-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1606857 INFO  (jetty-launcher-5656-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1606857 INFO  (jetty-launcher-5656-thread-2) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1606858 INFO  (jetty-launcher-5656-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@39987162{/solr,null,AVAILABLE}
   [junit4]   2> 1606863 INFO  (jetty-launcher-5656-thread-3) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:27:37-04:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1606864 INFO  (jetty-launcher-5656-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:27:37-04:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1606864 INFO  (jetty-launcher-5656-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@87a3343{SSL,[ssl, http/1.1]}{127.0.0.1:54508}
   [junit4]   2> 1606864 INFO  (jetty-launcher-5656-thread-4) [    ] o.e.j.s.Server Started @1610487ms
   [junit4]   2> 1606864 INFO  (jetty-launcher-5656-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=54508}
   [junit4]   2> 1606864 ERROR (jetty-launcher-5656-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1606864 INFO  (jetty-launcher-5656-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1606864 INFO  (jetty-launcher-5656-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1606864 INFO  (jetty-launcher-5656-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1606864 INFO  (jetty-launcher-5656-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-15T10:29:35.304Z
   [junit4]   2> 1606865 INFO  (jetty-launcher-5656-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@616415f1{SSL,[ssl, http/1.1]}{127.0.0.1:53270}
   [junit4]   2> 1606865 INFO  (jetty-launcher-5656-thread-2) [    ] o.e.j.s.Server Started @1610489ms
   [junit4]   2> 1606865 INFO  (jetty-launcher-5656-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=53270}
   [junit4]   2> 1606865 ERROR (jetty-launcher-5656-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1606865 INFO  (jetty-launcher-5656-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1606866 INFO  (jetty-launcher-5656-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1606866 INFO  (jetty-launcher-5656-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1606866 INFO  (jetty-launcher-5656-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-15T10:29:35.306Z
   [junit4]   2> 1606879 INFO  (jetty-launcher-5656-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1606879 INFO  (jetty-launcher-5656-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1606880 INFO  (jetty-launcher-5656-thread-1) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 1606880 INFO  (jetty-launcher-5656-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@20afd3b{/solr,null,AVAILABLE}
   [junit4]   2> 1606881 INFO  (jetty-launcher-5656-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@51e94094{SSL,[ssl, http/1.1]}{127.0.0.1:48453}
   [junit4]   2> 1606881 INFO  (jetty-launcher-5656-thread-1) [    ] o.e.j.s.Server Started @1610504ms
   [junit4]   2> 1606881 INFO  (jetty-launcher-5656-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=48453}
   [junit4]   2> 1606881 ERROR (jetty-launcher-5656-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1606881 INFO  (jetty-launcher-5656-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1606881 INFO  (jetty-launcher-5656-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1606881 INFO  (jetty-launcher-5656-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1606881 INFO  (jetty-launcher-5656-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-15T10:29:35.321Z
   [junit4]   2> 1606887 INFO  (jetty-launcher-5656-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1606887 INFO  (jetty-launcher-5656-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1606888 INFO  (jetty-launcher-5656-thread-3) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 1606888 INFO  (zkConnectionManagerCallback-5663-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1606888 INFO  (zkConnectionManagerCallback-5665-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1606888 INFO  (jetty-launcher-5656-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1606889 INFO  (jetty-launcher-5656-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@449b7ada{/solr,null,AVAILABLE}
   [junit4]   2> 1606889 INFO  (jetty-launcher-5656-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4738a207{SSL,[ssl, http/1.1]}{127.0.0.1:57785}
   [junit4]   2> 1606889 INFO  (jetty-launcher-5656-thread-3) [    ] o.e.j.s.Server Started @1610513ms
   [junit4]   2> 1606889 INFO  (jetty-launcher-5656-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57785}
   [junit4]   2> 1606889 ERROR (jetty-launcher-5656-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1606890 INFO  (jetty-launcher-5656-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1606890 INFO  (jetty-launcher-5656-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1606890 INFO  (jetty-launcher-5656-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1606890 INFO  (jetty-launcher-5656-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-15T10:29:35.330Z
   [junit4]   2> 1606890 INFO  (jetty-launcher-5656-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1606900 INFO  (zkConnectionManagerCallback-5667-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1606900 INFO  (jetty-launcher-5656-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1606900 INFO  (zkConnectionManagerCallback-5661-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1606902 INFO  (jetty-launcher-5656-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1606906 INFO  (jetty-launcher-5656-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41384/solr
   [junit4]   2> 1606909 INFO  (jetty-launcher-5656-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41384/solr
   [junit4]   2> 1606915 INFO  (jetty-launcher-5656-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41384/solr
   [junit4]   2> 1606918 INFO  (jetty-launcher-5656-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41384/solr
   [junit4]   2> 1606923 INFO  (zkConnectionManagerCallback-5672-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1606923 INFO  (zkConnectionManagerCallback-5675-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1606923 INFO  (zkConnectionManagerCallback-5679-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1606923 INFO  (zkConnectionManagerCallback-5683-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1606924 DEBUG (jetty-launcher-5656-thread-2) [n:127.0.0.1:53270_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$47/1449554970@63bd850e
   [junit4]   2> 1606924 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x104f94155c20005, likely client has closed socket
   [junit4]   2> 1606925 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x104f94155c20006, likely client has closed socket
   [junit4]   2> 1606925 DEBUG (jetty-launcher-5656-thread-1) [n:127.0.0.1:48453_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$47/1449554970@501fdc60
   [junit4]   2> 1606926 DEBUG (jetty-launcher-5656-thread-3) [n:127.0.0.1:57785_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$47/1449554970@2615e2bf
   [junit4]   2> 1606926 DEBUG (jetty-launcher-5656-thread-4) [n:127.0.0.1:54508_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$47/1449554970@3cffaf98
   [junit4]   2> 1606927 INFO  (zkConnectionManagerCallback-5687-thread-1-processing-n:127.0.0.1:48453_solr) [n:127.0.0.1:48453_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1606928 INFO  (zkConnectionManagerCallback-5685-thread-1-processing-n:127.0.0.1:53270_solr) [n:127.0.0.1:53270_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1606928 INFO  (zkConnectionManagerCallback-5691-thread-1-processing-n:127.0.0.1:54508_solr) [n:127.0.0.1:54508_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1606928 INFO  (zkConnectionManagerCallback-5689-thread-1-processing-n:127.0.0.1:57785_solr) [n:127.0.0.1:57785_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1606958 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1606959 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1606959 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1606960 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1606960 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1606961 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 1606967 INFO  (jetty-launcher-5656-thread-1) [n:127.0.0.1:48453_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1606968 DEBUG (jetty-launcher-5656-thread-1) [n:127.0.0.1:48453_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/73457552951738377-127.0.0.1:48453_solr-n_0000000000
   [junit4]   2> 1606969 INFO  (jetty-launcher-5656-thread-1) [n:127.0.0.1:48453_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:48453_solr
   [junit4]   2> 1606971 INFO  (jetty-launcher-5656-thread-1) [n:127.0.0.1:48453_solr    ] o.a.s.c.Overseer Overseer (id=73457552951738377-127.0.0.1:48453_solr-n_0000000000) starting
   [junit4]   2> 1606990 INFO  (zkConnectionManagerCallback-5699-thread-1-processing-n:127.0.0.1:48453_solr) [n:127.0.0.1:48453_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1606991 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1606991 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1606991 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1606991 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1606992 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1606993 INFO  (jetty-launcher-5656-thread-1) [n:127.0.0.1:48453_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41384/solr ready
   [junit4]   2> 1606995 DEBUG (OverseerStateUpdate-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1606997 DEBUG (OverseerCollectionConfigSetProcessor-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1606998 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue-work
   [junit4]   2> 1606999 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 1607002 DEBUG (OverseerCollectionConfigSetProcessor-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
   [junit4]   2> 1607003 DEBUG (OverseerCollectionConfigSetProcessor-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
   [junit4]   2> 1607003 DEBUG (OverseerCollectionConfigSetProcessor-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
   [junit4]   2> 1607003 DEBUG (OverseerCollectionConfigSetProcessor-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
   [junit4]   2> 1607003 DEBUG (OverseerCollectionConfigSetProcessor-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
   [junit4]   2> 1607002 INFO  (jetty-launcher-5656-thread-1) [n:127.0.0.1:48453_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48453_solr
   [junit4]   2> 1607003 DEBUG (OverseerCollectionConfigSetProcessor-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
   [junit4]   2> 1607004 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
   [junit4]   2> 1607005 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1607005 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1607009 DEBUG (OverseerAutoScalingTriggerThread-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1607009 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1607009 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1607009 INFO  (zkCallback-5698-thread-1-processing-n:127.0.0.1:48453_solr) [n:127.0.0.1:48453_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1607009 INFO  (zkCallback-5686-thread-1-processing-n:127.0.0.1:48453_solr) [n:127.0.0.1:48453_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1607011 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/autoscaling.json
   [junit4]   2> 1607012 DEBUG (OverseerAutoScalingTriggerThread-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 1607017 DEBUG (OverseerAutoScalingTriggerThread-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:48453_solr]
   [junit4]   2> 1607017 DEBUG (OverseerAutoScalingTriggerThread-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 1607017 DEBUG (OverseerAutoScalingTriggerThread-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 1607039 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 1607047 DEBUG (OverseerAutoScalingTriggerThread-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1607048 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 1607048 DEBUG (OverseerAutoScalingTriggerThread-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 1607049 DEBUG (ScheduledTrigger-5617-thread-2) [n:127.0.0.1:48453_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1607106 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1607106 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1607106 INFO  (jetty-launcher-5656-thread-2) [n:127.0.0.1:53270_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1607107 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1607107 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1607108 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1607108 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 1607110 INFO  (jetty-launcher-5656-thread-2) [n:127.0.0.1:53270_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1607111 DEBUG (jetty-launcher-5656-thread-2) [n:127.0.0.1:53270_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/73457552951738378-127.0.0.1:53270_solr-n_0000000001
   [junit4]   2> 1607111 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/73457552951738377-127.0.0.1:48453_solr-n_0000000000
   [junit4]   2> 1607111 DEBUG (jetty-launcher-5656-thread-2) [n:127.0.0.1:53270_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/73457552951738377-127.0.0.1:48453_solr-n_0000000000 to know if I could be the leader
   [junit4]   2> 1607112 DEBUG (jetty-launcher-5656-thread-2) [n:127.0.0.1:53270_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:53270_solr as DOWN
   [junit4]   2> 1607115 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 1607115 INFO  (jetty-launcher-5656-thread-2) [n:127.0.0.1:53270_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53270_solr
   [junit4]   2> 1607116 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1607116 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1607116 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1607123 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 1607124 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1607124 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1607124 INFO  (zkCallback-5686-thread-1-processing-n:127.0.0.1:48453_solr) [n:127.0.0.1:48453_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1607124 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1607124 INFO  (zkCallback-5684-thread-1-processing-n:127.0.0.1:53270_solr) [n:127.0.0.1:53270_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1607126 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1607126 DEBUG (OverseerStateUpdate-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:53270_solr"} current state version: 0
   [junit4]   2> 1607126 DEBUG (OverseerStateUpdate-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:53270_solr
   [junit4]   2> 1607126 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1607126 INFO  (jetty-launcher-5656-thread-4) [n:127.0.0.1:54508_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1607127 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1607127 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1607128 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1607128 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1607129 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1607129 INFO  (jetty-launcher-5656-thread-3) [n:127.0.0.1:57785_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1607129 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 1607130 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1607130 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1607131 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1607132 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
   [junit4]   2> 1607132 INFO  (jetty-launcher-5656-thread-4) [n:127.0.0.1:54508_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1607136 INFO  (zkCallback-5698-thread-1-processing-n:127.0.0.1:48453_solr) [n:127.0.0.1:48453_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1607137 DEBUG (jetty-launcher-5656-thread-4) [n:127.0.0.1:54508_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/73457552951738380-127.0.0.1:54508_solr-n_0000000002
   [junit4]   2> 1607138 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/73457552951738378-127.0.0.1:53270_solr-n_0000000001
   [junit4]   2> 1607138 DEBUG (jetty-launcher-5656-thread-4) [n:127.0.0.1:54508_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/73457552951738378-127.0.0.1:53270_solr-n_0000000001 to know if I could be the leader
   [junit4]   2> 1607138 DEBUG (jetty-launcher-5656-thread-4) [n:127.0.0.1:54508_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:54508_solr as DOWN
   [junit4]   2> 1607139 INFO  (jetty-launcher-5656-thread-3) [n:127.0.0.1:57785_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1607139 DEBUG (jetty-launcher-5656-thread-3) [n:127.0.0.1:57785_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/73457552951738379-127.0.0.1:57785_solr-n_0000000003
   [junit4]   2> 1607140 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/73457552951738380-127.0.0.1:54508_solr-n_0000000002
   [junit4]   2> 1607140 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 1607140 DEBUG (jetty-launcher-5656-thread-3) [n:127.0.0.1:57785_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/73457552951738380-127.0.0.1:54508_solr-n_0000000002 to know if I could be the leader
   [junit4]   2> 1607140 INFO  (jetty-launcher-5656-thread-4) [n:127.0.0.1:54508_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54508_solr
   [junit4]   2> 1607141 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 1607141 DEBUG (jetty-launcher-5656-thread-3) [n:127.0.0.1:57785_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:57785_solr as DOWN
   [junit4]   2> 1607141 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1607142 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1607142 DEBUG (OverseerStateUpdate-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:54508_solr"} current state version: 0
   [junit4]   2> 1607142 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1607142 DEBUG (OverseerStateUpdate-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:54508_solr
   [junit4]   2> 1607142 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1607142 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1607143 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1607143 INFO  (zkCallback-5698-thread-1-processing-n:127.0.0.1:48453_solr) [n:127.0.0.1:48453_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1607147 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1607147 INFO  (zkCallback-5686-thread-1-processing-n:127.0.0.1:48453_solr) [n:127.0.0.1:48453_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1607147 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 1607147 INFO  (jetty-launcher-5656-thread-3) [n:127.0.0.1:57785_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57785_solr
   [junit4]   2> 1607148 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 1607148 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1607148 DEBUG (OverseerStateUpdate-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:57785_solr"} current state version: 0
   [junit4]   2> 1607148 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
   [junit4]   2> 1607148 DEBUG (OverseerStateUpdate-73457552951738377-127.0.0.1:48453_solr-n_0000000000) [n:127.0.0.1:48453_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:57785_solr
   [junit4]   2> 1607151 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1607151 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1607151 INFO  (zkCallback-5690-thread-1-processing-n:127.0.0.1:54508_solr) [n:127.0.0.1:54508_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
   [junit4]   2> 1607151 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1607151 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1607151 INFO  (zkCallback-5684-thread-1-processing-n:127.0.0.1:53270_solr) [n:127.0.0.1:53270_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
   [junit4]   2> 1607151 INFO  (zkCallback-5698-thread-1-processing-n:127.0.0.1:48453_solr) [n:127.0.0.1:48453_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1607152 INFO  (zkCallback-5686-thread-1-processing-n:127.0.0.1:48453_solr) [n:127.0.0.1:48453_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1607154 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1607155 INFO  (zkCallback-5688-thread-1-processing-n:127.0.0.1:57785_solr) [n:127.0.0.1:57785_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
   [junit4]   2> 1607159 INFO  (jetty-launcher-5656-thread-1) [n:127.0.0.1:48453_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48453.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@508e8f9e
   [junit4]   2> 1607172 INFO  (jetty-launcher-5656-thread-1) [n:127.0.0.1:48453_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48453.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@508e8f9e
   [junit4]   2> 1607172 INFO  (jetty-launcher-5656-thread-1) [n:127.0.0.1:48453_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48453.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@508e8f9e
   [junit4]   2> 1607174 INFO  (jetty-launcher-5656-thread-1) [n:127.0.0.1:48453_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.MoveReplicaHDFSTest_20FCAB38253F4A3F-001/tempDir-001/node1/.
   [junit4]   2> 1607249 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
   [junit4]   2> 1607250 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
   [junit4]   2> 1607255 INFO  (jetty-launcher-5656-thread-2) [n:127.0.0.1:53270_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53270.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@508e8f9e
   [junit4]   2> 1607272 INFO  (jetty-launcher-5656-thread-2) [n:127.0.0.1:53270_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53270.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@508e8f9e
   [junit4]   2> 1607272 INFO  (jetty-launcher-5656-thread-2) [n:127.0.0.1:53270_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53270.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@508e8f9e
   [junit4]   2> 1607275 INFO  (jetty-launcher-5656-thread-2) [n:127.0.0.1:53270_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.MoveReplicaHDFSTest_20FCAB38253F4A3F-001/tempDir-001/node2/.
   [junit4]   2> 1607284 INFO  (jetty-launcher-5656-thread-3) [n:127.0.0.1:57785_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57785.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@508e8f9e
   [junit4]   2> 1607284 INFO  (jetty-launcher-5656-thread-4) [n:127.0.0.1:54508_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54508.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@508e8f9e
   [junit4]   2> 1607284 INFO  (zkConnectionManagerCallback-5703-thread-1-processing-n:127.0.0.1:53270_solr) [n:127.0.0.1:53270_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1607285 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1607285 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1607286 INFO  (jetty-launcher-5656-thread-2) [n:127.0.0.1:53270_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1607286 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1607286 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1607286 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1607287 INFO  (jetty-launcher-5656-thread-2) [n:127.0.0.1:53270_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41384/solr ready
   [junit4]   2> 1607299 INFO  (jetty-launcher-5656-thread-3) [n:127.0.0.1:57785_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57785.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@508e8f9e
   [junit4]   2> 1607299 INFO  (jetty-launcher-5656-thread-3) [n:127.0.0.1:57785_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57785.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@508e8f9e
   [junit4]   2> 1607300 INFO  (jetty-launcher-5656-thread-4) [n:127.0.0.1:54508_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54508.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@508e8f9e
   [junit4]   2> 1607300 INFO  (jetty-launcher-5656-thread-4) [n:127.0.0.1:54508_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54508.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@508e8f9e
   [junit4]   2> 1607301 INFO  (jetty-launcher-5656-thread-3) [n:127.0.0.1:57785_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.MoveReplicaHDFSTest_20FCAB38253F4A3F-001/tempDir-001/node3/.
   [junit4]   2> 1607302 INFO  (jetty-launcher-5656-thread-4) [n:127.0.0.1:54508_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.MoveReplicaHDFSTest_20FCAB38253F4A3F-001/tempDir-001/node4/.
   [junit4]   2> 1607306 INFO  (zkConnectionManagerCallback-5708-thread-1-processing-n:127.0.0.1:57785_solr) [n:127.0.0.1:57785_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1607306 INFO  (zkConnectionManagerCallback-5711-thread-1-processing-n:127.0.0.1:54508_solr) [n:127.0.0.1:54508_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1607307 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1607307 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
   [junit4]   2> 1607308 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1607308 INFO  (jetty-launcher-5656-thread-3) [n:127.0.0.1:57785_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1607308 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
   [junit4]   2> 1607308 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1607308 INFO  (jetty-launcher-5656-thread-4) [n:127.0.0.1:54508_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1607308 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1607309 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
   [junit4]   2> 1607309 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
   [junit4]   2> 1607309 DEBUG (SyncThread:0) [    ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
   [junit4]   2> 1607309 INFO  (jetty-launcher-5656-thread-3) [n:127.0.0.1:57785_solr    ] o.a.s.c.s.i.ZkClientClusterStateP

[...truncated too long message...]

d1
   [junit4]   2> 9 /solr/configs/conf1
   [junit4]   2> 7 /solr/collections/MoveReplicaHDFSTest_coll_true/terms/shard2
   [junit4]   2> 6 /solr/security.json
   [junit4]   2> 5 /solr/collections/MoveReplicaHDFSTest_coll_true/collectionprops.json
   [junit4]   2> 4 /solr/collections/MoveReplicaHDFSTest_failed_coll_true/collectionprops.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 64 /solr/collections/MoveReplicaHDFSTest_coll_true/state.json
   [junit4]   2> 37 /solr/collections/MoveReplicaHDFSTest_failed_coll_true/state.json
   [junit4]   2> 15 /solr/clusterprops.json
   [junit4]   2> 15 /solr/clusterstate.json
   [junit4]   2> 2 /solr/overseer_elect/election/73457552951738379-127.0.0.1:57785_solr-n_0000000003
   [junit4]   2> 2 /solr/collections/MoveReplicaHDFSTest_coll_true/leader_elect/shard1/election/73457552951738379-core_node5-n_0000000001
   [junit4]   2> 2 /solr/autoscaling.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 19 /solr/live_nodes
   [junit4]   2> 18 /solr/collections
   [junit4]   2> 2 /solr/overseer/queue
   [junit4]   2> 2 /solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 2 /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 2 /solr/overseer/queue-work
   [junit4]   2> 2 /solr/overseer/collection-queue-work
   [junit4]   2>
   [junit4]   2> 1659125 WARN  (SUITE-MoveReplicaHDFSTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
   [junit4]   2> 1659141 INFO  (SUITE-MoveReplicaHDFSTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 1659243 WARN  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.MoveReplicaHDFSTest_20FCAB38253F4A3F-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.MoveReplicaHDFSTest_20FCAB38253F4A3F-001/tempDir-002/hdfsBaseDir/data/data4/]]  heartbeating to localhost/127.0.0.1:43893) [    ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
   [junit4]   2> 1659243 WARN  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.MoveReplicaHDFSTest_20FCAB38253F4A3F-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.MoveReplicaHDFSTest_20FCAB38253F4A3F-001/tempDir-002/hdfsBaseDir/data/data4/]]  heartbeating to localhost/127.0.0.1:43893) [    ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-803209357-10.41.0.5-1521109775972 (Datanode Uuid 2394ae94-ab09-4994-8b34-62d6d486f6f7) service to localhost/127.0.0.1:43893
   [junit4]   2> 1659249 WARN  (SUITE-MoveReplicaHDFSTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
   [junit4]   2> 1659285 INFO  (SUITE-MoveReplicaHDFSTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 1659288 WARN  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.MoveReplicaHDFSTest_20FCAB38253F4A3F-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.MoveReplicaHDFSTest_20FCAB38253F4A3F-001/tempDir-002/hdfsBaseDir/data/data2/]]  heartbeating to localhost/127.0.0.1:43893) [    ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
   [junit4]   2> 1659288 WARN  (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.MoveReplicaHDFSTest_20FCAB38253F4A3F-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.MoveReplicaHDFSTest_20FCAB38253F4A3F-001/tempDir-002/hdfsBaseDir/data/data2/]]  heartbeating to localhost/127.0.0.1:43893) [    ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-803209357-10.41.0.5-1521109775972 (Datanode Uuid 60548c55-1ea3-47e9-97ee-784dfabf8ad2) service to localhost/127.0.0.1:43893
   [junit4]   2> 1659313 INFO  (SUITE-MoveReplicaHDFSTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
   [junit4]   2> 1659314 WARN  (1019232314@qtp-1779831164-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:48420) [    ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false. Rechecking.
   [junit4]   2> 1659314 WARN  (1019232314@qtp-1779831164-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:48420) [    ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false
   [junit4]   2> 1659436 INFO  (SUITE-MoveReplicaHDFSTest-seed#[20FCAB38253F4A3F]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41384 41384
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.MoveReplicaHDFSTest_20FCAB38253F4A3F-001
   [junit4]   2> Mar 15, 2018 10:30:28 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {id=BlockTreeOrds(blocksize=128)}, docValues:{_version_=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=904, maxMBSortInHeap=6.331402310818973, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@64d80b4f), locale=en-GB, timezone=GMT0
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=119452568,total=517472256
   [junit4]   2> NOTE: All tests run in this JVM: [HttpPartitionTest, CheckHdfsIndexTest, XsltUpdateRequestHandlerTest, ScheduledMaintenanceTriggerTest, ConcurrentDeleteAndCreateCollectionTest, HLLUtilTest, FastVectorHighlighterTest, TestStressRecovery, TestCloudInspectUtil, TestSolrCloudWithSecureImpersonation, TestClusterProperties, BadComponentTest, TestLegacyNumericRangeQueryBuilder, TestSkipOverseerOperations, TestAuthenticationFramework, TestCloudPseudoReturnFields, IgnoreCommitOptimizeUpdateProcessorFactoryTest, BasicAuthIntegrationTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestXmlQParserPlugin, TestPullReplicaErrorHandling, DistributedIntervalFacetingTest, TestSolrConfigHandlerConcurrent, TestSQLHandler, TestSlowCompositeReaderWrapper, AtomicUpdatesTest, TestAnalyzeInfixSuggestions, SuggestComponentTest, TestScoreJoinQPNoScore, TriLevelCompositeIdRoutingTest, TestNodeAddedTrigger, CdcrReplicationHandlerTest, TestReload, AutoAddReplicasPlanActionTest, ClassificationUpdateProcessorTest, ShardSplitTest, ResponseBuilderTest, JsonLoaderTest, PreAnalyzedFieldManagedSchemaCloudTest, PolyFieldTest, HdfsUnloadDistributedZkTest, PathHierarchyTokenizerFactoryTest, TestPartialUpdateDeduplication, PeerSyncTest, TestInPlaceUpdatesDistrib, MoveReplicaTest, TestTestInjection, TestFieldTypeResource, BlockJoinFacetDistribTest, TestGraphMLResponseWriter, TestXIncludeConfig, TestManagedSynonymGraphFilterFactory, TestSearchPerf, TestTrie, ChaosMonkeySafeLeaderTest, V2StandaloneTest, JSONWriterTest, DataDrivenBlockJoinTest, TestJsonFacets, TestSolrJ, TestReplicationHandlerBackup, TestSurroundQueryParser, TestStressVersions, SSLMigrationTest, TestRawResponseWriter, SliceStateTest, TestManagedSchema, CursorMarkTest, SuggesterTest, FileUtilsTest, TestPostingsSolrHighlighter, TestCustomStream, ReplicationFactorTest, OverseerTaskQueueTest, TestDocBasedVersionConstraints, FieldMutatingUpdateProcessorTest, TestFastLRUCache, BasicZkTest, TestFieldCacheVsDocValues, QueryParsingTest, TestRecoveryHdfs, BigEndianAscendingWordDeserializerTest, TestObjectReleaseTracker, TestExclusionRuleCollectionAccess, TestSegmentSorting, TestSolrCloudSnapshots, CollectionsAPISolrJTest, ConfigureRecoveryStrategyTest, DirectUpdateHandlerOptimizeTest, TestTolerantUpdateProcessorRandomCloud, SpatialRPTFieldTypeTest, TestSimDistributedQueue, TestManagedResourceStorage, FacetPivotSmallTest, TestCollectionAPIs, ExitableDirectoryReaderTest, TestLuceneIndexBackCompat, TestFieldCacheReopen, CollectionTooManyReplicasTest, TestSolrCloudWithHadoopAuthPlugin, DeleteInactiveReplicaTest, TestConfigSetProperties, TestSystemIdResolver, TestFuzzyAnalyzedSuggestions, TestOrdValues, TestPayloadCheckQParserPlugin, SimplePostToolTest, TestFieldCollectionResource, TestZkChroot, TestFreeTextSuggestions, RequiredFieldsTest, DistributedQueryComponentCustomSortTest, TestDFISimilarityFactory, SystemInfoHandlerTest, TestCollectionsAPIViaSolrCloudCluster, TestSolrIndexConfig, TestSha256AuthenticationProvider, TestSubQueryTransformerCrossCore, DocValuesMissingTest, SegmentsInfoRequestHandlerTest, SortByFunctionTest, TestSortableTextField, TestUniqueKeyFieldResource, TestHttpShardHandlerFactory, BlockDirectoryTest, CustomHighlightComponentTest, JavabinLoaderTest, ResourceLoaderTest, SyncSliceTest, OverseerTest, TestRandomFaceting, ShardRoutingCustomTest, TestDistributedSearch, TestGroupingSearch, TestSort, SoftAutoCommitTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, DistribCursorPagingTest, ForceLeaderTest, HttpPartitionOnCommitTest, LegacyCloudClusterPropTest, MissingSegmentRecoveryTest, MoveReplicaHDFSTest]
   [junit4] Completed [559/785 (2!)] on J1 in 62.21s, 4 tests, 1 error, 2 skipped <<< FAILURES!

[...truncated 50315 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]