[JENKINS-EA] Lucene-Solr-master-Windows (64bit/jdk-9-ea+181) - Build # 6819 - Unstable!

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[JENKINS-EA] Lucene-Solr-master-Windows (64bit/jdk-9-ea+181) - Build # 6819 - Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6819/
Java: 64bit/jdk-9-ea+181 -XX:+UseCompressedOops -XX:+UseG1GC --illegal-access=deny

4 tests failed.
FAILED:  org.apache.solr.cloud.HttpPartitionTest.test

Error Message:
Doc with id=1 not found in http://127.0.0.1:61659/collMinRf_1x3 due to: Path not found: /id; rsp={doc=null}

Stack Trace:
java.lang.AssertionError: Doc with id=1 not found in http://127.0.0.1:61659/collMinRf_1x3 due to: Path not found: /id; rsp={doc=null}
        at __randomizedtesting.SeedInfo.seed([E1BDA6C84BBA429A:69E99912E5462F62]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:603)
        at org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:558)
        at org.apache.solr.cloud.HttpPartitionTest.testMinRf(HttpPartitionTest.java:249)
        at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:127)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestTlogReplica.testOnlyLeaderIndexes

Error Message:


Stack Trace:
java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([E1BDA6C84BBA429A:FDBCDB453E1F3C09]:0)
        at org.junit.Assert.fail(Assert.java:92)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertFalse(Assert.java:68)
        at org.junit.Assert.assertFalse(Assert.java:79)
        at org.apache.solr.cloud.TestTlogReplica.testOnlyLeaderIndexes(TestTlogReplica.java:497)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestTlogReplica

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.TestTlogReplica:     1) Thread[id=26796, name=indexFetcher-10981-thread-1, state=TIMED_WAITING, group=TGRP-TestTlogReplica]         at java.base@9/java.lang.Object.wait(Native Method)         at app//org.apache.solr.core.CachingDirectoryFactory.close(CachingDirectoryFactory.java:171)         at app//org.apache.solr.core.SolrCore.close(SolrCore.java:1612)         at app//org.apache.solr.handler.IndexFetcher.openNewSearcherAndUpdateCommitPoint(IndexFetcher.java:888)         at app//org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:632)         at app//org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:332)         at app//org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419)         at app//org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1183)         at app//org.apache.solr.handler.ReplicationHandler$$Lambda$540/464565583.run(Unknown Source)         at java.base@9/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514)         at java.base@9/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)         at java.base@9/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:300)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)         at java.base@9/java.lang.Thread.run(Thread.java:844)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestTlogReplica:
   1) Thread[id=26796, name=indexFetcher-10981-thread-1, state=TIMED_WAITING, group=TGRP-TestTlogReplica]
        at java.base@9/java.lang.Object.wait(Native Method)
        at app//org.apache.solr.core.CachingDirectoryFactory.close(CachingDirectoryFactory.java:171)
        at app//org.apache.solr.core.SolrCore.close(SolrCore.java:1612)
        at app//org.apache.solr.handler.IndexFetcher.openNewSearcherAndUpdateCommitPoint(IndexFetcher.java:888)
        at app//org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:632)
        at app//org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:332)
        at app//org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419)
        at app//org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1183)
        at app//org.apache.solr.handler.ReplicationHandler$$Lambda$540/464565583.run(Unknown Source)
        at java.base@9/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514)
        at java.base@9/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
        at java.base@9/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:300)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.base@9/java.lang.Thread.run(Thread.java:844)
        at __randomizedtesting.SeedInfo.seed([E1BDA6C84BBA429A]:0)


FAILED:  org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testExecutorStream

Error Message:
Error from server at https://127.0.0.1:49420/solr/mainCorpus_shard2_replica_n3: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 </title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /solr/mainCorpus_shard2_replica_n3/update. Reason: <pre>    Can not find: /solr/mainCorpus_shard2_replica_n3/update</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/> </body> </html>

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at https://127.0.0.1:49420/solr/mainCorpus_shard2_replica_n3: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/mainCorpus_shard2_replica_n3/update. Reason:
<pre>    Can not find: /solr/mainCorpus_shard2_replica_n3/update</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
</body>
</html>

        at __randomizedtesting.SeedInfo.seed([375A90A69E300888:159A115DBD5A2298]:0)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:539)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:993)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
        at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
        at org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testExecutorStream(StreamExpressionTest.java:6816)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:49420/solr/mainCorpus_shard2_replica_n3: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/mainCorpus_shard2_replica_n3/update. Reason:
<pre>    Can not find: /solr/mainCorpus_shard2_replica_n3/update</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
</body>
</html>

        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:591)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:516)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        ... 1 more




Build Log:
[...truncated 11133 lines...]
   [junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
   [junit4]   2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\init-core-data-001
   [junit4]   2> 616071 INFO  (SUITE-HttpPartitionTest-seed#[E1BDA6C84BBA429A]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 616072 INFO  (SUITE-HttpPartitionTest-seed#[E1BDA6C84BBA429A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl="https://issues.apache.org/jira/browse/SOLR-5776")
   [junit4]   2> 616072 INFO  (SUITE-HttpPartitionTest-seed#[E1BDA6C84BBA429A]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 616072 INFO  (SUITE-HttpPartitionTest-seed#[E1BDA6C84BBA429A]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 616074 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 616074 INFO  (Thread-1108) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 616074 INFO  (Thread-1108) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 616078 ERROR (Thread-1108) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 616175 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.ZkTestServer start zk server on port:61596
   [junit4]   2> 616191 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 616194 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 616196 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 616198 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 616200 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 616202 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 616204 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 616206 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 616207 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 616209 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 616211 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 616213 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
   [junit4]   2> 616579 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 616580 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7861dc3a{/,null,AVAILABLE}
   [junit4]   2> 616583 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@ab92fe3{HTTP/1.1,[http/1.1]}{127.0.0.1:61608}
   [junit4]   2> 616583 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.e.j.s.Server Started @618085ms
   [junit4]   2> 616583 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\tempDir-001/control/data, hostPort=61607, coreRootDirectory=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\control-001\cores}
   [junit4]   2> 616583 ERROR (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 616583 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 616583 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 616583 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 616584 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-12T17:38:13.974252900Z
   [junit4]   2> 616587 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15dd7865d5d0002, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 616587 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 616587 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\control-001\solr.xml
   [junit4]   2> 616592 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 616595 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:61596/solr
   [junit4]   2> 616700 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 616700 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:61607_
   [junit4]   2> 616701 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.Overseer Overseer (id=98471738963984388-127.0.0.1:61607_-n_0000000000) starting
   [junit4]   2> 616706 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61607_
   [junit4]   2> 616708 INFO  (OverseerStateUpdate-98471738963984388-127.0.0.1:61607_-n_0000000000) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 616782 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1
   [junit4]   2> 616786 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1
   [junit4]   2> 616786 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1
   [junit4]   2> 616787 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\control-001\cores
   [junit4]   2> 616861 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 616864 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:61596/solr ready
   [junit4]   2> 616865 INFO  (SocketProxy-Acceptor-61607) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=61635,localport=61607], receiveBufferSize:65536
   [junit4]   2> 616865 INFO  (SocketProxy-Acceptor-61607) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=61608,localport=61636], receiveBufferSize=65536
   [junit4]   2> 616875 INFO  (qtp352509678-5805) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:61607_&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 616881 INFO  (OverseerThreadFactory-2491-thread-1) [    ] o.a.s.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 616990 INFO  (SocketProxy-Acceptor-61607) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=61638,localport=61607], receiveBufferSize:65536
   [junit4]   2> 616990 INFO  (SocketProxy-Acceptor-61607) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=61608,localport=61639], receiveBufferSize=65536
   [junit4]   2> 616990 INFO  (qtp352509678-5809) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 616991 INFO  (qtp352509678-5809) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 617097 INFO  (zkCallback-897-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 618008 INFO  (qtp352509678-5809) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 618033 INFO  (qtp352509678-5809) [    ] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 618121 INFO  (qtp352509678-5809) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 618147 INFO  (qtp352509678-5809) [    ] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 618147 INFO  (qtp352509678-5809) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1
   [junit4]   2> 618148 INFO  (qtp352509678-5809) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 618148 INFO  (qtp352509678-5809) [    ] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\control-001\cores\control_collection_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\control-001\cores\control_collection_shard1_replica_n1\data\]
   [junit4]   2> 618150 INFO  (qtp352509678-5809) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=20, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=9.953125, floorSegmentMB=1.5107421875, forceMergeDeletesPctAllowed=29.206913067574384, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.33991772440890844
   [junit4]   2> 618154 WARN  (qtp352509678-5809) [    ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 618238 INFO  (qtp352509678-5809) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 618238 INFO  (qtp352509678-5809) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 618240 INFO  (qtp352509678-5809) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 618240 INFO  (qtp352509678-5809) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 618241 INFO  (qtp352509678-5809) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=88.3837890625, floorSegmentMB=1.49609375, forceMergeDeletesPctAllowed=27.40377814024113, segmentsPerTier=32.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.33083607878004395
   [junit4]   2> 618242 INFO  (qtp352509678-5809) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@20b0ae27[control_collection_shard1_replica_n1] main]
   [junit4]   2> 618243 INFO  (qtp352509678-5809) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 618244 INFO  (qtp352509678-5809) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 618244 INFO  (qtp352509678-5809) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 618245 INFO  (searcherExecutor-2494-thread-1) [    ] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@20b0ae27[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 618245 INFO  (qtp352509678-5809) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575547825696014336
   [junit4]   2> 618252 INFO  (qtp352509678-5809) [    ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 618252 INFO  (qtp352509678-5809) [    ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 618252 INFO  (qtp352509678-5809) [    ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:61607/control_collection_shard1_replica_n1/
   [junit4]   2> 618252 INFO  (qtp352509678-5809) [    ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 618252 INFO  (qtp352509678-5809) [    ] o.a.s.c.SyncStrategy http://127.0.0.1:61607/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 618252 INFO  (qtp352509678-5809) [    ] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 618257 INFO  (qtp352509678-5809) [    ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:61607/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 618360 INFO  (zkCallback-897-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 618408 INFO  (qtp352509678-5809) [    ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 618409 INFO  (qtp352509678-5809) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1419
   [junit4]   2> 618415 INFO  (qtp352509678-5805) [    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 618514 INFO  (zkCallback-897-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 618884 INFO  (OverseerCollectionConfigSetProcessor-98471738963984388-127.0.0.1:61607_-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 619415 INFO  (qtp352509678-5805) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:61607_&wt=javabin&version=2} status=0 QTime=2539
   [junit4]   2> 619427 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 619430 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:61596/solr ready
   [junit4]   2> 619430 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 619431 INFO  (SocketProxy-Acceptor-61607) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=61657,localport=61607], receiveBufferSize:65536
   [junit4]   2> 619432 INFO  (SocketProxy-Acceptor-61607) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=61608,localport=61658], receiveBufferSize=65536
   [junit4]   2> 619433 INFO  (qtp352509678-5812) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 619437 INFO  (OverseerThreadFactory-2491-thread-2) [    ] o.a.s.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 619438 WARN  (OverseerThreadFactory-2491-thread-2) [    ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 619645 INFO  (qtp352509678-5812) [    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 619645 INFO  (qtp352509678-5812) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=213
   [junit4]   2> 620024 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\shard-1-001 of type TLOG
   [junit4]   2> 620025 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 620027 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@98d0e1{/,null,AVAILABLE}
   [junit4]   2> 620028 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@c28e61a{HTTP/1.1,[http/1.1]}{127.0.0.1:61660}
   [junit4]   2> 620028 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.e.j.s.Server Started @621530ms
   [junit4]   2> 620028 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\tempDir-001/jetty1, hostPort=61659, coreRootDirectory=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\shard-1-001\cores, replicaType=TLOG}
   [junit4]   2> 620029 ERROR (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 620029 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 620029 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 620029 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 620029 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-12T17:38:17.420312200Z
   [junit4]   2> 620041 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 620041 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\shard-1-001\solr.xml
   [junit4]   2> 620045 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 620048 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:61596/solr
   [junit4]   2> 620052 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15dd7865d5d0008, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 620058 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 620060 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 620062 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61659_
   [junit4]   2> 620064 INFO  (zkCallback-897-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 620064 INFO  (zkCallback-904-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 620064 INFO  (zkCallback-909-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 620145 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1
   [junit4]   2> 620149 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1
   [junit4]   2> 620150 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1
   [junit4]   2> 620151 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\shard-1-001\cores
   [junit4]   2> 620170 INFO  (SocketProxy-Acceptor-61659) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=61672,localport=61659], receiveBufferSize:65536
   [junit4]   2> 620170 INFO  (SocketProxy-Acceptor-61659) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=61660,localport=61673], receiveBufferSize=65536
   [junit4]   2> 620171 INFO  (qtp537886282-5853) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:61659_&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 620176 INFO  (OverseerCollectionConfigSetProcessor-98471738963984388-127.0.0.1:61607_-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 620177 INFO  (OverseerThreadFactory-2491-thread-3) [    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:61659_ for creating new replica
   [junit4]   2> 620179 INFO  (SocketProxy-Acceptor-61659) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=61674,localport=61659], receiveBufferSize:65536
   [junit4]   2> 620179 INFO  (SocketProxy-Acceptor-61659) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=61660,localport=61675], receiveBufferSize=65536
   [junit4]   2> 620180 INFO  (qtp537886282-5860) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 620181 INFO  (qtp537886282-5860) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 621197 INFO  (qtp537886282-5860) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 621221 INFO  (qtp537886282-5860) [    ] o.a.s.s.IndexSchema [collection1_shard2_replica_t41] Schema name=test
   [junit4]   2> 621322 INFO  (qtp537886282-5860) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 621340 INFO  (qtp537886282-5860) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t41' using configuration from collection collection1, trusted=true
   [junit4]   2> 621340 INFO  (qtp537886282-5860) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_t41' (registry 'solr.core.collection1.shard2.replica_t41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1
   [junit4]   2> 621340 INFO  (qtp537886282-5860) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 621340 INFO  (qtp537886282-5860) [    ] o.a.s.c.SolrCore [[collection1_shard2_replica_t41] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\shard-1-001\cores\collection1_shard2_replica_t41], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\shard-1-001\cores\collection1_shard2_replica_t41\data\]
   [junit4]   2> 621341 INFO  (qtp537886282-5860) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=20, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=9.953125, floorSegmentMB=1.5107421875, forceMergeDeletesPctAllowed=29.206913067574384, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.33991772440890844
   [junit4]   2> 621344 WARN  (qtp537886282-5860) [    ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 621400 INFO  (qtp537886282-5860) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 621400 INFO  (qtp537886282-5860) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 621409 INFO  (qtp537886282-5860) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 621409 INFO  (qtp537886282-5860) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 621410 INFO  (qtp537886282-5860) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=88.3837890625, floorSegmentMB=1.49609375, forceMergeDeletesPctAllowed=27.40377814024113, segmentsPerTier=32.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.33083607878004395
   [junit4]   2> 621410 INFO  (qtp537886282-5860) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@7c7ed0bb[collection1_shard2_replica_t41] main]
   [junit4]   2> 621411 INFO  (qtp537886282-5860) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 621411 INFO  (qtp537886282-5860) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 621411 INFO  (qtp537886282-5860) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 621412 INFO  (searcherExecutor-2505-thread-1) [    ] o.a.s.c.SolrCore [collection1_shard2_replica_t41] Registered new searcher Searcher@7c7ed0bb[collection1_shard2_replica_t41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 621412 INFO  (qtp537886282-5860) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575547829016854528
   [junit4]   2> 621419 INFO  (qtp537886282-5860) [    ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 621419 INFO  (qtp537886282-5860) [    ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 621419 INFO  (qtp537886282-5860) [    ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:61659/collection1_shard2_replica_t41/
   [junit4]   2> 621419 INFO  (qtp537886282-5860) [    ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 621419 INFO  (qtp537886282-5860) [    ] o.a.s.c.SyncStrategy http://127.0.0.1:61659/collection1_shard2_replica_t41/ has no replicas
   [junit4]   2> 621419 INFO  (qtp537886282-5860) [    ] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 621419 INFO  (qtp537886282-5860) [    ] o.a.s.c.ZkController collection1_shard2_replica_t41 stopping background replication from leader
   [junit4]   2> 621422 INFO  (qtp537886282-5860) [    ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:61659/collection1_shard2_replica_t41/ shard2
   [junit4]   2> 621575 INFO  (qtp537886282-5860) [    ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 621576 INFO  (qtp537886282-5860) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1396
   [junit4]   2> 621581 INFO  (qtp537886282-5853) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:61659_&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2} status=0 QTime=1410
   [junit4]   2> 621999 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\shard-2-001 of type TLOG
   [junit4]   2> 621999 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 622000 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@137fd0f6{/,null,AVAILABLE}
   [junit4]   2> 622001 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@56e9d0cc{HTTP/1.1,[http/1.1]}{127.0.0.1:61690}
   [junit4]   2> 622001 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.e.j.s.Server Started @623504ms
   [junit4]   2> 622001 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\tempDir-001/jetty2, hostPort=61689, coreRootDirectory=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\shard-2-001\cores, replicaType=TLOG}
   [junit4]   2> 622001 ERROR (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 622001 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 622001 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 622003 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 622003 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-12T17:38:19.394523900Z
   [junit4]   2> 622077 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 622077 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\shard-2-001\solr.xml
   [junit4]   2> 622081 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 622083 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:61596/solr
   [junit4]   2> 622110 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 622112 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 622114 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61689_
   [junit4]   2> 622117 INFO  (zkCallback-904-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 622118 INFO  (zkCallback-915-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 622118 INFO  (zkCallback-897-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 622118 INFO  (zkCallback-909-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 622205 INFO  (OverseerCollectionConfigSetProcessor-98471738963984388-127.0.0.1:61607_-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 622218 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1
   [junit4]   2> 622224 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1
   [junit4]   2> 622224 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1
   [junit4]   2> 622226 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\shard-2-001\cores
   [junit4]   2> 622276 INFO  (SocketProxy-Acceptor-61689) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=61703,localport=61689], receiveBufferSize:65536
   [junit4]   2> 622279 INFO  (SocketProxy-Acceptor-61689) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=61690,localport=61704], receiveBufferSize=65536
   [junit4]   2> 622280 INFO  (qtp1320871993-5891) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:61689_&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 622291 INFO  (OverseerThreadFactory-2491-thread-4) [    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:61689_ for creating new replica
   [junit4]   2> 622294 INFO  (SocketProxy-Acceptor-61689) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=61705,localport=61689], receiveBufferSize:65536
   [junit4]   2> 622295 INFO  (SocketProxy-Acceptor-61689) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=61690,localport=61706], receiveBufferSize=65536
   [junit4]   2> 622296 INFO  (qtp1320871993-5898) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 622297 INFO  (qtp1320871993-5898) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 623313 INFO  (qtp1320871993-5898) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 623324 INFO  (qtp1320871993-5898) [    ] o.a.s.s.IndexSchema [collection1_shard1_replica_t43] Schema name=test
   [junit4]   2> 623443 INFO  (qtp1320871993-5898) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 623466 INFO  (qtp1320871993-5898) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_t43' using configuration from collection collection1, trusted=true
   [junit4]   2> 623467 INFO  (qtp1320871993-5898) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_t43' (registry 'solr.core.collection1.shard1.replica_t43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1
   [junit4]   2> 623467 INFO  (qtp1320871993-5898) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 623467 INFO  (qtp1320871993-5898) [    ] o.a.s.c.SolrCore [[collection1_shard1_replica_t43] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\shard-2-001\cores\collection1_shard1_replica_t43], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\shard-2-001\cores\collection1_shard1_replica_t43\data\]
   [junit4]   2> 623470 INFO  (qtp1320871993-5898) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=20, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=9.953125, floorSegmentMB=1.5107421875, forceMergeDeletesPctAllowed=29.206913067574384, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.33991772440890844
   [junit4]   2> 623472 WARN  (qtp1320871993-5898) [    ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 623547 INFO  (qtp1320871993-5898) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 623547 INFO  (qtp1320871993-5898) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 623549 INFO  (qtp1320871993-5898) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 623549 INFO  (qtp1320871993-5898) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 623550 INFO  (qtp1320871993-5898) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=88.3837890625, floorSegmentMB=1.49609375, forceMergeDeletesPctAllowed=27.40377814024113, segmentsPerTier=32.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.33083607878004395
   [junit4]   2> 623550 INFO  (qtp1320871993-5898) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@793c2fe6[collection1_shard1_replica_t43] main]
   [junit4]   2> 623552 INFO  (qtp1320871993-5898) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 623552 INFO  (qtp1320871993-5898) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 623553 INFO  (qtp1320871993-5898) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 623554 INFO  (searcherExecutor-2516-thread-1) [    ] o.a.s.c.SolrCore [collection1_shard1_replica_t43] Registered new searcher Searcher@793c2fe6[collection1_shard1_replica_t43] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 623554 INFO  (qtp1320871993-5898) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575547831262904320
   [junit4]   2> 623559 INFO  (qtp1320871993-5898) [    ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 623559 INFO  (qtp1320871993-5898) [    ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 623559 INFO  (qtp1320871993-5898) [    ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:61689/collection1_shard1_replica_t43/
   [junit4]   2> 623559 INFO  (qtp1320871993-5898) [    ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 623559 INFO  (qtp1320871993-5898) [    ] o.a.s.c.SyncStrategy http://127.0.0.1:61689/collection1_shard1_replica_t43/ has no replicas
   [junit4]   2> 623559 INFO  (qtp1320871993-5898) [    ] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 623559 INFO  (qtp1320871993-5898) [    ] o.a.s.c.ZkController collection1_shard1_replica_t43 stopping background replication from leader
   [junit4]   2> 623564 INFO  (qtp1320871993-5898) [    ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:61689/collection1_shard1_replica_t43/ shard1
   [junit4]   2> 623716 INFO  (qtp1320871993-5898) [    ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 623718 INFO  (qtp1320871993-5898) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1421
   [junit4]   2> 623721 INFO  (qtp1320871993-5891) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:61689_&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2} status=0 QTime=1440
   [junit4]   2> 624207 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\shard-3-001 of type TLOG
   [junit4]   2> 624211 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 624212 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@700a97b2{/,null,AVAILABLE}
   [junit4]   2> 624213 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@63d4ee0d{HTTP/1.1,[http/1.1]}{127.0.0.1:61724}
   [junit4]   2> 624214 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.e.j.s.Server Started @625715ms
   [junit4]   2> 624214 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\tempDir-001/jetty3, hostPort=61723, coreRootDirectory=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\shard-3-001\cores, replicaType=TLOG}
   [junit4]   2> 624214 ERROR (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 624214 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 624215 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 624215 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 624215 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-12T17:38:21.606201600Z
   [junit4]   2> 624220 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 624220 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\shard-3-001\solr.xml
   [junit4]   2> 624223 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 624227 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:61596/solr
   [junit4]   2> 624243 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 624247 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 624251 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61723_
   [junit4]   2> 624253 INFO  (zkCallback-915-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 624253 INFO  (zkCallback-904-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 624254 INFO  (zkCallback-897-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 624254 INFO  (zkCallback-921-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 624254 INFO  (zkCallback-909-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 624291 INFO  (OverseerCollectionConfigSetProcessor-98471738963984388-127.0.0.1:61607_-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 624311 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1
   [junit4]   2> 624316 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1
   [junit4]   2> 624316 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1
   [junit4]   2> 624318 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\shard-3-001\cores
   [junit4]   2> 624346 INFO  (qtp1320871993-5896) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:61723_&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 624349 INFO  (OverseerThreadFactory-2491-thread-5) [    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:61723_ for creating new replica
   [junit4]   2> 624351 INFO  (SocketProxy-Acceptor-61723) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=61736,localport=61723], receiveBufferSize:65536
   [junit4]   2> 624352 INFO  (SocketProxy-Acceptor-61723) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=61724,localport=61737], receiveBufferSize=65536
   [junit4]   2> 624358 INFO  (qtp1108063077-5930) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 624358 INFO  (qtp1108063077-5930) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 625382 INFO  (qtp1108063077-5930) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 625399 INFO  (qtp1108063077-5930) [    ] o.a.s.s.IndexSchema [collection1_shard2_replica_t45] Schema name=test
   [junit4]   2> 625493 INFO  (qtp1108063077-5930) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 625521 INFO  (qtp1108063077-5930) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t45' using configuration from collection collection1, trusted=true
   [junit4]   2> 625521 INFO  (qtp1108063077-5930) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_t45' (registry 'solr.core.collection1.shard2.replica_t45') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6e8d83a1
   [junit4]   2> 625521 INFO  (qtp1108063077-5930) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 625522 INFO  (qtp1108063077-5930) [    ] o.a.s.c.SolrCore [[collection1_shard2_replica_t45] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\shard-3-001\cores\collection1_shard2_replica_t45], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_E1BDA6C84BBA429A-001\shard-3-001\cores\collection1_shard2_replica_t45\data\]
   [junit4]   2> 625524 INFO  (qtp1108063077-5930) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=20, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=9.953125, floorSegmentMB=1.5107421875, forceMergeDeletesPctAllowed=29.206913067574384, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.33991772440890844
   [junit4]   2> 625526 WARN  (qtp1108063077-5930) [    ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 625580 INFO  (qtp1108063077-5930) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 625580 INFO  (qtp1108063077-5930) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 625582 INFO  (qtp1108063077-5930) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 625582 INFO  (qtp1108063077-5930) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 625584 INFO  (qtp1108063077-5930) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=88.3837890625, floorSegmentMB=1.49609375, forceMergeDeletesPctAllowed=27.40377814024113, segmentsPerTier=32.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.33083607878004395
   [junit4]   2> 625584 INFO  (qtp1108063077-5930) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@242d64d1[collection1_shard2_replica_t45] main]
   [junit4]   2> 625584 INFO  (qtp1108063077-5930) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 625585 INFO  (qtp1108063077-5930) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 625585 INFO  (qtp1108063077-5930) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 625587 INFO  (searcherExecutor-2527-thread-1) [    ] o.a.s.c.SolrCore [collection1_shard2_replica_t45] Registered new searcher Searcher@242d64d1[collection1_shard2_replica_t45] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 625587 INFO  (qtp1108063077-5930) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575547833394659328
   [junit4]   2> 625589 INFO  (qtp1108063077-5930) [    ] o.a.s.c.ZkController Core needs to recover:collection1_shard2_replica_t45
   [junit4]   2> 625589 INFO  (qtp1108063077-5930) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1231
   [junit4]   2> 625589 INFO  (updateExecutor-918-thread-1) [    ] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 625590 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 625591 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 625591 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.c.ZkController collection1_shard2_replica_t45 stopping background replication from leader
   [junit4]   2> 625591 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard2_replica_t45]
   [junit4]   2> 625591 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 625591 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard2_replica_t45] as recovering, leader is [http://127.0.0.1:61659/collection1_shard2_replica_t41/] and I am [http://127.0.0.1:61723/collection1_shard2_replica_t45/]
   [junit4]   2> 625593 INFO  (qtp1320871993-5896) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:61723_&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2} status=0 QTime=1246
   [junit4]   2> 625593 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:61659]; [WaitForState: action=PREPRECOVERY&core=collection1_shard2_replica_t41&nodeName=127.0.0.1:61723_&coreNodeName=core_node46&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 625594 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 625594 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Wait for recoveries to finish - wait 30000 for each attempt
   [junit4]   2> 625594 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):30000
   [junit4]   2> 625595 INFO  (SocketProxy-Acceptor-61659) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=61740,localport=61659], receiveBufferSize:65536
   [junit4]   2> 625596 INFO  (SocketProxy-Acceptor-61659) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=61660,localport=61741], receiveBufferSize=65536
   [junit4]   2> 625596 INFO  (qtp537886282-5858) [    ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node46, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
   [junit4]   2> 625596 INFO  (qtp537886282-5858) [    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1_shard2_replica_t41, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:61723_, coreNodeName=core_node46, onlyIfActiveCheckResult=false, nodeProps: core_node46:{"core":"collection1_shard2_replica_t45","base_url":"http://127.0.0.1:61723","node_name":"127.0.0.1:61723_","state":"down","type":"TLOG"}
   [junit4]   2> 626350 INFO  (OverseerCollectionConfigSetProcessor-98471738963984388-127.0.0.1:61607_-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000008 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 626597 INFO  (qtp537886282-5858) [    ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1_shard2_replica_t41, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:61723_, coreNodeName=core_node46, onlyIfActiveCheckResult=false, nodeProps: core_node46:{"core":"collection1_shard2_replica_t45","base_url":"http://127.0.0.1:61723","node_name":"127.0.0.1:61723_","state":"recovering","type":"TLOG"}
   [junit4]   2> 626597 INFO  (qtp537886282-5858) [    ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node46, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
   [junit4]   2> 626597 INFO  (qtp537886282-5858) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:61723_&onlyIfLeaderActive=true&core=collection1_shard2_replica_t41&coreNodeName=core_node46&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1000
   [junit4]   2> 627098 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.c.RecoveryStrategy Starting Replication Recovery.
   [junit4]   2> 627098 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.c.RecoveryStrategy Attempting to replicate from [http://127.0.0.1:61659/collection1_shard2_replica_t41/].
   [junit4]   2> 627099 INFO  (SocketProxy-Acceptor-61659) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=61753,localport=61659], receiveBufferSize:65536
   [junit4]   2> 627100 INFO  (SocketProxy-Acceptor-61659) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=61660,localport=61754], receiveBufferSize=65536
   [junit4]   2> 627100 INFO  (qtp537886282-5866) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1575547834981154816,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 627101 INFO  (qtp537886282-5866) [    ] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 627101 INFO  (qtp537886282-5866) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 627101 INFO  (qtp537886282-5866) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_t41]  webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&commit_end_point=true&wt=javabin&version=2}{commit=} 0 1
   [junit4]   2> 627103 INFO  (SocketProxy-Acceptor-61659) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=61755,localport=61659], receiveBufferSize:65536
   [junit4]   2> 627103 INFO  (SocketProxy-Acceptor-61659) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=61660,localport=61756], receiveBufferSize=65536
   [junit4]   2> 627104 INFO  (qtp537886282-5860) [    ] o.a.s.c.S.Request [collection1_shard2_replica_t41]  webapp= path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 627104 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 627104 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 627104 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 627104 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 627104 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.c.RecoveryStrategy Replication Recovery was successful.
   [junit4]   2> 627104 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
   [junit4]   2> 627104 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.c.ZkController collection1_shard2_replica_t45 starting background replication from leader
   [junit4]   2> 627104 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
   [junit4]   2> 627105 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
   [junit4]   2> 627105 INFO  (indexFetcher-2533-thread-1) [    ] o.a.s.h.IndexFetcher Updated masterUrl to http://127.0.0.1:61659/collection1_shard2_replica_t41/
   [junit4]   2> 627106 INFO  (qtp537886282-5853) [    ] o.a.s.c.S.Request [collection1_shard2_replica_t41]  webapp= path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 627106 INFO  (indexFetcher-2533-thread-1) [    ] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 627106 INFO  (indexFetcher-2533-thread-1) [    ] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 627106 INFO  (indexFetcher-2533-thread-1) [    ] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 627106 INFO  (indexFetcher-2533-thread-1) [    ] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 627106 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.c.RecoveryStrategy Updating version bucket highest from index after successful recovery.
   [junit4]   2> 627106 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575547834987446272
   [junit4]   2> 627108 INFO  (recoveryExecutor-919-thread-1) [    ] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[true]
   [junit4]   2> 627599 INFO  (TEST-HttpPartitionTest.test-seed#[E1BDA6C84BBA429A]) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: collection1
   [junit4]   2> 627600 INFO  (qtp352509678-5808) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1575547835505442816,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 627601 INFO  (qtp352509678-5808) [    ] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.


[...truncated too long message...]

.solr.core.SolrCore@6ae37f8c
   [junit4]   2> 199377 INFO  (coreCloseExecutor-366-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.mainCorpus.shard2.leader, tag=117983536
   [junit4]   2> 199377 INFO  (coreCloseExecutor-366-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.workQueue.shard2.replica_n2, tag=1419827271
   [junit4]   2> 199380 INFO  (coreCloseExecutor-366-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.workQueue.shard2.leader, tag=1419827271
   [junit4]   2> 199381 INFO  (coreCloseExecutor-367-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_n1, tag=484350149
   [junit4]   2> 199381 INFO  (coreCloseExecutor-368-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard2.replica_n2, tag=1793294220
   [junit4]   2> 199385 INFO  (coreCloseExecutor-368-thread-2) [    ] o.a.s.c.SolrCore [destination_shard2_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@11648b5b
   [junit4]   2> 199386 INFO  (coreCloseExecutor-369-thread-2) [    ] o.a.s.c.SolrCore [destination_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@3d4d0861
   [junit4]   2> 199386 INFO  (coreCloseExecutor-369-thread-1) [    ] o.a.s.c.SolrCore [workQueue_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@572e16cf
   [junit4]   2> 199387 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15dd7aee3010054, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 199390 INFO  (coreCloseExecutor-369-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.workQueue.shard1.replica_n1, tag=1462638287
   [junit4]   2> 199391 WARN  (zkCallback-196-thread-2) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 199392 INFO  (jetty-closer-177-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@ef9111f{/solr,null,UNAVAILABLE}
   [junit4]   2> 199392 INFO  (zkCallback-195-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 199392 INFO  (zkCallback-198-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 199392 INFO  (zkCallback-197-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 199394 INFO  (coreCloseExecutor-367-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=484350149
   [junit4]   2> 199403 INFO  (coreCloseExecutor-368-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard2.leader, tag=1793294220
   [junit4]   2> 199405 INFO  (coreCloseExecutor-368-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.destination.shard2.replica_n2, tag=291801947
   [junit4]   2> 199407 INFO  (coreCloseExecutor-369-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.workQueue.shard1.leader, tag=1462638287
   [junit4]   2> 199408 INFO  (coreCloseExecutor-369-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.destination.shard1.replica_n1, tag=1028458593
   [junit4]   2> 199408 INFO  (coreCloseExecutor-368-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.destination.shard2.leader, tag=291801947
   [junit4]   2> 199410 INFO  (coreCloseExecutor-369-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.destination.shard1.leader, tag=1028458593
   [junit4]   2> 199412 INFO  (jetty-closer-177-thread-4) [    ] o.a.s.c.Overseer Overseer (id=98471913004793864-127.0.0.1:49420_solr-n_0000000000) closing
   [junit4]   2> 199413 INFO  (OverseerStateUpdate-98471913004793864-127.0.0.1:49420_solr-n_0000000000) [    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:49420_solr
   [junit4]   2> 199413 INFO  (zkCallback-195-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 199413 INFO  (zkCallback-198-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 199415 INFO  (zkCallback-198-thread-2) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:49410_solr
   [junit4]   2> 199415 INFO  (zkCallback-198-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 199419 INFO  (coreCloseExecutor-367-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.mainCorpus.shard1.replica_n1, tag=1724957103
   [junit4]   2> 199421 INFO  (coreCloseExecutor-367-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.mainCorpus.shard1.leader, tag=1724957103
   [junit4]   2> 199423 WARN  (zkCallback-198-thread-2) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 199423 INFO  (jetty-closer-177-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@16d6795d{/solr,null,UNAVAILABLE}
   [junit4]   2> 200915 WARN  (zkCallback-197-thread-1) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 200916 INFO  (jetty-closer-177-thread-3) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@40fd4a9c{/solr,null,UNAVAILABLE}
   [junit4]   2> 200918 WARN  (zkCallback-195-thread-2) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 200918 INFO  (jetty-closer-177-thread-4) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1c06fdfa{/solr,null,UNAVAILABLE}
   [junit4]   2> 200922 ERROR (SUITE-StreamExpressionTest-seed#[375A90A69E300888]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 200922 INFO  (SUITE-StreamExpressionTest-seed#[375A90A69E300888]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:49406 49406
   [junit4]   2> 200924 INFO  (Thread-136) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:49406 49406
   [junit4]   2> 200924 WARN  (Thread-136) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 92 /solr/aliases.json
   [junit4]   2> 14 /solr/configs/conf
   [junit4]   2> 8 /solr/configs/ml
   [junit4]   2> 4 /solr/security.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 92 /solr/clusterprops.json
   [junit4]   2> 92 /solr/clusterstate.json
   [junit4]   2> 40 /solr/collections/destinationCollection/state.json
   [junit4]   2> 28 /solr/collections/parallelDestinationCollection1/state.json
   [junit4]   2> 22 /solr/collections/workQueue/state.json
   [junit4]   2> 20 /solr/collections/destination/state.json
   [junit4]   2> 20 /solr/collections/parallelDestinationCollection/state.json
   [junit4]   2> 20 /solr/collections/mainCorpus/state.json
   [junit4]   2> 10 /solr/collections/collection1/state.json
   [junit4]   2> 10 /solr/collections/uknownCollection/state.json
   [junit4]   2> 10 /solr/collections/modelCollection/state.json
   [junit4]   2> 10 /solr/collections/checkpointCollection/state.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 92 /solr/live_nodes
   [junit4]   2> 92 /solr/collections
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.io.stream.StreamExpressionTest_375A90A69E300888-001
   [junit4]   2> Aug 12, 2017 6:24:41 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70), sim=RandomSimilarity(queryNorm=false): {}, locale=en-LR, timezone=America/Sao_Paulo
   [junit4]   2> NOTE: Windows 10 10.0 amd64/Oracle Corporation 9 (64-bit)/cpus=3,threads=1,free=201209640,total=305135616
   [junit4]   2> NOTE: All tests run in this JVM: [SolrExampleXMLTest, TestSpellCheckResponse, JdbcDriverTest, JettyWebappTest, TestSolrJErrorHandling, TestConfigSetAdminRequest, AbsoluteValueEvaluatorTest, JDBCStreamTest, NamedListTest, GreaterThanEqualToEvaluatorTest, TestPathTrie, HyperbolicTangentEvaluatorTest, SolrDocumentTest, TestValidatingJsonMap, GraphExpressionTest, CollectionAdminRequestRequiredParamsTest, StreamExpressionToExplanationTest, NaturalLogEvaluatorTest, TestJavaBinCodec, ArrayEvaluatorTest, TestSolrProperties, StreamExpressionTest]
   [junit4] Completed [80/135 (1!)] on J0 in 132.98s, 94 tests, 1 error <<< FAILURES!

[...truncated 35805 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk1.8.0_144) - Build # 6820 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6820/
Java: 64bit/jdk1.8.0_144 -XX:+UseCompressedOops -XX:+UseSerialGC

2 tests failed.
FAILED:  org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly

Error Message:
Unexpected number of elements in the group for intGSF: 8

Stack Trace:
java.lang.AssertionError: Unexpected number of elements in the group for intGSF: 8
        at __randomizedtesting.SeedInfo.seed([723BDDF842FD1BE5:E980B3A00FA529BB]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly(DocValuesNotIndexedTest.java:377)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.HealthCheckHandlerTest.testHealthCheckHandlerSolrJ

Error Message:
Error from server at http://127.0.0.1:52432/solr: Host Unavailable: Not connected to zk

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:52432/solr: Host Unavailable: Not connected to zk
        at __randomizedtesting.SeedInfo.seed([723BDDF842FD1BE5:D8C802EC391B53E4]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
        at org.apache.solr.cloud.HealthCheckHandlerTest.testHealthCheckHandlerSolrJ(HealthCheckHandlerTest.java:78)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 11071 lines...]
   [junit4] Suite: org.apache.solr.cloud.HealthCheckHandlerTest
   [junit4]   2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HealthCheckHandlerTest_723BDDF842FD1BE5-001\init-core-data-001
   [junit4]   2> 382198 WARN  (SUITE-HealthCheckHandlerTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
   [junit4]   2> 382198 INFO  (SUITE-HealthCheckHandlerTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 382202 INFO  (SUITE-HealthCheckHandlerTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 382202 INFO  (SUITE-HealthCheckHandlerTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 382204 INFO  (SUITE-HealthCheckHandlerTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HealthCheckHandlerTest_723BDDF842FD1BE5-001\tempDir-001
   [junit4]   2> 382204 INFO  (SUITE-HealthCheckHandlerTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 382205 INFO  (Thread-1296) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 382205 INFO  (Thread-1296) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 382209 ERROR (Thread-1296) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 382305 INFO  (SUITE-HealthCheckHandlerTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:52428
   [junit4]   2> 382335 INFO  (jetty-launcher-1240-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 382335 INFO  (jetty-launcher-1240-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1f20b3d{/solr,null,AVAILABLE}
   [junit4]   2> 382341 INFO  (jetty-launcher-1240-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@45775159{HTTP/1.1,[http/1.1]}{127.0.0.1:52432}
   [junit4]   2> 382341 INFO  (jetty-launcher-1240-thread-1) [    ] o.e.j.s.Server Started @390549ms
   [junit4]   2> 382341 INFO  (jetty-launcher-1240-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52432}
   [junit4]   2> 382342 ERROR (jetty-launcher-1240-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 382342 INFO  (jetty-launcher-1240-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 382342 INFO  (jetty-launcher-1240-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 382342 INFO  (jetty-launcher-1240-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 382342 INFO  (jetty-launcher-1240-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-13T18:33:55.797Z
   [junit4]   2> 382357 INFO  (jetty-launcher-1240-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 382370 INFO  (jetty-launcher-1240-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52428/solr
   [junit4]   2> 382373 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15ddcdfb8d00002, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 382475 INFO  (jetty-launcher-1240-thread-1) [n:127.0.0.1:52432_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 382476 INFO  (jetty-launcher-1240-thread-1) [n:127.0.0.1:52432_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52432_solr
   [junit4]   2> 382476 INFO  (jetty-launcher-1240-thread-1) [n:127.0.0.1:52432_solr    ] o.a.s.c.Overseer Overseer (id=98477620308475907-127.0.0.1:52432_solr-n_0000000000) starting
   [junit4]   2> 382479 INFO  (jetty-launcher-1240-thread-1) [n:127.0.0.1:52432_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52432_solr
   [junit4]   2> 382485 INFO  (zkCallback-1247-thread-1-processing-n:127.0.0.1:52432_solr) [n:127.0.0.1:52432_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 382615 INFO  (jetty-launcher-1240-thread-1) [n:127.0.0.1:52432_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52432.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74cc2851
   [junit4]   2> 382617 INFO  (jetty-launcher-1240-thread-1) [n:127.0.0.1:52432_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52432.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74cc2851
   [junit4]   2> 382617 INFO  (jetty-launcher-1240-thread-1) [n:127.0.0.1:52432_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52432.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74cc2851
   [junit4]   2> 382618 INFO  (jetty-launcher-1240-thread-1) [n:127.0.0.1:52432_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HealthCheckHandlerTest_723BDDF842FD1BE5-001\tempDir-001\node1\.
   [junit4]   2> 382642 INFO  (SUITE-HealthCheckHandlerTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 382643 INFO  (SUITE-HealthCheckHandlerTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:52428/solr ready
   [junit4]   2> 382652 INFO  (TEST-HealthCheckHandlerTest.testHealthCheckHandlerWithCloudClient-seed#[723BDDF842FD1BE5]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testHealthCheckHandlerWithCloudClient
   [junit4]   2> 382656 INFO  (TEST-HealthCheckHandlerTest.testHealthCheckHandlerWithCloudClient-seed#[723BDDF842FD1BE5]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testHealthCheckHandlerWithCloudClient
   [junit4]   2> 382665 INFO  (TEST-HealthCheckHandlerTest.testHealthCheckHandler-seed#[723BDDF842FD1BE5]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testHealthCheckHandler
   [junit4]   2> 382669 INFO  (qtp1373278149-6630) [n:127.0.0.1:52432_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/health params={wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 382670 INFO  (TEST-HealthCheckHandlerTest.testHealthCheckHandler-seed#[723BDDF842FD1BE5]) [    ] o.a.s.c.MiniSolrCloudCluster Expired zookeeper session 98477620308475907 from node http://127.0.0.1:52432/solr
   [junit4]   2> 382671 INFO  (zkCallback-1252-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 382772 INFO  (qtp1373278149-6625) [n:127.0.0.1:52432_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/health params={wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 382772 INFO  (TEST-HealthCheckHandlerTest.testHealthCheckHandler-seed#[723BDDF842FD1BE5]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testHealthCheckHandler
   [junit4]   2> 382773 WARN  (zkCallback-1247-thread-8-processing-n:127.0.0.1:52432_solr) [n:127.0.0.1:52432_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@22e3542a name: ZooKeeperConnection Watcher:127.0.0.1:52428/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 382774 WARN  (zkCallback-1247-thread-8-processing-n:127.0.0.1:52432_solr) [n:127.0.0.1:52432_solr    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 382794 INFO  (TEST-HealthCheckHandlerTest.testHealthCheckHandlerSolrJ-seed#[723BDDF842FD1BE5]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testHealthCheckHandlerSolrJ
   [junit4]   2> 382808 INFO  (qtp1373278149-6622) [n:127.0.0.1:52432_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/health params={wt=javabin&version=2} status=503 QTime=0
   [junit4]   2> 382809 INFO  (TEST-HealthCheckHandlerTest.testHealthCheckHandlerSolrJ-seed#[723BDDF842FD1BE5]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testHealthCheckHandlerSolrJ
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HealthCheckHandlerTest -Dtests.method=testHealthCheckHandlerSolrJ -Dtests.seed=723BDDF842FD1BE5 -Dtests.slow=true -Dtests.locale=nl-NL -Dtests.timezone=America/Los_Angeles -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.04s J0 | HealthCheckHandlerTest.testHealthCheckHandlerSolrJ <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:52432/solr: Host Unavailable: Not connected to zk
   [junit4]    > at __randomizedtesting.SeedInfo.seed([723BDDF842FD1BE5:D8C802EC391B53E4]:0)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
   [junit4]    > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
   [junit4]    > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
   [junit4]    > at org.apache.solr.cloud.HealthCheckHandlerTest.testHealthCheckHandlerSolrJ(HealthCheckHandlerTest.java:78)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 382816 INFO  (jetty-closer-1241-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@45775159{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 382816 INFO  (jetty-closer-1241-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=710518707
   [junit4]   2> 382817 INFO  (jetty-closer-1241-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 382819 INFO  (jetty-closer-1241-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 382821 INFO  (jetty-closer-1241-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 383808 WARN  (jetty-launcher-1240-thread-1-SendThread(127.0.0.1:52428)) [n:127.0.0.1:52432_solr    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x15ddcdfb8d00003 has expired
   [junit4]   2> 383808 WARN  (zkCallback-1247-thread-4-processing-n:127.0.0.1:52432_solr) [n:127.0.0.1:52432_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@22e3542a name: ZooKeeperConnection Watcher:127.0.0.1:52428/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 383808 WARN  (zkCallback-1247-thread-4-processing-n:127.0.0.1:52432_solr) [n:127.0.0.1:52432_solr    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 383808 WARN  (OverseerStateUpdate-98477620308475907-127.0.0.1:52432_solr-n_0000000000) [n:127.0.0.1:52432_solr    ] o.a.s.c.Overseer Solr cannot talk to ZK, exiting Overseer main queue loop
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2> at org.apache.solr.cloud.DistributedQueue.fetchZkChildren(DistributedQueue.java:305)
   [junit4]   2> at org.apache.solr.cloud.DistributedQueue.firstChild(DistributedQueue.java:282)
   [junit4]   2> at org.apache.solr.cloud.DistributedQueue.firstElement(DistributedQueue.java:398)
   [junit4]   2> at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:161)
   [junit4]   2> at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:139)
   [junit4]   2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:184)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 383808 INFO  (OverseerStateUpdate-98477620308475907-127.0.0.1:52432_solr-n_0000000000) [n:127.0.0.1:52432_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:52432_solr
   [junit4]   2> 383808 INFO  (zkCallback-1247-thread-4-processing-n:127.0.0.1:52432_solr) [n:127.0.0.1:52432_solr    ] o.a.s.c.Overseer Overseer (id=98477620308475907-127.0.0.1:52432_solr-n_0000000000) closing
   [junit4]   2> 383808 WARN  (OverseerCollectionConfigSetProcessor-98477620308475907-127.0.0.1:52432_solr-n_0000000000) [n:127.0.0.1:52432_solr    ] o.a.s.c.OverseerTaskProcessor Overseer cannot talk to ZK
   [junit4]   2> 383809 INFO  (zkCallback-1247-thread-4-processing-n:127.0.0.1:52432_solr) [n:127.0.0.1:52432_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 383809 WARN  (zkCallback-1247-thread-4-processing-n:127.0.0.1:52432_solr) [n:127.0.0.1:52432_solr    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 383811 INFO  (zkCallback-1247-thread-4-processing-n:127.0.0.1:52432_solr) [n:127.0.0.1:52432_solr    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 383811 INFO  (zkCallback-1247-thread-4-processing-n:127.0.0.1:52432_solr) [n:127.0.0.1:52432_solr    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 383912 ERROR (jetty-closer-1241-thread-1) [    ] o.a.s.c.Overseer Could not create Overseer node
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$9.execute(SolrZkClient.java:392)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$9.execute(SolrZkClient.java:389)
   [junit4]   2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:389)
   [junit4]   2> at org.apache.solr.cloud.Overseer.createOverseerNode(Overseer.java:758)
   [junit4]   2> at org.apache.solr.cloud.Overseer.getStateUpdateQueue(Overseer.java:631)
   [junit4]   2> at org.apache.solr.cloud.Overseer.getStateUpdateQueue(Overseer.java:618)
   [junit4]   2> at org.apache.solr.cloud.ZkController.publishNodeAsDown(ZkController.java:2515)
   [junit4]   2> at org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:743)
   [junit4]   2> at org.apache.solr.servlet.SolrDispatchFilter.destroy(SolrDispatchFilter.java:313)
   [junit4]   2> at org.eclipse.jetty.servlet.FilterHolder.destroyInstance(FilterHolder.java:172)
   [junit4]   2> at org.eclipse.jetty.servlet.FilterHolder.doStop(FilterHolder.java:150)
   [junit4]   2> at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doStop(ServletHandler.java:238)
   [junit4]   2> at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:142)
   [junit4]   2> at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:160)
   [junit4]   2> at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:73)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doStop(SessionHandler.java:127)
   [junit4]   2> at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:142)
   [junit4]   2> at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:160)
   [junit4]   2> at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:73)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.stopContext(ContextHandler.java:825)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletContextHandler.stopContext(ServletContextHandler.java:356)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doStop(ContextHandler.java:880)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletContextHandler.doStop(ServletContextHandler.java:272)
   [junit4]   2> at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:142)
   [junit4]   2> at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:160)
   [junit4]   2> at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:73)
   [junit4]   2> at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> at org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:142)
   [junit4]   2> at org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:160)
   [junit4]   2> at org.eclipse.jetty.server.handler.AbstractHandler.doStop(AbstractHandler.java:73)
   [junit4]   2> at org.eclipse.jetty.server.Server.doStop(Server.java:493)
   [junit4]   2> at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> at org.apache.solr.client.solrj.embedded.JettySolrRunner.stop(JettySolrRunner.java:428)
   [junit4]   2> at org.apache.solr.cloud.MiniSolrCloudCluster.stopJettySolrRunner(MiniSolrCloudCluster.java:423)
   [junit4]   2> at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$shutdown$1(MiniSolrCloudCluster.java:460)
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 383912 WARN  (jetty-closer-1241-thread-1) [    ] o.a.s.c.ZkController Could not publish node as down: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer
   [junit4]   2> 383913 INFO  (jetty-closer-1241-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 384272 WARN  (zkCallback-1247-thread-1-processing-n:127.0.0.1:52432_solr) [n:127.0.0.1:52432_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 384273 ERROR (zkCallback-1247-thread-4-processing-n:127.0.0.1:52432_solr) [n:127.0.0.1:52432_solr    ] o.a.s.c.ZkController :org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /live_nodes
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2> at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:674)
   [junit4]   2> at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:391)
   [junit4]   2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:320)
   [junit4]   2> at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2> at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$3.lambda$process$0(SolrZkClient.java:269)
   [junit4]   2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>
   [junit4]   2> 384273 ERROR (zkCallback-1247-thread-4-processing-n:127.0.0.1:52432_solr) [n:127.0.0.1:52432_solr    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed:org.apache.solr.common.cloud.ZooKeeperException:
   [junit4]   2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:405)
   [junit4]   2> at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:168)
   [junit4]   2> at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:142)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$3.lambda$process$0(SolrZkClient.java:269)
   [junit4]   2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /live_nodes
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1532)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:340)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:337)
   [junit4]   2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:337)
   [junit4]   2> at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:674)
   [junit4]   2> at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:391)
   [junit4]   2> at org.apache.solr.cloud.ZkController$1.command(ZkController.java:320)
   [junit4]   2> ... 10 more
   [junit4]   2>
   [junit4]   2> 384274 WARN  (zkCallback-1247-thread-4-processing-n:127.0.0.1:52432_solr) [n:127.0.0.1:52432_solr    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
   [junit4]   2> 384275 INFO  (zkCallback-1247-thread-4-processing-n:127.0.0.1:52432_solr) [n:127.0.0.1:52432_solr    ] o.a.s.c.c.ConnectionManager Connected:true
   [junit4]   2> 384276 INFO  (jetty-closer-1241-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1f20b3d{/solr,null,UNAVAILABLE}
   [junit4]   2> 384279 ERROR (SUITE-HealthCheckHandlerTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 384279 INFO  (SUITE-HealthCheckHandlerTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:52428 52428
   [junit4]   2> 384280 INFO  (Thread-1296) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:52428 52428
   [junit4]   2> 384284 WARN  (Thread-1296) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/clusterprops.json
   [junit4]   2> 2 /solr/aliases.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 2 /solr/clusterstate.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 2 /solr/live_nodes
   [junit4]   2> 2 /solr/collections
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HealthCheckHandlerTest_723BDDF842FD1BE5-001
   [junit4]   2> Aug 13, 2017 6:33:57 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=true): {}, locale=nl-NL, timezone=America/Los_Angeles
   [junit4]   2> NOTE: Windows 10 10.0 amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=3,threads=1,free=58543712,total=222011392
   [junit4]   2> NOTE: All tests run in this JVM: [TestImpersonationWithHadoopAuth, DefaultValueUpdateProcessorTest, TestSizeLimitedDistributedMap, TestNestedDocsSort, TestReplicationHandler, PrimitiveFieldTypeTest, TestManagedSchemaAPI, TestFunctionQuery, TestSolrQueryResponse, PropertiesRequestHandlerTest, SliceStateTest, ClusterStateTest, TestMacros, TestTrackingShardHandlerFactory, SolrIndexSplitterTest, HdfsRecoveryZkTest, CdcrUpdateLogTest, TestExactStatsCache, TestSmileRequest, BasicAuthIntegrationTest, NumericFieldsTest, SyncSliceTest, TestFuzzyAnalyzedSuggestions, V2StandaloneTest, SimpleFacetsTest, TestManagedResourceStorage, TestPKIAuthenticationPlugin, OverseerTaskQueueTest, TestSubQueryTransformer, VersionInfoTest, TestSha256AuthenticationProvider, TestStressLiveNodes, TimeZoneUtilsTest, DistributedFacetExistsSmallTest, UniqFieldsUpdateProcessorFactoryTest, ResponseHeaderTest, DateMathParserTest, SparseHLLTest, CircularListTest, TestPointFields, TestJmxIntegration, TestStressRecovery, DistributedDebugComponentTest, TestSystemCollAutoCreate, TestLegacyNumericRangeQueryBuilder, TestQueryWrapperFilter, TestCustomDocTransformer, DirectSolrConnectionTest, SSLMigrationTest, TestStressCloudBlindAtomicUpdates, BigEndianAscendingWordSerializerTest, TestNoOpRegenerator, TestConfigOverlay, TestNRTOpen, SolrShardReporterTest, TestOrdValues, TestSolrCloudSnapshots, FastVectorHighlighterTest, PathHierarchyTokenizerFactoryTest, HealthCheckHandlerTest]
   [junit4] Completed [110/729 (1!)] on J0 in 2.62s, 3 tests, 1 error <<< FAILURES!

[...truncated 1384 lines...]
   [junit4] Suite: org.apache.solr.cloud.DocValuesNotIndexedTest
   [junit4]   2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.DocValuesNotIndexedTest_723BDDF842FD1BE5-001\init-core-data-001
   [junit4]   2> 1989111 WARN  (SUITE-DocValuesNotIndexedTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=7 numCloses=7
   [junit4]   2> 1989111 INFO  (SUITE-DocValuesNotIndexedTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1989117 INFO  (SUITE-DocValuesNotIndexedTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 1989117 INFO  (SUITE-DocValuesNotIndexedTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1989119 INFO  (SUITE-DocValuesNotIndexedTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.DocValuesNotIndexedTest_723BDDF842FD1BE5-001\tempDir-001
   [junit4]   2> 1989120 INFO  (SUITE-DocValuesNotIndexedTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1989120 INFO  (Thread-5329) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1989120 INFO  (Thread-5329) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1989126 ERROR (Thread-5329) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1989220 INFO  (SUITE-DocValuesNotIndexedTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:64942
   [junit4]   2> 1989231 INFO  (jetty-launcher-3214-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1989231 INFO  (jetty-launcher-3214-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1989233 INFO  (jetty-launcher-3214-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2a087f8e{/solr,null,AVAILABLE}
   [junit4]   2> 1989234 INFO  (jetty-launcher-3214-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3ce1e42f{/solr,null,AVAILABLE}
   [junit4]   2> 1989235 INFO  (jetty-launcher-3214-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@401ded7c{HTTP/1.1,[http/1.1]}{127.0.0.1:64946}
   [junit4]   2> 1989235 INFO  (jetty-launcher-3214-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@66942c1a{HTTP/1.1,[http/1.1]}{127.0.0.1:64947}
   [junit4]   2> 1989236 INFO  (jetty-launcher-3214-thread-2) [    ] o.e.j.s.Server Started @1997230ms
   [junit4]   2> 1989235 INFO  (jetty-launcher-3214-thread-1) [    ] o.e.j.s.Server Started @1997229ms
   [junit4]   2> 1989237 INFO  (jetty-launcher-3214-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=64947}
   [junit4]   2> 1989237 INFO  (jetty-launcher-3214-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=64946}
   [junit4]   2> 1989237 ERROR (jetty-launcher-3214-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1989237 ERROR (jetty-launcher-3214-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1989237 INFO  (jetty-launcher-3214-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1989237 INFO  (jetty-launcher-3214-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1989237 INFO  (jetty-launcher-3214-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1989237 INFO  (jetty-launcher-3214-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1989237 INFO  (jetty-launcher-3214-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1989237 INFO  (jetty-launcher-3214-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1989237 INFO  (jetty-launcher-3214-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-13T19:00:42.478Z
   [junit4]   2> 1989237 INFO  (jetty-launcher-3214-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-13T19:00:42.478Z
   [junit4]   2> 1989243 INFO  (jetty-launcher-3214-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1989243 INFO  (jetty-launcher-3214-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1989263 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15ddcf83cfe0001, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1989266 INFO  (jetty-launcher-3214-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:64942/solr
   [junit4]   2> 1989267 INFO  (jetty-launcher-3214-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:64942/solr
   [junit4]   2> 1989270 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15ddcf83cfe0003, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1989412 INFO  (jetty-launcher-3214-thread-2) [n:127.0.0.1:64947_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1989412 INFO  (jetty-launcher-3214-thread-1) [n:127.0.0.1:64946_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1989412 INFO  (jetty-launcher-3214-thread-2) [n:127.0.0.1:64947_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:64947_solr
   [junit4]   2> 1989413 INFO  (jetty-launcher-3214-thread-1) [n:127.0.0.1:64946_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64946_solr
   [junit4]   2> 1989414 INFO  (jetty-launcher-3214-thread-2) [n:127.0.0.1:64947_solr    ] o.a.s.c.Overseer Overseer (id=98477725605298181-127.0.0.1:64947_solr-n_0000000000) starting
   [junit4]   2> 1989415 INFO  (zkCallback-3225-thread-1-processing-n:127.0.0.1:64946_solr) [n:127.0.0.1:64946_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1989416 INFO  (zkCallback-3226-thread-1-processing-n:127.0.0.1:64947_solr) [n:127.0.0.1:64947_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1989424 INFO  (jetty-launcher-3214-thread-2) [n:127.0.0.1:64947_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64947_solr
   [junit4]   2> 1989427 INFO  (zkCallback-3225-thread-1-processing-n:127.0.0.1:64946_solr) [n:127.0.0.1:64946_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1989427 INFO  (zkCallback-3226-thread-1-processing-n:127.0.0.1:64947_solr) [n:127.0.0.1:64947_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1989543 INFO  (jetty-launcher-3214-thread-1) [n:127.0.0.1:64946_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64946.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446fd10c
   [junit4]   2> 1989543 INFO  (jetty-launcher-3214-thread-2) [n:127.0.0.1:64947_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64947.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446fd10c
   [junit4]   2> 1989544 INFO  (jetty-launcher-3214-thread-1) [n:127.0.0.1:64946_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64946.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446fd10c
   [junit4]   2> 1989545 INFO  (jetty-launcher-3214-thread-1) [n:127.0.0.1:64946_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64946.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446fd10c
   [junit4]   2> 1989546 INFO  (jetty-launcher-3214-thread-1) [n:127.0.0.1:64946_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.DocValuesNotIndexedTest_723BDDF842FD1BE5-001\tempDir-001\node1\.
   [junit4]   2> 1989547 INFO  (jetty-launcher-3214-thread-2) [n:127.0.0.1:64947_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64947.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446fd10c
   [junit4]   2> 1989548 INFO  (jetty-launcher-3214-thread-2) [n:127.0.0.1:64947_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64947.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446fd10c
   [junit4]   2> 1989549 INFO  (jetty-launcher-3214-thread-2) [n:127.0.0.1:64947_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.DocValuesNotIndexedTest_723BDDF842FD1BE5-001\tempDir-001\node2\.
   [junit4]   2> 1989605 INFO  (SUITE-DocValuesNotIndexedTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1989607 INFO  (SUITE-DocValuesNotIndexedTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:64942/solr ready
   [junit4]   2> 1989613 INFO  (qtp1657275198-20565) [n:127.0.0.1:64946_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&maxShardsPerNode=2&name=dv_coll&nrtReplicas=1&action=CREATE&numShards=4&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1989615 INFO  (OverseerThreadFactory-9551-thread-1-processing-n:127.0.0.1:64947_solr) [n:127.0.0.1:64947_solr    ] o.a.s.c.CreateCollectionCmd Create collection dv_coll
   [junit4]   2> 1989727 INFO  (OverseerStateUpdate-98477725605298181-127.0.0.1:64947_solr-n_0000000000) [n:127.0.0.1:64947_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"dv_coll",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"dv_coll_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:64946/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1989733 INFO  (OverseerStateUpdate-98477725605298181-127.0.0.1:64947_solr-n_0000000000) [n:127.0.0.1:64947_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"dv_coll",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"dv_coll_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:64947/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1989736 INFO  (OverseerStateUpdate-98477725605298181-127.0.0.1:64947_solr-n_0000000000) [n:127.0.0.1:64947_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"dv_coll",
   [junit4]   2>   "shard":"shard3",
   [junit4]   2>   "core":"dv_coll_shard3_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:64946/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1989740 INFO  (OverseerStateUpdate-98477725605298181-127.0.0.1:64947_solr-n_0000000000) [n:127.0.0.1:64947_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"dv_coll",
   [junit4]   2>   "shard":"shard4",
   [junit4]   2>   "core":"dv_coll_shard4_replica_n6",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:64947/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1989944 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=dv_coll_shard1_replica_n1&action=CREATE&numShards=4&collection=dv_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1989944 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=dv_coll_shard2_replica_n2&action=CREATE&numShards=4&collection=dv_coll&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1989944 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=dv_coll_shard3_replica_n4&action=CREATE&numShards=4&collection=dv_coll&shard=shard3&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1989945 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1989946 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1989946 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=dv_coll_shard4_replica_n6&action=CREATE&numShards=4&collection=dv_coll&shard=shard4&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1990061 INFO  (zkCallback-3226-thread-1-processing-n:127.0.0.1:64947_solr) [n:127.0.0.1:64947_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1990061 INFO  (zkCallback-3225-thread-1-processing-n:127.0.0.1:64946_solr) [n:127.0.0.1:64946_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1990061 INFO  (zkCallback-3226-thread-2-processing-n:127.0.0.1:64947_solr) [n:127.0.0.1:64947_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1990061 INFO  (zkCallback-3225-thread-2-processing-n:127.0.0.1:64946_solr) [n:127.0.0.1:64946_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1991002 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1991003 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1991015 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.s.IndexSchema [dv_coll_shard4_replica_n6] Schema name=minimal
   [junit4]   2> 1991015 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.s.IndexSchema [dv_coll_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 1991042 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1991042 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'dv_coll_shard4_replica_n6' using configuration from collection dv_coll, trusted=true
   [junit4]   2> 1991043 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64947.solr.core.dv_coll.shard4.replica_n6' (registry 'solr.core.dv_coll.shard4.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446fd10c
   [junit4]   2> 1991043 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1991044 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.SolrCore [[dv_coll_shard4_replica_n6] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.DocValuesNotIndexedTest_723BDDF842FD1BE5-001\tempDir-001\node2\dv_coll_shard4_replica_n6], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.DocValuesNotIndexedTest_723BDDF842FD1BE5-001\tempDir-001\node2\.\dv_coll_shard4_replica_n6\data\]
   [junit4]   2> 1991044 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1991044 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'dv_coll_shard1_replica_n1' using configuration from collection dv_coll, trusted=true
   [junit4]   2> 1991045 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64946.solr.core.dv_coll.shard1.replica_n1' (registry 'solr.core.dv_coll.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446fd10c
   [junit4]   2> 1991045 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1991045 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.SolrCore [[dv_coll_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.DocValuesNotIndexedTest_723BDDF842FD1BE5-001\tempDir-001\node1\dv_coll_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.DocValuesNotIndexedTest_723BDDF842FD1BE5-001\tempDir-001\node1\.\dv_coll_shard1_replica_n1\data\]
   [junit4]   2> 1991065 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1991066 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1991078 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.IndexSchema [dv_coll_shard2_replica_n2] Schema name=minimal
   [junit4]   2> 1991078 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.s.IndexSchema [dv_coll_shard3_replica_n4] Schema name=minimal
   [junit4]   2> 1991081 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1991081 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1991081 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'dv_coll_shard3_replica_n4' using configuration from collection dv_coll, trusted=true
   [junit4]   2> 1991081 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'dv_coll_shard2_replica_n2' using configuration from collection dv_coll, trusted=true
   [junit4]   2> 1991082 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64947.solr.core.dv_coll.shard2.replica_n2' (registry 'solr.core.dv_coll.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446fd10c
   [junit4]   2> 1991082 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1991082 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SolrCore [[dv_coll_shard2_replica_n2] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.DocValuesNotIndexedTest_723BDDF842FD1BE5-001\tempDir-001\node2\dv_coll_shard2_replica_n2], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.DocValuesNotIndexedTest_723BDDF842FD1BE5-001\tempDir-001\node2\.\dv_coll_shard2_replica_n2\data\]
   [junit4]   2> 1991091 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64946.solr.core.dv_coll.shard3.replica_n4' (registry 'solr.core.dv_coll.shard3.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@446fd10c
   [junit4]   2> 1991091 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1991091 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.SolrCore [[dv_coll_shard3_replica_n4] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.DocValuesNotIndexedTest_723BDDF842FD1BE5-001\tempDir-001\node1\dv_coll_shard3_replica_n4], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.DocValuesNotIndexedTest_723BDDF842FD1BE5-001\tempDir-001\node1\.\dv_coll_shard3_replica_n4\data\]
   [junit4]   2> 1991275 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1991275 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1991279 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1991279 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1991281 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1991281 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1991282 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1991282 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1991363 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1991363 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1991365 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1991365 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1991365 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1991365 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1991367 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1991367 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1991369 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@7440e658[dv_coll_shard4_replica_n6] main]
   [junit4]   2> 1991370 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1991371 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1991371 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1991371 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@7bb39830[dv_coll_shard3_replica_n4] main]
   [junit4]   2> 1991371 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 1991373 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1991373 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575643612041969664
   [junit4]   2> 1991374 INFO  (searcherExecutor-9556-thread-1-processing-n:127.0.0.1:64947_solr x:dv_coll_shard4_replica_n6 s:shard4 c:dv_coll r:core_node8) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.SolrCore [dv_coll_shard4_replica_n6] Registered new searcher Searcher@7440e658[dv_coll_shard4_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1991375 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1991375 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1991375 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1991376 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 1991376 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4598242c[dv_coll_shard1_replica_n1] main]
   [junit4]   2> 1991376 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1991377 INFO  (searcherExecutor-9558-thread-1-processing-n:127.0.0.1:64946_solr x:dv_coll_shard3_replica_n4 s:shard3 c:dv_coll r:core_node7) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.SolrCore [dv_coll_shard3_replica_n4] Registered new searcher Searcher@7bb39830[dv_coll_shard3_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1991378 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575643612047212544
   [junit4]   2> 1991382 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@75ff59d4[dv_coll_shard2_replica_n2] main]
   [junit4]   2> 1991382 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1991382 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1991382 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:64947/solr/dv_coll_shard4_replica_n6/
   [junit4]   2> 1991382 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1991383 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.SyncStrategy http://127.0.0.1:64947/solr/dv_coll_shard4_replica_n6/ has no replicas
   [junit4]   2> 1991383 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1991383 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1991383 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1991384 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1991384 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1991384 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1991384 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/conf1/managed-schema
   [junit4]   2> 1991385 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 1991386 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 1991386 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1991386 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575643612055601152
   [junit4]   2> 1991386 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1991387 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575643612056649728
   [junit4]   2> 1991387 INFO  (searcherExecutor-9557-thread-1-processing-n:127.0.0.1:64946_solr x:dv_coll_shard1_replica_n1 s:shard1 c:dv_coll r:core_node3) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.SolrCore [dv_coll_shard1_replica_n1] Registered new searcher Searcher@4598242c[dv_coll_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1991389 INFO  (searcherExecutor-9559-thread-1-processing-n:127.0.0.1:64947_solr x:dv_coll_shard2_replica_n2 s:shard2 c:dv_coll r:core_node5) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SolrCore [dv_coll_shard2_replica_n2] Registered new searcher Searcher@75ff59d4[dv_coll_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1991393 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1991393 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1991393 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:64946/solr/dv_coll_shard3_replica_n4/
   [junit4]   2> 1991393 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1991393 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.SyncStrategy http://127.0.0.1:64946/solr/dv_coll_shard3_replica_n4/ has no replicas
   [junit4]   2> 1991393 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1991396 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:64947/solr/dv_coll_shard4_replica_n6/ shard4
   [junit4]   2> 1991402 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1991402 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1991402 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:64947/solr/dv_coll_shard2_replica_n2/
   [junit4]   2> 1991402 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1991402 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1991402 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1991403 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:64946/solr/dv_coll_shard1_replica_n1/
   [junit4]   2> 1991403 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1991403 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:64946/solr/dv_coll_shard1_replica_n1/ has no replicas
   [junit4]   2> 1991403 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1991402 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:64947/solr/dv_coll_shard2_replica_n2/ has no replicas
   [junit4]   2> 1991403 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1991408 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:64946/solr/dv_coll_shard3_replica_n4/ shard3
   [junit4]   2> 1991411 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:64946/solr/dv_coll_shard1_replica_n1/ shard1
   [junit4]   2> 1991411 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:64947/solr/dv_coll_shard2_replica_n2/ shard2
   [junit4]   2> 1991519 INFO  (zkCallback-3226-thread-2-processing-n:127.0.0.1:64947_solr) [n:127.0.0.1:64947_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1991519 INFO  (zkCallback-3226-thread-1-processing-n:127.0.0.1:64947_solr) [n:127.0.0.1:64947_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1991519 INFO  (zkCallback-3225-thread-1-processing-n:127.0.0.1:64946_solr) [n:127.0.0.1:64946_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1991519 INFO  (zkCallback-3225-thread-2-processing-n:127.0.0.1:64946_solr) [n:127.0.0.1:64946_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1991550 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1991552 INFO  (qtp1379579731-20572) [n:127.0.0.1:64947_solr c:dv_coll s:shard4 r:core_node8 x:dv_coll_shard4_replica_n6] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=dv_coll_shard4_replica_n6&action=CREATE&numShards=4&collection=dv_coll&shard=shard4&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1604
   [junit4]   2> 1991561 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1991562 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1991563 INFO  (qtp1657275198-20568) [n:127.0.0.1:64946_solr c:dv_coll s:shard3 r:core_node7 x:dv_coll_shard3_replica_n4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=dv_coll_shard3_replica_n4&action=CREATE&numShards=4&collection=dv_coll&shard=shard3&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1619
   [junit4]   2> 1991565 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1991565 INFO  (qtp1657275198-20566) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=dv_coll_shard1_replica_n1&action=CREATE&numShards=4&collection=dv_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1620
   [junit4]   2> 1991565 INFO  (qtp1379579731-20579) [n:127.0.0.1:64947_solr c:dv_coll s:shard2 r:core_node5 x:dv_coll_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=dv_coll_shard2_replica_n2&action=CREATE&numShards=4&collection=dv_coll&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1622
   [junit4]   2> 1991570 INFO  (qtp1657275198-20565) [n:127.0.0.1:64946_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1991620 INFO  (OverseerCollectionConfigSetProcessor-98477725605298181-127.0.0.1:64947_solr-n_0000000000) [n:127.0.0.1:64947_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1991699 INFO  (zkCallback-3226-thread-1-processing-n:127.0.0.1:64947_solr) [n:127.0.0.1:64947_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1991699 INFO  (zkCallback-3226-thread-2-processing-n:127.0.0.1:64947_solr) [n:127.0.0.1:64947_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1991699 INFO  (zkCallback-3225-thread-2-processing-n:127.0.0.1:64946_solr) [n:127.0.0.1:64946_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1991699 INFO  (zkCallback-3225-thread-1-processing-n:127.0.0.1:64946_solr) [n:127.0.0.1:64946_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/dv_coll/state.json] for collection [dv_coll] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1992575 INFO  (qtp1657275198-20565) [n:127.0.0.1:64946_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&maxShardsPerNode=2&name=dv_coll&nrtReplicas=1&action=CREATE&numShards=4&wt=javabin&version=2} status=0 QTime=2961
   [junit4]   2> 1992587 INFO  (qtp1657275198-20569) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.s.SchemaManager managed schema loaded . version : 0
   [junit4]   2> 1992602 INFO  (qtp1657275198-20569) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.s.IndexSchema [dv_coll_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 1992616 INFO  (qtp1657275198-20569) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1992652 INFO  (qtp1657275198-20569) [n:127.0.0.1:64946_solr c:dv_coll s:shard1 r:core_node3 x:dv_coll_shard1_replica_n1] o.a.s.c.ZkController Persisted config data to node /configs/conf1/managed-schema
   [junit4]   2> 1992653 INFO  (zkCallback-3226-thread-2-processing-n:127.0.0.1:64947_solr) [n:127.0.0.1:64947_solr    ] o.a.s.s.ZkIndexSchemaReader A schema change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/configs/conf1/managed-schema, has occurred - updating schema from ZooKeeper ...
   [junit4]   2> 1992653 INFO  (zkCallback-3226-thread-1-processing-n:127.0.0.1:64947_solr) [n:127.0.0.1:64947_solr    ] o.a.s.s.ZkIndexSchemaReader A schema change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/configs/conf1/managed-schema, has occurred - updating schema from ZooKeeper ...
   [junit4]   2> 1992653 INFO  (zkCallback-3225-thread-1-processing-n:127.0.0.1:64946_solr) [n:127.0.0.1:64946_solr    ] o.a.s.s.ZkIndexSchemaReader A schema change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/configs/conf1/managed-schema, has occurred - updating schema from ZooKeeper ...
   [junit4]   2> 1992653 INFO  (zkCallback-3225-thread-2-processing-n:127.0.0.1:64946_solr) [n:127.0.0.1:64946_solr    ] o.a.s.s.ZkIndexSchemaReader A schema change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/configs/conf1/managed-schema, has occurred - updating schema from ZooKeeper ...
   [junit4]   2> 1992657 INFO  (zkCallback-3226-thread-2-processing-n:127.0.0.1:64947_solr) [n:127.0.0.1:64947_solr    ] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 1 from ZooKeeper
   [junit4]   2> 1992658 INFO  (zkCallback-3226-thread-1-processing-n:127.0.0.1:64947_solr) [n:127.0.0.1:64947_solr    ] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 1 from ZooKeeper
   [junit4]   2> 1992661 INFO  (zkCallback-3226-thread-2-processing-n:127.0.0.1:64947_solr) [n:127.0.0.1:64947_s

[...truncated too long message...]

MetricManager Closing metric reporters for registry=solr.collection.dv_coll.shard1.leader, tag=116345878
   [junit4]   2> 1993995 INFO  (jetty-closer-3215-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2a087f8e{/solr,null,UNAVAILABLE}
   [junit4]   2> 1993996 ERROR (SUITE-DocValuesNotIndexedTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1993997 INFO  (SUITE-DocValuesNotIndexedTest-seed#[723BDDF842FD1BE5]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:64942 64942
   [junit4]   2> 1993998 INFO  (Thread-5329) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:64942 64942
   [junit4]   2> 1993999 WARN  (Thread-5329) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 6 /solr/configs/conf1/managed-schema
   [junit4]   2> 4 /solr/configs/conf1
   [junit4]   2> 3 /solr/aliases.json
   [junit4]   2> 3 /solr/clusterprops.json
   [junit4]   2> 2 /solr/security.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 12 /solr/collections/dv_coll/state.json
   [junit4]   2> 3 /solr/clusterstate.json
   [junit4]   2> 2 /solr/configs/conf1/managed-schema
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/live_nodes
   [junit4]   2> 3 /solr/collections
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.DocValuesNotIndexedTest_723BDDF842FD1BE5-001
   [junit4]   2> Aug 13, 2017 7:00:47 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 3 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {_version_=PostingsFormat(name=LuceneVarGapFixedInterval), id=PostingsFormat(name=Memory)}, docValues:{floatField=DocValuesFormat(name=Lucene70), doubleField=DocValuesFormat(name=Memory), intFieldMulti=DocValuesFormat(name=Lucene70), boolFieldMulti=DocValuesFormat(name=Memory), longFieldMulti=DocValuesFormat(name=Lucene70), floatGSL=DocValuesFormat(name=Asserting), intField=DocValuesFormat(name=Memory), boolField=DocValuesFormat(name=Asserting), stringFieldMulti=DocValuesFormat(name=Asserting), dateFieldMulti=DocValuesFormat(name=Lucene70), doubleFieldMulti=DocValuesFormat(name=Asserting), longGSF=DocValuesFormat(name=Lucene70), stringGSL=DocValuesFormat(name=Lucene70), doubleGSF=DocValuesFormat(name=Memory), doubleGSL=DocValuesFormat(name=Lucene70), stringGSF=DocValuesFormat(name=Memory), intGSL=DocValuesFormat(name=Memory), longField=DocValuesFormat(name=Lucene70), dateField=DocValuesFormat(name=Asserting), dateGSF=DocValuesFormat(name=Asserting), boolGSF=DocValuesFormat(name=Asserting), longGSL=DocValuesFormat(name=Asserting), boolGSL=DocValuesFormat(name=Lucene70), floatFieldMulti=DocValuesFormat(name=Memory), floatGSF=DocValuesFormat(name=Lucene70), dateGSL=DocValuesFormat(name=Lucene70), intGSF=DocValuesFormat(name=Lucene70), stringField=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=531, maxMBSortInHeap=5.3471448074457095, sim=RandomSimilarity(queryNorm=true): {}, locale=bg, timezone=Asia/Jerusalem
   [junit4]   2> NOTE: Windows 10 10.0 amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=3,threads=1,free=253749512,total=491409408
   [junit4]   2> NOTE: All tests run in this JVM: [SimpleMLTQParserTest, HdfsUnloadDistributedZkTest, TestShortCircuitedRequests, TestDistributedMissingSort, SolrCloudReportersTest, ConjunctionSolrSpellCheckerTest, TestMultiWordSynonyms, OpenCloseCoreStressTest, TestReloadAndDeleteDocs, V2ApiIntegrationTest, TestExclusionRuleCollectionAccess, ConvertedLegacyTest, SolrIndexConfigTest, QueryEqualityTest, TestSimpleQParserPlugin, ReplicationFactorTest, TestInitParams, BasicZkTest, StressHdfsTest, MoreLikeThisHandlerTest, UpdateParamsTest, BigEndianAscendingWordDeserializerTest, OpenExchangeRatesOrgProviderTest, TestExpandComponent, SpatialRPTFieldTypeTest, SuggesterTSTTest, BasicDistributedZk2Test, DistributedIntervalFacetingTest, TestSolrCloudWithHadoopAuthPlugin, MergeStrategyTest, SaslZkACLProviderTest, SpatialHeatmapFacetsTest, AddSchemaFieldsUpdateProcessorFactoryTest, OverseerStatusTest, TestCollectionAPIs, ChaosMonkeySafeLeaderTest, TestOnReconnectListenerSupport, TestRecovery, TestObjectReleaseTracker, ReplicaListTransformerTest, TestRuleBasedAuthorizationPlugin, TestAnalyzeInfixSuggestions, TestRawTransformer, TestSolrFieldCacheBean, TestFiltering, PreAnalyzedFieldManagedSchemaCloudTest, StatsComponentTest, TriLevelCompositeIdRoutingTest, SolrCloudExampleTest, ChaosMonkeySafeLeaderWithPullReplicasTest, HLLUtilTest, FacetPivotSmallTest, HttpPartitionTest, TestMaxScoreQueryParser, TestCollectionsAPIViaSolrCloudCluster, CoreMergeIndexesAdminHandlerTest, UpdateRequestProcessorFactoryTest, ShardRoutingCustomTest, TestCoreDiscovery, TestManagedSchemaThreadSafety, ActionThrottleTest, DistributedFacetPivotWhiteBoxTest, LeaderFailoverAfterPartitionTest, TestInitQParser, TestDFISimilarityFactory, DistributedQueryElevationComponentTest, TestCloudManagedSchema, JSONWriterTest, TestRemoteStreaming, CoreAdminCreateDiscoverTest, TestScoreJoinQPNoScore, TestSchemaVersionResource, TestSolrCLIRunExample, DocumentBuilderTest, BadCopyFieldTest, CursorMarkTest, SystemInfoHandlerTest, TestExceedMaxTermLength, SuggestComponentTest, TolerantUpdateProcessorTest, TestManagedResource, SortSpecParsingTest, TestSolrXml, DateRangeFieldTest, TestMergePolicyConfig, TestPerFieldSimilarityWithDefaultOverride, TestPerFieldSimilarity, TestCollationFieldDocValues, TestUseDocValuesAsStored, DirectSolrSpellCheckerTest, NodeMutatorTest, CollectionsAPISolrJTest, TestLMJelinekMercerSimilarityFactory, TestRebalanceLeaders, TestHighFrequencyDictionaryFactory, TestManagedSynonymFilterFactory, BitVectorTest, TestLeaderInitiatedRecoveryThread, TestReloadDeadlock, TestTolerantUpdateProcessorRandomCloud, TestXmlQParser, SolrGangliaReporterTest, TestChildDocTransformer, SuggesterFSTTest, TestDelegationWithHadoopAuth, TestFieldSortValues, TestAuthenticationFramework, BlockDirectoryTest, TestSolrConfigHandlerConcurrent, SchemaVersionSpecificBehaviorTest, CleanupOldIndexTest, FileBasedSpellCheckerTest, TestManagedSchema, TestSystemIdResolver, TestCollationField, TestSubQueryTransformerCrossCore, TestFastLRUCache, TestFieldCacheReopen, TestCopyFieldCollectionResource, TestNumericTokenStream, CdcrBootstrapTest, TestIntervalFaceting, TestSchemaManager, PluginInfoTest, TestBackupRepositoryFactory, CacheHeaderTest, TestPartialUpdateDeduplication, PingRequestHandlerTest, TestSSLRandomization, TestCoreAdminApis, TestConfigReload, DistributedTermsComponentTest, DistribCursorPagingTest, TestCloudPivotFacet, TestReRankQParserPlugin, CollectionTooManyReplicasTest, HdfsDirectoryFactoryTest, TestWordDelimiterFilterFactory, LeaderInitiatedRecoveryOnCommitTest, DistribJoinFromCollectionTest, OverseerModifyCollectionTest, DistributedFacetPivotLargeTest, CopyFieldTest, CdcrVersionReplicationTest, RollingRestartTest, CdcrReplicationHandlerTest, TestCSVLoader, TestConfigSets, MetricsHandlerTest, TemplateUpdateProcessorTest, TestPushWriter, TestSolr4Spatial2, MBeansHandlerTest, TestConfigSetsAPIExclusivity, TestZkChroot, TestLRUStatsCache, TestConfigSetImmutable, TestStressInPlaceUpdates, DeleteShardTest, TestRandomCollapseQParserPlugin, TestJsonRequest, TestUniqueKeyFieldResource, LukeRequestHandlerTest, BasicFunctionalityTest, TestFieldTypeResource, TestOverriddenPrefixQueryForCustomFieldType, StatsReloadRaceTest, TestDocSet, TestCustomStream, TestSQLHandlerNonCloud, TestSolrJ, DistributedMLTComponentTest, TestSolr4Spatial, TestFileDictionaryLookup, EchoParamsTest, TestReplicationHandlerBackup, TestGraphTermsQParserPlugin, JvmMetricsTest, DOMUtilTest, SpellCheckCollatorTest, TestAuthorizationFramework, MetricUtilsTest, UnloadDistributedZkTest, LeaderElectionIntegrationTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest, ZkSolrClientTest, AutoCommitTest, TestFaceting, TermVectorComponentDistributedTest, TestJoin, TestStressVersions, TestRangeQuery, TestGroupingSearch, PeerSyncTest, TestBadConfig, TestIndexSearcher, SignatureUpdateProcessorFactoryTest, SolrRequestParserTest, TestStressLucene, TestTrie, WordBreakSolrSpellCheckerTest, FieldMutatingUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, TestSolrDeletionPolicy1, XsltUpdateRequestHandlerTest, IndexBasedSpellCheckerTest, TestWriterPerf, TestOmitPositions, TermsComponentTest, RequestHandlersTest, TermVectorComponentTest, RequiredFieldsTest, TestSolrQueryParser, LoggingHandlerTest, SolrPluginUtilsTest, ReturnFieldsTest, JsonLoaderTest, SearchHandlerTest, TestSearchPerf, TestElisionMultitermQuery, OutputWriterTest, TestPostingsSolrHighlighter, TestLuceneMatchVersion, SpellPossibilityIteratorTest, TestCharFilters, TestCodecSupport, TestDFRSimilarityFactory, TestIBSimilarityFactory, TestFastWriter, TestFastOutputStream, PreAnalyzedFieldTest, DateFieldTest, TestUtils, ZkNodePropsTest, FileUtilsTest, SolrTestCaseJ4Test, TestHighlightDedupGrouping, TestEmbeddedSolrServerConstructors, TestEmbeddedSolrServerSchemaAPI, TestJettySolrRunner, ConnectionReuseTest, AsyncCallRequestStatusResponseTest, CdcrRequestHandlerTest, CloudExitableDirectoryReaderTest, CollectionReloadTest, CollectionStateFormat2Test, CollectionsAPIAsyncDistributedZkTest, DeleteInactiveReplicaTest, DeleteLastCustomShardedReplicaTest, DeleteNodeTest, DistribDocExpirationUpdateProcessorTest, DocValuesNotIndexedTest]
   [junit4] Completed [521/729 (2!)] on J1 in 5.92s, 4 tests, 1 failure <<< FAILURES!

[...truncated 44003 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS-EA] Lucene-Solr-master-Windows (64bit/jdk-9-ea+181) - Build # 6821 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6821/
Java: 64bit/jdk-9-ea+181 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC --illegal-access=deny

2 tests failed.
FAILED:  org.apache.solr.cloud.TestAuthenticationFramework.testBasics

Error Message:
Error from server at http://127.0.0.1:60862/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 </title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason: <pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/> </body> </html>

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:60862/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
</body>
</html>

        at __randomizedtesting.SeedInfo.seed([6E0B4B59FBAA5993:53D3E575C34407E3]:0)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:539)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:993)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
        at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
        at org.apache.solr.cloud.TestAuthenticationFramework.collectionCreateSearchDeleteTwice(TestAuthenticationFramework.java:126)
        at org.apache.solr.cloud.TestAuthenticationFramework.testBasics(TestAuthenticationFramework.java:74)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:60862/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
</body>
</html>

        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:591)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:516)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        ... 1 more


FAILED:  junit.framework.TestSuite.org.apache.solr.client.solrj.SolrExceptionTest

Error Message:
The test or suite printed 10352 bytes to stdout and stderr, even though the limit was set to 8192 bytes. Increase the limit with @Limit, ignore it completely with @SuppressSysoutChecks or run with -Dtests.verbose=true

Stack Trace:
java.lang.AssertionError: The test or suite printed 10352 bytes to stdout and stderr, even though the limit was set to 8192 bytes. Increase the limit with @Limit, ignore it completely with @SuppressSysoutChecks or run with -Dtests.verbose=true
        at __randomizedtesting.SeedInfo.seed([2F1A85A3BAE96670]:0)
        at org.apache.lucene.util.TestRuleLimitSysouts.afterIfSuccessful(TestRuleLimitSysouts.java:211)
        at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterIfSuccessful(TestRuleAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:37)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 1716 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\temp\junit4-J1-20170814_021038_7104292229564139069153.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\temp\junit4-J0-20170814_021038_7134879879969604959486.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 304 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\test-framework\test\temp\junit4-J1-20170814_021539_09215476035976913177239.syserr
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\test-framework\test\temp\junit4-J0-20170814_021539_09215288740287526434713.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 1037 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\analysis\common\test\temp\junit4-J1-20170814_021635_5418903629254942519833.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\analysis\common\test\temp\junit4-J0-20170814_021635_54111944258541979883626.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 220 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\analysis\icu\test\temp\junit4-J1-20170814_021841_668432873514478007445.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\analysis\icu\test\temp\junit4-J0-20170814_021841_6685874948682379890931.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 238 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\analysis\kuromoji\test\temp\junit4-J0-20170814_021853_41511006772407907749832.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\analysis\kuromoji\test\temp\junit4-J1-20170814_021853_4159855660853514197813.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 147 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\analysis\morfologik\test\temp\junit4-J0-20170814_021920_6338772710864396173596.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\analysis\morfologik\test\temp\junit4-J1-20170814_021920_6333611477943139087780.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 159 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\analysis\phonetic\test\temp\junit4-J1-20170814_021924_50111716674036314046614.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\analysis\phonetic\test\temp\junit4-J0-20170814_021924_5015450210987259238437.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 144 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\analysis\smartcn\test\temp\junit4-J1-20170814_021934_88468939911612261173.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\analysis\smartcn\test\temp\junit4-J0-20170814_021934_88418336228863333776502.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 149 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\analysis\stempel\test\temp\junit4-J1-20170814_021942_2135776122759531463018.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\analysis\stempel\test\temp\junit4-J0-20170814_021942_213604858563713868906.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 160 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\analysis\uima\test\temp\junit4-J0-20170814_021945_59714409102967117889137.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 145 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\backward-codecs\test\temp\junit4-J1-20170814_022002_88510998710904234228219.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 25 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\backward-codecs\test\temp\junit4-J0-20170814_022002_88513109673456808855964.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 1220 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\benchmark\test\temp\junit4-J0-20170814_022015_5898399669492612487101.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\benchmark\test\temp\junit4-J1-20170814_022015_58711462843951488440272.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 219 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\classification\test\temp\junit4-J1-20170814_022025_9558539858314692119895.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 6 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\classification\test\temp\junit4-J0-20170814_022025_955779210561321201489.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 252 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\codecs\test\temp\junit4-J0-20170814_022042_84815030355988694470607.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\codecs\test\temp\junit4-J1-20170814_022042_84818120719375832360235.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 226 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\demo\test\temp\junit4-J0-20170814_022218_58417125024962492650746.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\demo\test\temp\junit4-J1-20170814_022218_5849097344387740550272.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 166 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\expressions\test\temp\junit4-J0-20170814_022223_5971292863026253851353.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\expressions\test\temp\junit4-J1-20170814_022223_5996136700157280681314.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 211 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\facet\test\temp\junit4-J0-20170814_022230_29717093286965753990050.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\facet\test\temp\junit4-J1-20170814_022230_297290203677744739657.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 172 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\grouping\test\temp\junit4-J0-20170814_022301_3117594398724643346416.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\grouping\test\temp\junit4-J1-20170814_022301_3116600454436495940987.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 236 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\highlighter\test\temp\junit4-J1-20170814_022312_833965472068277254675.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\highlighter\test\temp\junit4-J0-20170814_022312_83316132085977129014295.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 153 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\join\test\temp\junit4-J1-20170814_022330_184555375787896006566.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 6 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\join\test\temp\junit4-J0-20170814_022330_1844230797827254472033.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 140 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\memory\test\temp\junit4-J0-20170814_022345_74615937872511350751695.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\memory\test\temp\junit4-J1-20170814_022345_7487023295042035135689.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 174 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\misc\test\temp\junit4-J0-20170814_022353_59015870399746791771880.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 6 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\misc\test\temp\junit4-J1-20170814_022353_5901641189021943038876.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 305 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\queries\test\temp\junit4-J1-20170814_022414_3246864015236729818626.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\queries\test\temp\junit4-J0-20170814_022414_3248499128973686390452.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 217 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\queryparser\test\temp\junit4-J0-20170814_022425_9429044765619687306254.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\queryparser\test\temp\junit4-J1-20170814_022425_94218114743996859448273.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 197 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\replicator\test\temp\junit4-J1-20170814_022436_6488459410867713866906.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 6 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\replicator\test\temp\junit4-J0-20170814_022436_6483295859987720354909.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 198 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\sandbox\test\temp\junit4-J0-20170814_022456_5216003652067953715230.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 7 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\sandbox\test\temp\junit4-J1-20170814_022456_5219658299962697221791.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 211 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\spatial-extras\test\temp\junit4-J1-20170814_022524_3079392807097566434981.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\spatial-extras\test\temp\junit4-J0-20170814_022524_30711399252397215569943.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 172 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\spatial3d\test\temp\junit4-J1-20170814_022539_5353575686002710336793.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 5 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\spatial3d\test\temp\junit4-J0-20170814_022539_53513804524839260083442.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 139 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\spatial\test\temp\junit4-J0-20170814_022559_01415132119324842755779.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 237 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\suggest\test\temp\junit4-J0-20170814_022602_8746106610764691218550.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\suggest\test\temp\junit4-J1-20170814_022602_87410363812398841457814.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3175 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestAuthenticationFramework
   [junit4]   2> 2246243 INFO  (SUITE-TestAuthenticationFramework-seed#[6E0B4B59FBAA5993]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestAuthenticationFramework_6E0B4B59FBAA5993-001\init-core-data-001
   [junit4]   2> 2246249 INFO  (SUITE-TestAuthenticationFramework-seed#[6E0B4B59FBAA5993]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 2246251 INFO  (SUITE-TestAuthenticationFramework-seed#[6E0B4B59FBAA5993]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0)
   [junit4]   2> 2246270 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[6E0B4B59FBAA5993]) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestAuthenticationFramework_6E0B4B59FBAA5993-001\tempDir-001
   [junit4]   2> 2246271 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[6E0B4B59FBAA5993]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2246271 INFO  (Thread-3676) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2246271 INFO  (Thread-3676) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2246289 ERROR (Thread-3676) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2246374 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[6E0B4B59FBAA5993]) [    ] o.a.s.c.ZkTestServer start zk server on port:60858
   [junit4]   2> 2246537 INFO  (jetty-launcher-3062-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2246537 INFO  (jetty-launcher-3062-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@53edc915{/solr,null,AVAILABLE}
   [junit4]   2> 2246537 INFO  (jetty-launcher-3062-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 2246537 INFO  (jetty-launcher-3062-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@725e9749{/solr,null,AVAILABLE}
   [junit4]   2> 2246537 INFO  (jetty-launcher-3062-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@6d335019{HTTP/1.1,[http/1.1]}{127.0.0.1:60863}
   [junit4]   2> 2246537 INFO  (jetty-launcher-3062-thread-2) [    ] o.e.j.s.Server Started @2248390ms
   [junit4]   2> 2246537 INFO  (jetty-launcher-3062-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=60863}
   [junit4]   2> 2246537 ERROR (jetty-launcher-3062-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2246537 INFO  (jetty-launcher-3062-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 2246537 INFO  (jetty-launcher-3062-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2246537 INFO  (jetty-launcher-3062-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 2246537 INFO  (jetty-launcher-3062-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-14T03:04:55.441926500Z
   [junit4]   2> 2246537 INFO  (jetty-launcher-3062-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@6b0c6beb{HTTP/1.1,[http/1.1]}{127.0.0.1:60862}
   [junit4]   2> 2246537 INFO  (jetty-launcher-3062-thread-1) [    ] o.e.j.s.Server Started @2248393ms
   [junit4]   2> 2246537 INFO  (jetty-launcher-3062-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=60862}
   [junit4]   2> 2246537 ERROR (jetty-launcher-3062-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2246649 INFO  (jetty-launcher-3062-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 2246649 INFO  (jetty-launcher-3062-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2246649 INFO  (jetty-launcher-3062-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 2246649 INFO  (jetty-launcher-3062-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-14T03:04:55.553481400Z
   [junit4]   2> 2246663 INFO  (jetty-launcher-3062-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2246663 INFO  (jetty-launcher-3062-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2246690 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15ddeb38c990001, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 2246692 INFO  (jetty-launcher-3062-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60858/solr
   [junit4]   2> 2246692 INFO  (jetty-launcher-3062-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60858/solr
   [junit4]   2> 2248196 INFO  (jetty-launcher-3062-thread-2) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2248196 INFO  (jetty-launcher-3062-thread-2) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:60863_solr
   [junit4]   2> 2248196 INFO  (jetty-launcher-3062-thread-2) [    ] o.a.s.c.Overseer Overseer (id=98479629611368454-127.0.0.1:60863_solr-n_0000000000) starting
   [junit4]   2> 2248280 INFO  (jetty-launcher-3062-thread-2) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60863_solr
   [junit4]   2> 2248342 INFO  (jetty-launcher-3062-thread-2) [    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin
   [junit4]   2> 2248369 INFO  (zkCallback-3074-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2248525 INFO  (jetty-launcher-3062-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2248542 INFO  (jetty-launcher-3062-thread-1) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2248554 INFO  (jetty-launcher-3062-thread-1) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60862_solr
   [junit4]   2> 2248556 INFO  (zkCallback-3074-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2248604 INFO  (zkCallback-3073-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2248640 INFO  (jetty-launcher-3062-thread-2) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60863.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@10160e09
   [junit4]   2> 2248640 INFO  (jetty-launcher-3062-thread-2) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60863.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@10160e09
   [junit4]   2> 2248640 INFO  (jetty-launcher-3062-thread-2) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60863.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@10160e09
   [junit4]   2> 2248669 INFO  (jetty-launcher-3062-thread-2) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestAuthenticationFramework_6E0B4B59FBAA5993-001\tempDir-001\node2\.
   [junit4]   2> 2248748 INFO  (jetty-launcher-3062-thread-1) [    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.cloud.TestAuthenticationFramework$MockAuthenticationPlugin
   [junit4]   2> 2248886 INFO  (jetty-launcher-3062-thread-1) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60862.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@10160e09
   [junit4]   2> 2248890 INFO  (jetty-launcher-3062-thread-1) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60862.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@10160e09
   [junit4]   2> 2248890 INFO  (jetty-launcher-3062-thread-1) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60862.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@10160e09
   [junit4]   2> 2248891 INFO  (jetty-launcher-3062-thread-1) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestAuthenticationFramework_6E0B4B59FBAA5993-001\tempDir-001\node1\.
   [junit4]   2> 2248978 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[6E0B4B59FBAA5993]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2248979 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[6E0B4B59FBAA5993]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:60858/solr ready
   [junit4]   2> 2248987 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[6E0B4B59FBAA5993]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testBasics
   [junit4]   2> 2249005 INFO  (qtp132410550-20676) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=279077d4-ce89-4730-9d99-b7ed3be8fd2e&replicationFactor=2&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=2&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2249025 INFO  (qtp132410550-20676) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=279077d4-ce89-4730-9d99-b7ed3be8fd2e&replicationFactor=2&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=2&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=20
   [junit4]   2> 2249026 INFO  (qtp132410550-20676) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=279077d4-ce89-4730-9d99-b7ed3be8fd2e&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2249029 INFO  (qtp132410550-20676) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=279077d4-ce89-4730-9d99-b7ed3be8fd2e&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 2249041 INFO  (OverseerThreadFactory-6585-thread-1) [    ] o.a.s.c.CreateCollectionCmd Create collection testcollection
   [junit4]   2> 2249170 INFO  (OverseerStateUpdate-98479629611368454-127.0.0.1:60863_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testcollection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:60863/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 2249197 INFO  (OverseerStateUpdate-98479629611368454-127.0.0.1:60863_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"testcollection_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:60862/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 2249211 INFO  (OverseerStateUpdate-98479629611368454-127.0.0.1:60863_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testcollection_shard2_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:60863/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 2249261 INFO  (OverseerStateUpdate-98479629611368454-127.0.0.1:60863_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"testcollection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"testcollection_shard2_replica_n6",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:60862/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 2249445 INFO  (qtp1887918083-20671) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=279077d4-ce89-4730-9d99-b7ed3be8fd2e394918883739558&coreNodeName=core_node5&name=testcollection_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=0
   [junit4]   2> 2249451 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=279077d4-ce89-4730-9d99-b7ed3be8fd2e394918883739558&coreNodeName=core_node5&name=testcollection_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 2249451 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2249452 INFO  (qtp132410550-20675) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=279077d4-ce89-4730-9d99-b7ed3be8fd2e394918868528033&coreNodeName=core_node3&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=0
   [junit4]   2> 2249452 INFO  (qtp132410550-20675) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=279077d4-ce89-4730-9d99-b7ed3be8fd2e394918895559650&coreNodeName=core_node7&name=testcollection_shard2_replica_n4&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=0
   [junit4]   2> 2249491 INFO  (parallelCoreAdminExecutor-6586-thread-2) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=279077d4-ce89-4730-9d99-b7ed3be8fd2e394918895559650&coreNodeName=core_node7&name=testcollection_shard2_replica_n4&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 2249491 INFO  (parallelCoreAdminExecutor-6586-thread-2) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2249492 INFO  (parallelCoreAdminExecutor-6586-thread-1) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=279077d4-ce89-4730-9d99-b7ed3be8fd2e394918868528033&coreNodeName=core_node3&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 2249497 INFO  (qtp1887918083-20669) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=279077d4-ce89-4730-9d99-b7ed3be8fd2e394918900450958&coreNodeName=core_node8&name=testcollection_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=0
   [junit4]   2> 2249501 INFO  (qtp132410550-20675) [    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 279077d4-ce89-4730-9d99-b7ed3be8fd2e394918895559650
   [junit4]   2> 2249502 INFO  (qtp132410550-20675) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=279077d4-ce89-4730-9d99-b7ed3be8fd2e394918895559650&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2249505 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&async=279077d4-ce89-4730-9d99-b7ed3be8fd2e394918900450958&coreNodeName=core_node8&name=testcollection_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 2249667 INFO  (zkCallback-3073-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2249667 INFO  (zkCallback-3074-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2249669 INFO  (zkCallback-3073-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2249669 INFO  (zkCallback-3074-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2250033 INFO  (qtp132410550-20675) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=279077d4-ce89-4730-9d99-b7ed3be8fd2e&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2250034 INFO  (qtp132410550-20675) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=279077d4-ce89-4730-9d99-b7ed3be8fd2e&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 2250503 INFO  (qtp132410550-20678) [    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 279077d4-ce89-4730-9d99-b7ed3be8fd2e394918895559650
   [junit4]   2> 2250503 INFO  (qtp132410550-20678) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=279077d4-ce89-4730-9d99-b7ed3be8fd2e394918895559650&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2250526 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2250600 INFO  (parallelCoreAdminExecutor-6586-thread-2) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2250605 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.s.IndexSchema [testcollection_shard1_replica_n2] Schema name=minimal
   [junit4]   2> 2250605 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2250605 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard1_replica_n2' using configuration from collection testcollection, trusted=true
   [junit4]   2> 2250605 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60862.solr.core.testcollection.shard1.replica_n2' (registry 'solr.core.testcollection.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@10160e09
   [junit4]   2> 2250609 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2250609 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.c.SolrCore [[testcollection_shard1_replica_n2] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestAuthenticationFramework_6E0B4B59FBAA5993-001\tempDir-001\node1\testcollection_shard1_replica_n2], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestAuthenticationFramework_6E0B4B59FBAA5993-001\tempDir-001\node1\.\testcollection_shard1_replica_n2\data\]
   [junit4]   2> 2250614 INFO  (parallelCoreAdminExecutor-6586-thread-2) [    ] o.a.s.s.IndexSchema [testcollection_shard2_replica_n4] Schema name=minimal
   [junit4]   2> 2250616 INFO  (parallelCoreAdminExecutor-6586-thread-2) [    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2250616 INFO  (parallelCoreAdminExecutor-6586-thread-2) [    ] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard2_replica_n4' using configuration from collection testcollection, trusted=true
   [junit4]   2> 2250625 INFO  (parallelCoreAdminExecutor-6586-thread-2) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60863.solr.core.testcollection.shard2.replica_n4' (registry 'solr.core.testcollection.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@10160e09
   [junit4]   2> 2250625 INFO  (parallelCoreAdminExecutor-6586-thread-2) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2250626 INFO  (parallelCoreAdminExecutor-6586-thread-2) [    ] o.a.s.c.SolrCore [[testcollection_shard2_replica_n4] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestAuthenticationFramework_6E0B4B59FBAA5993-001\tempDir-001\node2\testcollection_shard2_replica_n4], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestAuthenticationFramework_6E0B4B59FBAA5993-001\tempDir-001\node2\.\testcollection_shard2_replica_n4\data\]
   [junit4]   2> 2250629 INFO  (parallelCoreAdminExecutor-6586-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2250635 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2250667 INFO  (parallelCoreAdminExecutor-6586-thread-1) [    ] o.a.s.s.IndexSchema [testcollection_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 2250670 INFO  (parallelCoreAdminExecutor-6586-thread-1) [    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2250670 INFO  (parallelCoreAdminExecutor-6586-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard1_replica_n1' using configuration from collection testcollection, trusted=true
   [junit4]   2> 2250670 INFO  (parallelCoreAdminExecutor-6586-thread-1) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60863.solr.core.testcollection.shard1.replica_n1' (registry 'solr.core.testcollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@10160e09
   [junit4]   2> 2250671 INFO  (parallelCoreAdminExecutor-6586-thread-1) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2250671 INFO  (parallelCoreAdminExecutor-6586-thread-1) [    ] o.a.s.c.SolrCore [[testcollection_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestAuthenticationFramework_6E0B4B59FBAA5993-001\tempDir-001\node2\testcollection_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestAuthenticationFramework_6E0B4B59FBAA5993-001\tempDir-001\node2\.\testcollection_shard1_replica_n1\data\]
   [junit4]   2> 2250679 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.s.IndexSchema [testcollection_shard2_replica_n6] Schema name=minimal
   [junit4]   2> 2250682 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2250682 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.c.CoreContainer Creating SolrCore 'testcollection_shard2_replica_n6' using configuration from collection testcollection, trusted=true
   [junit4]   2> 2250682 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60862.solr.core.testcollection.shard2.replica_n6' (registry 'solr.core.testcollection.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@10160e09
   [junit4]   2> 2250682 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2250683 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.c.SolrCore [[testcollection_shard2_replica_n6] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestAuthenticationFramework_6E0B4B59FBAA5993-001\tempDir-001\node1\testcollection_shard2_replica_n6], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestAuthenticationFramework_6E0B4B59FBAA5993-001\tempDir-001\node1\.\testcollection_shard2_replica_n6\data\]
   [junit4]   2> 2250760 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2250760 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2250763 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2250763 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2250852 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@4b323619[testcollection_shard1_replica_n2] main]
   [junit4]   2> 2250878 INFO  (parallelCoreAdminExecutor-6586-thread-1) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2250878 INFO  (parallelCoreAdminExecutor-6586-thread-1) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2250884 INFO  (parallelCoreAdminExecutor-6586-thread-1) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2250884 INFO  (parallelCoreAdminExecutor-6586-thread-1) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2250889 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2250889 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2250891 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2250891 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2250897 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 2250898 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 2250899 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2250904 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575674078620024832
   [junit4]   2> 2250906 INFO  (searcherExecutor-6590-thread-1) [    ] o.a.s.c.SolrCore [testcollection_shard1_replica_n2] Registered new searcher Searcher@4b323619[testcollection_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2250912 INFO  (parallelCoreAdminExecutor-6586-thread-2) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2250912 INFO  (parallelCoreAdminExecutor-6586-thread-2) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2250916 INFO  (parallelCoreAdminExecutor-6586-thread-2) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2250916 INFO  (parallelCoreAdminExecutor-6586-thread-2) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2250917 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 2250917 INFO  (parallelCoreAdminExecutor-6586-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@498b0a11[testcollection_shard1_replica_n1] main]
   [junit4]   2> 2250917 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@261bb24[testcollection_shard2_replica_n6] main]
   [junit4]   2> 2250917 INFO  (parallelCoreAdminExecutor-6586-thread-2) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@5c4a38d6[testcollection_shard2_replica_n4] main]
   [junit4]   2> 2250927 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 2250928 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 2250929 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2250932 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575674078650433536
   [junit4]   2> 2250934 INFO  (searcherExecutor-6593-thread-1) [    ] o.a.s.c.SolrCore [testcollection_shard2_replica_n6] Registered new searcher Searcher@261bb24[testcollection_shard2_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2250942 INFO  (parallelCoreAdminExecutor-6586-thread-2) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 2250942 INFO  (parallelCoreAdminExecutor-6586-thread-1) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
   [junit4]   2> 2250946 INFO  (parallelCoreAdminExecutor-6586-thread-2) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 2250946 INFO  (parallelCoreAdminExecutor-6586-thread-2) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2250946 INFO  (parallelCoreAdminExecutor-6586-thread-2) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575674078665113600
   [junit4]   2> 2250946 INFO  (parallelCoreAdminExecutor-6586-thread-1) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
   [junit4]   2> 2250946 INFO  (parallelCoreAdminExecutor-6586-thread-1) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2250946 INFO  (parallelCoreAdminExecutor-6586-thread-1) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575674078665113600
   [junit4]   2> 2250946 INFO  (searcherExecutor-6591-thread-1) [    ] o.a.s.c.SolrCore [testcollection_shard2_replica_n4] Registered new searcher Searcher@5c4a38d6[testcollection_shard2_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2250946 INFO  (searcherExecutor-6592-thread-1) [    ] o.a.s.c.SolrCore [testcollection_shard1_replica_n1] Registered new searcher Searcher@498b0a11[testcollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2250982 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2250982 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2250982 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:60862/solr/testcollection_shard2_replica_n6/
   [junit4]   2> 2250983 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.u.PeerSync PeerSync: core=testcollection_shard2_replica_n6 url=http://127.0.0.1:60862/solr START replicas=[http://127.0.0.1:60863/solr/testcollection_shard2_replica_n4/] nUpdates=100
   [junit4]   2> 2251009 INFO  (qtp132410550-20678) [    ] o.a.s.c.S.Request [testcollection_shard2_replica_n4]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=7
   [junit4]   2> 2251037 INFO  (qtp132410550-20686) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=279077d4-ce89-4730-9d99-b7ed3be8fd2e&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2251038 INFO  (qtp132410550-20686) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=279077d4-ce89-4730-9d99-b7ed3be8fd2e&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 2251086 INFO  (zkCallback-3074-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251086 INFO  (zkCallback-3074-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251087 INFO  (zkCallback-3073-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251087 INFO  (zkCallback-3073-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251285 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.u.PeerSync PeerSync: core=testcollection_shard2_replica_n6 url=http://127.0.0.1:60862/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 2251285 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 2251285 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 2251285 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2251290 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:60862/solr/testcollection_shard2_replica_n6/ shard2
   [junit4]   2> 2251394 INFO  (zkCallback-3074-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251394 INFO  (zkCallback-3074-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251395 INFO  (zkCallback-3073-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251395 INFO  (zkCallback-3073-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251417 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2251417 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2251417 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:60862/solr/testcollection_shard1_replica_n2/
   [junit4]   2> 2251417 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.u.PeerSync PeerSync: core=testcollection_shard1_replica_n2 url=http://127.0.0.1:60862/solr START replicas=[http://127.0.0.1:60863/solr/testcollection_shard1_replica_n1/] nUpdates=100
   [junit4]   2> 2251432 INFO  (qtp132410550-20688) [    ] o.a.s.c.S.Request [testcollection_shard1_replica_n1]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=7
   [junit4]   2> 2251444 INFO  (parallelCoreAdminExecutor-6588-thread-2) [    ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2251530 INFO  (qtp132410550-20680) [    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 279077d4-ce89-4730-9d99-b7ed3be8fd2e394918895559650
   [junit4]   2> 2251530 INFO  (qtp132410550-20680) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=279077d4-ce89-4730-9d99-b7ed3be8fd2e394918895559650&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2251552 INFO  (zkCallback-3074-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251553 INFO  (zkCallback-3074-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251553 INFO  (zkCallback-3073-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251553 INFO  (zkCallback-3073-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251726 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.u.PeerSync PeerSync: core=testcollection_shard1_replica_n2 url=http://127.0.0.1:60862/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 2251726 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 2251726 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 2251726 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2251734 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:60862/solr/testcollection_shard1_replica_n2/ shard1
   [junit4]   2> 2251838 INFO  (zkCallback-3074-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251838 INFO  (zkCallback-3074-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251838 INFO  (zkCallback-3073-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251839 INFO  (zkCallback-3073-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251841 INFO  (parallelCoreAdminExecutor-6588-thread-1) [    ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2251960 INFO  (zkCallback-3073-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251960 INFO  (zkCallback-3073-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251961 INFO  (zkCallback-3074-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251961 INFO  (zkCallback-3074-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2252042 INFO  (qtp132410550-20680) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=279077d4-ce89-4730-9d99-b7ed3be8fd2e&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2252044 INFO  (qtp132410550-20680) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=279077d4-ce89-4730-9d99-b7ed3be8fd2e&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 2252094 INFO  (zkCallback-3074-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2252094 INFO  (zkCallback-3074-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2252095 INFO  (zkCallback-3073-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2252095 INFO  (zkCallback-3073-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2252567 INFO  (qtp132410550-20674) [    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 279077d4-ce89-4730-9d99-b7ed3be8fd2e394918895559650
   [junit4]   2> 2252567 INFO  (qtp132410550-20674) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=279077d4-ce89-4730-9d99-b7ed3be8fd2e394918895559650&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2252596 INFO  (qtp1887918083-20669) [    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 279077d4-ce89-4730-9d99-b7ed3be8fd2e394918900450958
   [junit4]   2> 2252596 INFO  (qtp1887918083-20669) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=279077d4-ce89-4730-9d99-b7ed3be8fd2e394918900450958&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2252603 INFO  (OverseerThreadFactory-6585-thread-1) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2253045 INFO  (qtp132410550-20674) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=279077d4-ce89-4730-9d99-b7ed3be8fd2e&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2253046 INFO  (qtp132410550-20674) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=279077d4-ce89-4730-9d99-b7ed3be8fd2e&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 2253047 INFO  (qtp132410550-20675) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=279077d4-ce89-4730-9d99-b7ed3be8fd2e&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2253048 INFO  (qtp132410550-20675) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=279077d4-ce89-4730-9d99-b7ed3be8fd2e&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2253048 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[6E0B4B59FBAA5993]) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: testcollection failOnTimeout:true timeout (sec):330
   [junit4]   1> -
   [junit4]   1> replica:core_node3 rstate:active live:true
   [junit4]   1> replica:core_node5 rstate:active live:true
   [junit4]   1> replica:core_node7 rstate:active live:true
   [junit4]   1> replica:core_node8 rstate:active live:true
   [junit4]   1> no one is recoverying
   [junit4]   2> 2253049 INFO  (TEST-TestAuthenticationFramework.testBasics-seed#[6E0B4B59FBAA5993]) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: testcollection
   [junit4]   2> 2253052 INFO  (qtp132410550-20687) [    ] o.a.s.c.S.Request [testcollection_shard2_replica_n4]  webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testcollection:8&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=http://127.0.0.1:60863/solr/testcollection_shard2_replica_n4/|http://127.0.0.1:60862/solr/testcollection_shard2_replica_n6/&rows=10&version=2&q=*:*&NOW=1502679901954&isShard=true&wt=javabin} hits=0 status=0 QTime=0
   [junit4]   2> 2253056 INFO  (OverseerCollectionConfigSetProcessor-98479629611368454-127.0.0.1:60863_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2253060 INFO  (qtp1887918083-20666) [    ] o.a.s.c.S.Request [testcollection_shard1_replica_n2]  webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=testcollection:8&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=http://127.0.0.1:60862/solr/testcollection_shard1_replica_n2/|http://127.0.0.1:60863/solr/testcollection_shard1_replica_n1/&rows=10&version=2&q=*:*&NOW=1502679901954&isShard=true&wt=javabin} hits=0 status=0 QTime=0
   [junit4]   2> 2253061 INFO  (qtp1887918083-20667) [    ] o.a.s.c.S.Request [testcollection_shard1_replica_n2]  webapp=/solr path=/select params={q=*:*&_stateVer_=testcollection:8&wt=javabin&version=2} hits=0 status=0 QTime=10
   [junit4]   2> 2253078 INFO  (qtp132410550-20687) [    ] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:60862/solr/testcollection_shard1_replica_n2/&wt=javabin&version=2}{add=[1 (1575674080883900416)]} 0 12
   [junit4]   2> 2253078 INFO  (qtp1887918083-20666) [    ] o.a.s.u.p.LogUpdateProcessorFactory [testcollection_shard1_replica_n2]  webapp=/solr path=/update params={_stateVer_=testcollection:8&wt=javabin&version=2}{add=[1 (1575674080883900416)]} 0 16
   [junit4]   2> 2253080 INFO  (qtp132410550-20686) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1575674080902774784,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2253081 INFO  (qtp132410550-20686) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@665e83ab commitCommandVersion:1575674080902774784
   [junit4]   2> 2253083 INFO  (qtp1887918083-20668) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1575674080905920512,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2253083 INFO  (qtp1887918083-20668) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@474e2808 commitCommandVersion:1575674080905920512
   [junit4]   2> 2253085 INFO  (qtp132410550-20688) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1575674080908017664,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2253085 INFO  (qtp132410550-20688) [    ] o.a.s.u.DirectUpdateHandler2 No uncommitted changes.

[...truncated too long message...]

a HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 21 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-solrj\test\temp\junit4-J1-20170814_032315_77213045793425063851476.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 1243 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-analysis-extras\test\temp\junit4-J1-20170814_032724_6176716400318901579496.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-analysis-extras\test\temp\junit4-J0-20170814_032724_6171374979136766294126.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 548 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-analytics\test\temp\junit4-J0-20170814_032753_24812424713378240569384.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-analytics\test\temp\junit4-J1-20170814_032753_248391509270669835381.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 500 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-clustering\test\temp\junit4-J1-20170814_032855_09612009046018467469528.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-clustering\test\temp\junit4-J0-20170814_032855_09610700194625179621605.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 1130 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-dataimporthandler-extras\test\temp\junit4-J1-20170814_032934_1689673047144999316576.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-dataimporthandler-extras\test\temp\junit4-J0-20170814_032934_1683233707141507278643.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 593 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-dataimporthandler\test\temp\junit4-J1-20170814_032953_75610812157105389716829.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-dataimporthandler\test\temp\junit4-J0-20170814_032953_7569968236565701299867.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 501 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-cell\test\temp\junit4-J1-20170814_033045_2975512088647937926129.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 6 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-cell\test\temp\junit4-J0-20170814_033045_2974273361909553782403.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 506 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-langid\test\temp\junit4-J0-20170814_033110_705536983780677471784.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-langid\test\temp\junit4-J1-20170814_033110_7056506875894524747339.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 595 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-ltr\test\temp\junit4-J0-20170814_033133_02114936524606739575956.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-ltr\test\temp\junit4-J1-20170814_033133_02114485596261355618515.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 538 lines...]
   [junit4] JVM J1: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-uima\test\temp\junit4-J1-20170814_033226_9481345498419179713218.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-uima\test\temp\junit4-J0-20170814_033226_9485872799122180437572.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 495 lines...]
   [junit4] JVM J0: stderr was not empty, see: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\contrib\solr-velocity\test\temp\junit4-J0-20170814_033254_3073261508167017166503.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 28963 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS-EA] Lucene-Solr-master-Windows (32bit/jdk-9-ea+181) - Build # 6822 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6822/
Java: 32bit/jdk-9-ea+181 -client -XX:+UseSerialGC --illegal-access=deny

1 tests failed.
FAILED:  org.apache.solr.cloud.CloudExitableDirectoryReaderTest.test

Error Message:
partialResults were expected expected:<true> but was:<null>

Stack Trace:
java.lang.AssertionError: partialResults were expected expected:<true> but was:<null>
        at __randomizedtesting.SeedInfo.seed([2281E7508B9A74DA:AAD5D88A25661922]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.apache.solr.cloud.CloudExitableDirectoryReaderTest.assertPartialResults(CloudExitableDirectoryReaderTest.java:107)
        at org.apache.solr.cloud.CloudExitableDirectoryReaderTest.doTimeoutTests(CloudExitableDirectoryReaderTest.java:90)
        at org.apache.solr.cloud.CloudExitableDirectoryReaderTest.test(CloudExitableDirectoryReaderTest.java:56)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 12343 lines...]
   [junit4] Suite: org.apache.solr.cloud.CloudExitableDirectoryReaderTest
   [junit4]   2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CloudExitableDirectoryReaderTest_2281E7508B9A74DA-001\init-core-data-001
   [junit4]   2> 1936328 WARN  (SUITE-CloudExitableDirectoryReaderTest-seed#[2281E7508B9A74DA]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=37 numCloses=37
   [junit4]   2> 1936329 INFO  (SUITE-CloudExitableDirectoryReaderTest-seed#[2281E7508B9A74DA]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1936331 INFO  (SUITE-CloudExitableDirectoryReaderTest-seed#[2281E7508B9A74DA]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0)
   [junit4]   2> 1936333 INFO  (SUITE-CloudExitableDirectoryReaderTest-seed#[2281E7508B9A74DA]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1936334 INFO  (SUITE-CloudExitableDirectoryReaderTest-seed#[2281E7508B9A74DA]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CloudExitableDirectoryReaderTest_2281E7508B9A74DA-001\tempDir-001
   [junit4]   2> 1936335 INFO  (SUITE-CloudExitableDirectoryReaderTest-seed#[2281E7508B9A74DA]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1936335 INFO  (Thread-3941) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1936336 INFO  (Thread-3941) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1936345 ERROR (Thread-3941) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1936436 INFO  (SUITE-CloudExitableDirectoryReaderTest-seed#[2281E7508B9A74DA]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:54158
   [junit4]   2> 1936480 INFO  (jetty-launcher-2897-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1936480 INFO  (jetty-launcher-2897-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1936482 INFO  (jetty-launcher-2897-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@887d39{/solr,null,AVAILABLE}
   [junit4]   2> 1936482 INFO  (jetty-launcher-2897-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@12d1bbc{/solr,null,AVAILABLE}
   [junit4]   2> 1936485 INFO  (jetty-launcher-2897-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1796732{SSL,[ssl, http/1.1]}{127.0.0.1:?????}
   [junit4]   2> 1936485 INFO  (jetty-launcher-2897-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@169cfdd{SSL,[ssl, http/1.1]}{127.0.0.1:?????}
   [junit4]   2> 1936485 INFO  (jetty-launcher-2897-thread-2) [    ] o.e.j.s.Server Started @???????ms
   [junit4]   2> 1936485 INFO  (jetty-launcher-2897-thread-1) [    ] o.e.j.s.Server Started @???????ms
   [junit4]   2> 1936486 INFO  (jetty-launcher-2897-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=54163}
   [junit4]   2> 1936486 INFO  (jetty-launcher-2897-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=54162}
   [junit4]   2> 1936486 ERROR (jetty-launcher-2897-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1936486 ERROR (jetty-launcher-2897-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1936486 INFO  (jetty-launcher-2897-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1936486 INFO  (jetty-launcher-2897-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1936486 INFO  (jetty-launcher-2897-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1936486 INFO  (jetty-launcher-2897-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1936486 INFO  (jetty-launcher-2897-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1936487 INFO  (jetty-launcher-2897-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1936487 INFO  (jetty-launcher-2897-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-14T11:03:22.284486900Z
   [junit4]   2> 1936487 INFO  (jetty-launcher-2897-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-14T11:03:22.284486900Z
   [junit4]   2> 1936491 INFO  (jetty-launcher-2897-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1936491 INFO  (jetty-launcher-2897-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1936498 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15de069951c0001, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 1936503 INFO  (jetty-launcher-2897-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54158/solr
   [junit4]   2> 1936503 INFO  (jetty-launcher-2897-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54158/solr
   [junit4]   2> 1936508 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15de069951c0004, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 1936743 INFO  (jetty-launcher-2897-thread-1) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1936745 INFO  (jetty-launcher-2897-thread-1) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54162_solr
   [junit4]   2> 1936747 INFO  (jetty-launcher-2897-thread-1) [    ] o.a.s.c.Overseer Overseer (id=98481510949847045-127.0.0.1:54162_solr-n_0000000000) starting
   [junit4]   2> 1936756 INFO  (jetty-launcher-2897-thread-2) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1936759 INFO  (jetty-launcher-2897-thread-2) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54163_solr
   [junit4]   2> 1936773 INFO  (zkCallback-2908-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1936773 INFO  (zkCallback-2909-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1936773 INFO  (jetty-launcher-2897-thread-1) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54162_solr
   [junit4]   2> 1936868 INFO  (zkCallback-2908-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1936868 INFO  (zkCallback-2909-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1936992 INFO  (jetty-launcher-2897-thread-2) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54163.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a9520a
   [junit4]   2> 1936996 INFO  (jetty-launcher-2897-thread-2) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54163.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a9520a
   [junit4]   2> 1936996 INFO  (jetty-launcher-2897-thread-2) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54163.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a9520a
   [junit4]   2> 1936998 INFO  (jetty-launcher-2897-thread-2) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CloudExitableDirectoryReaderTest_2281E7508B9A74DA-001\tempDir-001\node2\.
   [junit4]   2> 1937204 INFO  (jetty-launcher-2897-thread-1) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54162.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a9520a
   [junit4]   2> 1937208 INFO  (jetty-launcher-2897-thread-1) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54162.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a9520a
   [junit4]   2> 1937208 INFO  (jetty-launcher-2897-thread-1) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54162.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a9520a
   [junit4]   2> 1937230 INFO  (jetty-launcher-2897-thread-1) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CloudExitableDirectoryReaderTest_2281E7508B9A74DA-001\tempDir-001\node1\.
   [junit4]   2> 1937258 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15de069951c0007, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 1937265 INFO  (SUITE-CloudExitableDirectoryReaderTest-seed#[2281E7508B9A74DA]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1937266 INFO  (SUITE-CloudExitableDirectoryReaderTest-seed#[2281E7508B9A74DA]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:54158/solr ready
   [junit4]   2> 1937333 INFO  (qtp16987356-17861) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d&replicationFactor=1&collection.configName=conf&name=exitable&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1937335 INFO  (qtp16987356-17861) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d&replicationFactor=1&collection.configName=conf&name=exitable&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 1937337 INFO  (qtp16987356-17863) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1937338 INFO  (qtp16987356-17863) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1937340 INFO  (OverseerThreadFactory-8427-thread-1) [    ] o.a.s.c.CreateCollectionCmd Create collection exitable
   [junit4]   2> 1937454 INFO  (OverseerStateUpdate-98481510949847045-127.0.0.1:54162_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"exitable",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"exitable_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:54162/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1937459 INFO  (OverseerStateUpdate-98481510949847045-127.0.0.1:54162_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"exitable",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"exitable_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:54163/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1937677 INFO  (qtp16987356-17852) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=exitable&version=2&replicaType=NRT&async=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d423624134968642&coreNodeName=core_node4&name=exitable_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=0
   [junit4]   2> 1937678 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=exitable&version=2&replicaType=NRT&async=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d423624134968642&coreNodeName=core_node4&name=exitable_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 1937679 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1937716 INFO  (qtp20511519-17857) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&collection=exitable&version=2&replicaType=NRT&async=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d423624129477441&coreNodeName=core_node3&name=exitable_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=0
   [junit4]   2> 1937716 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&collection=exitable&version=2&replicaType=NRT&async=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d423624129477441&coreNodeName=core_node3&name=exitable_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 1937717 INFO  (qtp16987356-17856) [    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 5349ecc0-4f77-4f6c-ab15-dc35c7a5417d423624134968642
   [junit4]   2> 1937718 INFO  (qtp16987356-17856) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d423624134968642&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1937718 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1937828 INFO  (zkCallback-2908-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/exitable/state.json] for collection [exitable] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1937828 INFO  (zkCallback-2909-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/exitable/state.json] for collection [exitable] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1937829 INFO  (zkCallback-2908-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/exitable/state.json] for collection [exitable] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1937829 INFO  (zkCallback-2909-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/exitable/state.json] for collection [exitable] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1938341 INFO  (qtp16987356-17858) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1938342 INFO  (qtp16987356-17858) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1938701 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1938710 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.s.IndexSchema [exitable_shard2_replica_n2] Schema name=minimal
   [junit4]   2> 1938712 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1938712 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'exitable_shard2_replica_n2' using configuration from collection exitable, trusted=true
   [junit4]   2> 1938713 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54163.solr.core.exitable.shard2.replica_n2' (registry 'solr.core.exitable.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a9520a
   [junit4]   2> 1938713 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1938713 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.c.SolrCore [[exitable_shard2_replica_n2] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CloudExitableDirectoryReaderTest_2281E7508B9A74DA-001\tempDir-001\node2\exitable_shard2_replica_n2], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CloudExitableDirectoryReaderTest_2281E7508B9A74DA-001\tempDir-001\node2\.\exitable_shard2_replica_n2\data\]
   [junit4]   2> 1938720 INFO  (qtp16987356-17861) [    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 5349ecc0-4f77-4f6c-ab15-dc35c7a5417d423624134968642
   [junit4]   2> 1938720 INFO  (qtp16987356-17861) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d423624134968642&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1938737 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1938751 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.s.IndexSchema [exitable_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 1938754 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1938754 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'exitable_shard1_replica_n1' using configuration from collection exitable, trusted=true
   [junit4]   2> 1938755 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54162.solr.core.exitable.shard1.replica_n1' (registry 'solr.core.exitable.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@a9520a
   [junit4]   2> 1938755 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1938755 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.c.SolrCore [[exitable_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CloudExitableDirectoryReaderTest_2281E7508B9A74DA-001\tempDir-001\node1\exitable_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CloudExitableDirectoryReaderTest_2281E7508B9A74DA-001\tempDir-001\node1\.\exitable_shard1_replica_n1\data\]
   [junit4]   2> 1938855 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1938855 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1938858 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1938858 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1938862 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@191aaf[exitable_shard2_replica_n2] main]
   [junit4]   2> 1938863 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1938863 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1938865 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1938866 INFO  (searcherExecutor-8430-thread-1) [    ] o.a.s.c.SolrCore [exitable_shard2_replica_n2] Registered new searcher Searcher@191aaf[exitable_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1938866 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575704177843109888
   [junit4]   2> 1938874 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1938874 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1938874 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:54163/solr/exitable_shard2_replica_n2/
   [junit4]   2> 1938874 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1938874 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.c.SyncStrategy https://127.0.0.1:54163/solr/exitable_shard2_replica_n2/ has no replicas
   [junit4]   2> 1938874 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1938882 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:54163/solr/exitable_shard2_replica_n2/ shard2
   [junit4]   2> 1938885 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1938885 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1938888 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1938888 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1938890 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@16d8f8d[exitable_shard1_replica_n1] main]
   [junit4]   2> 1938892 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1938893 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1938894 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1938895 INFO  (searcherExecutor-8431-thread-1) [    ] o.a.s.c.SolrCore [exitable_shard1_replica_n1] Registered new searcher Searcher@16d8f8d[exitable_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1938895 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575704177873518592
   [junit4]   2> 1938904 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1938904 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1938904 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:54162/solr/exitable_shard1_replica_n1/
   [junit4]   2> 1938904 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1938905 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.c.SyncStrategy https://127.0.0.1:54162/solr/exitable_shard1_replica_n1/ has no replicas
   [junit4]   2> 1938905 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1938909 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:54162/solr/exitable_shard1_replica_n1/ shard1
   [junit4]   2> 1939014 INFO  (zkCallback-2909-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/exitable/state.json] for collection [exitable] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1939014 INFO  (zkCallback-2908-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/exitable/state.json] for collection [exitable] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1939014 INFO  (zkCallback-2909-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/exitable/state.json] for collection [exitable] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1939014 INFO  (zkCallback-2908-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/exitable/state.json] for collection [exitable] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1939033 INFO  (parallelCoreAdminExecutor-8425-thread-1) [    ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1939061 INFO  (parallelCoreAdminExecutor-8428-thread-1) [    ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1939168 INFO  (zkCallback-2908-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/exitable/state.json] for collection [exitable] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1939168 INFO  (zkCallback-2909-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/exitable/state.json] for collection [exitable] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1939168 INFO  (zkCallback-2908-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/exitable/state.json] for collection [exitable] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1939168 INFO  (zkCallback-2909-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/exitable/state.json] for collection [exitable] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1939344 INFO  (qtp16987356-17863) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1939345 INFO  (qtp16987356-17863) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1939721 INFO  (qtp16987356-17866) [    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 5349ecc0-4f77-4f6c-ab15-dc35c7a5417d423624134968642
   [junit4]   2> 1939721 INFO  (qtp16987356-17866) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d423624134968642&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1939725 INFO  (qtp20511519-17859) [    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 5349ecc0-4f77-4f6c-ab15-dc35c7a5417d423624129477441
   [junit4]   2> 1939725 INFO  (qtp20511519-17859) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d423624129477441&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1939734 INFO  (OverseerThreadFactory-8427-thread-1) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1940348 INFO  (qtp16987356-17866) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1940349 INFO  (qtp16987356-17866) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1940351 INFO  (qtp16987356-17852) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1940352 INFO  (qtp16987356-17852) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=5349ecc0-4f77-4f6c-ab15-dc35c7a5417d&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1940373 INFO  (TEST-CloudExitableDirectoryReaderTest.test-seed#[2281E7508B9A74DA]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 1940398 INFO  (qtp16987356-17856) [    ] o.a.s.u.p.LogUpdateProcessorFactory [exitable_shard2_replica_n2]  webapp=/solr path=/update params={_stateVer_=exitable:4&wt=javabin&version=2}{add=[2 (1575704179426459648), 3 (1575704179444285440), 5 (1575704179444285441), 6 (1575704179445334016), 7 (1575704179445334017), 9 (1575704179445334018), 17 (1575704179445334019), 18 (1575704179445334020), 19 (1575704179447431168), 21 (1575704179447431169), ... (24 adds)]} 0 20
   [junit4]   2> 1940398 INFO  (qtp20511519-17862) [    ] o.a.s.u.p.LogUpdateProcessorFactory [exitable_shard1_replica_n1]  webapp=/solr path=/update params={_stateVer_=exitable:4&wt=javabin&version=2}{add=[1 (1575704179441139712), 4 (1575704179444285440), 8 (1575704179445334016), 10 (1575704179445334017), 11 (1575704179445334018), 12 (1575704179445334019), 13 (1575704179445334020), 14 (1575704179445334021), 15 (1575704179445334022), 16 (1575704179445334023), ... (33 adds)]} 0 7
   [junit4]   2> 1940407 INFO  (qtp20511519-17853) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1575704179458965504,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1940407 INFO  (qtp20511519-17853) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@195df98 commitCommandVersion:1575704179458965504
   [junit4]   2> 1940407 INFO  (qtp16987356-17858) [    ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1575704179458965504,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1940408 INFO  (qtp16987356-17858) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1d7d809 commitCommandVersion:1575704179458965504
   [junit4]   2> 1940422 INFO  (qtp20511519-17853) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@312e96[exitable_shard1_replica_n1] main]
   [junit4]   2> 1940423 INFO  (qtp20511519-17853) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1940423 INFO  (qtp16987356-17858) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1cac29c[exitable_shard2_replica_n2] main]
   [junit4]   2> 1940423 INFO  (searcherExecutor-8431-thread-1) [    ] o.a.s.c.SolrCore [exitable_shard1_replica_n1] Registered new searcher Searcher@312e96[exitable_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C33)))}
   [junit4]   2> 1940423 INFO  (qtp20511519-17853) [    ] o.a.s.u.p.LogUpdateProcessorFactory [exitable_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:54162/solr/exitable_shard1_replica_n1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 19
   [junit4]   2> 1940430 INFO  (qtp16987356-17858) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1940431 INFO  (searcherExecutor-8430-thread-1) [    ] o.a.s.c.SolrCore [exitable_shard2_replica_n2] Registered new searcher Searcher@1cac29c[exitable_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C24)))}
   [junit4]   2> 1940431 INFO  (qtp16987356-17858) [    ] o.a.s.u.p.LogUpdateProcessorFactory [exitable_shard2_replica_n2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:54162/solr/exitable_shard1_replica_n1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 23
   [junit4]   2> 1940433 INFO  (qtp20511519-17850) [    ] o.a.s.u.p.LogUpdateProcessorFactory [exitable_shard1_replica_n1]  webapp=/solr path=/update params={_stateVer_=exitable:4&commit=true&wt=javabin&version=2}{commit=} 0 32
   [junit4]   2> 1940447 WARN  (qtp16987356-17864) [    ] o.a.s.s.SolrIndexSearcher Query: [name:a*]; The request took too long to iterate over terms. Timeout: timeoutAt: 423627123193101 (System.nanoTime(): 423627124984640), TermsEnum=org.apache.lucene.codecs.blocktree.IntersectTermsEnum@6eee35
   [junit4]   2> 1940452 INFO  (qtp16987356-17864) [    ] o.a.s.c.S.Request [exitable_shard2_replica_n2]  webapp=/solr path=/select params={distrib=false&_stateVer_=exitable:4&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=https://127.0.0.1:54163/solr/exitable_shard2_replica_n2/&rows=10&version=2&sleep=2&q=name:a*&NOW=1502708606232&isShard=true&timeAllowed=1&wt=javabin} hits=0 status=0 QTime=8
   [junit4]   2> 1940460 WARN  (qtp20511519-17859) [    ] o.a.s.s.SolrIndexSearcher Query: [name:a*]; The request took too long to iterate over terms. Timeout: timeoutAt: 423627137551007 (System.nanoTime(): 423627137928857), TermsEnum=org.apache.lucene.codecs.blocktree.IntersectTermsEnum@6c0528
   [junit4]   2> 1940461 INFO  (qtp20511519-17859) [    ] o.a.s.c.S.Request [exitable_shard1_replica_n1]  webapp=/solr path=/select params={distrib=false&_stateVer_=exitable:4&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=https://127.0.0.1:54162/solr/exitable_shard1_replica_n1/&rows=10&version=2&sleep=2&q=name:a*&NOW=1502708606232&isShard=true&timeAllowed=1&wt=javabin} hits=0 status=0 QTime=2
   [junit4]   2> 1940462 INFO  (qtp16987356-17863) [    ] o.a.s.c.S.Request [exitable_shard2_replica_n2]  webapp=/solr path=/select params={sleep=2&q=name:a*&_stateVer_=exitable:4&timeAllowed=1&wt=javabin&version=2} hits=0 status=0 QTime=26
   [junit4]   2> 1940467 INFO  (qtp20511519-17855) [    ] o.a.s.c.S.Request [exitable_shard1_replica_n1]  webapp=/solr path=/select params={distrib=false&_stateVer_=exitable:4&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=https://127.0.0.1:54162/solr/exitable_shard1_replica_n1/&rows=10&version=2&q=name:a*&NOW=1502708606261&isShard=true&timeAllowed=1818785124&wt=javabin} hits=10 status=0 QTime=1
   [junit4]   2> 1940467 INFO  (qtp16987356-17866) [    ] o.a.s.c.S.Request [exitable_shard2_replica_n2]  webapp=/solr path=/select params={distrib=false&_stateVer_=exitable:4&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=https://127.0.0.1:54163/solr/exitable_shard2_replica_n2/&rows=10&version=2&q=name:a*&NOW=1502708606261&isShard=true&timeAllowed=1818785124&wt=javabin} hits=9 status=0 QTime=1
   [junit4]   2> 1940470 INFO  (qtp20511519-17855) [    ] o.a.s.c.S.Request [exitable_shard1_replica_n1]  webapp=/solr path=/select params={q=name:a*&distrib=false&_stateVer_=exitable:4&shards.purpose=64&NOW=1502708606261&ids=11,12,1,13,14,15,4,16,8,10&isShard=true&timeAllowed=1818785124&shard.url=https://127.0.0.1:54162/solr/exitable_shard1_replica_n1/&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1940471 INFO  (qtp20511519-17862) [    ] o.a.s.c.S.Request [exitable_shard1_replica_n1]  webapp=/solr path=/select params={q=name:a*&_stateVer_=exitable:4&timeAllowed=1818785124&wt=javabin&version=2} hits=19 status=0 QTime=7
   [junit4]   2> 1940476 INFO  (qtp16987356-17866) [    ] o.a.s.c.S.Request [exitable_shard2_replica_n2]  webapp=/solr path=/select params={distrib=false&_stateVer_=exitable:4&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=https://127.0.0.1:54163/solr/exitable_shard2_replica_n2/&rows=10&version=2&sleep=2&q=name:a*&NOW=1502708606269&isShard=true&timeAllowed=1&wt=javabin} hits=9 status=0 QTime=1
   [junit4]   2> 1940476 INFO  (qtp20511519-17857) [    ] o.a.s.c.S.Request [exitable_shard1_replica_n1]  webapp=/solr path=/select params={distrib=false&_stateVer_=exitable:4&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=https://127.0.0.1:54162/solr/exitable_shard1_replica_n1/&rows=10&version=2&sleep=2&q=name:a*&NOW=1502708606269&isShard=true&timeAllowed=1&wt=javabin} hits=10 status=0 QTime=1
   [junit4]   2> 1940483 WARN  (qtp20511519-17860) [    ] o.a.s.h.c.SearchHandler Query: sleep=2&q=name:a*&distrib=false&_stateVer_=exitable:4&shards.purpose=64&NOW=1502708606269&ids=11,12,1,13,14,15,4,16,8,10&isShard=true&timeAllowed=1&shard.url=https://127.0.0.1:54162/solr/exitable_shard1_replica_n1/&wt=javabin&version=2; The request took too long to iterate over terms. Timeout: timeoutAt: 423627158205378 (System.nanoTime(): 423627160250641), TermsEnum=org.apache.lucene.codecs.blocktree.SegmentTermsEnum@1f807cd
   [junit4]   2> 1940483 INFO  (qtp20511519-17860) [    ] o.a.s.c.S.Request [exitable_shard1_replica_n1]  webapp=/solr path=/select params={sleep=2&q=name:a*&distrib=false&_stateVer_=exitable:4&shards.purpose=64&NOW=1502708606269&ids=11,12,1,13,14,15,4,16,8,10&isShard=true&timeAllowed=1&shard.url=https://127.0.0.1:54162/solr/exitable_shard1_replica_n1/&wt=javabin&version=2} status=0 QTime=4
   [junit4]   2> 1940484 INFO  (qtp20511519-17850) [    ] o.a.s.c.S.Request [exitable_shard1_replica_n1]  webapp=/solr path=/select params={sleep=2&q=name:a*&_stateVer_=exitable:4&timeAllowed=1&wt=javabin&version=2} hits=19 status=0 QTime=10
   [junit4]   2> 1940485 INFO  (TEST-CloudExitableDirectoryReaderTest.test-seed#[2281E7508B9A74DA]) [    ] o.a.s.SolrTestCaseJ4 ###Ending test
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CloudExitableDirectoryReaderTest -Dtests.method=test -Dtests.seed=2281E7508B9A74DA -Dtests.slow=true -Dtests.locale=ar-001 -Dtests.timezone=Etc/GMT-3 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.13s J0 | CloudExitableDirectoryReaderTest.test <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: partialResults were expected expected:<true> but was:<null>
   [junit4]    > at __randomizedtesting.SeedInfo.seed([2281E7508B9A74DA:AAD5D88A25661922]:0)
   [junit4]    > at org.apache.solr.cloud.CloudExitableDirectoryReaderTest.assertPartialResults(CloudExitableDirectoryReaderTest.java:107)
   [junit4]    > at org.apache.solr.cloud.CloudExitableDirectoryReaderTest.doTimeoutTests(CloudExitableDirectoryReaderTest.java:90)
   [junit4]    > at org.apache.solr.cloud.CloudExitableDirectoryReaderTest.test(CloudExitableDirectoryReaderTest.java:56)
   [junit4]    > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    > at java.base/java.lang.reflect.Method.invoke(Method.java:564)
   [junit4]    > at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 1940489 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15de069951c0008, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 1940494 INFO  (jetty-closer-2898-thread-2) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@169cfdd{SSL,[ssl, http/1.1]}{127.0.0.1:?}
   [junit4]   2> 1940495 INFO  (jetty-closer-2898-thread-2) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=21343354
   [junit4]   2> 1940495 INFO  (jetty-closer-2898-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@1796732{SSL,[ssl, http/1.1]}{127.0.0.1:?}
   [junit4]   2> 1940496 INFO  (jetty-closer-2898-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=6282150
   [junit4]   2> 1940496 INFO  (jetty-closer-2898-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1940496 INFO  (jetty-closer-2898-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1940500 INFO  (jetty-closer-2898-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1940501 INFO  (jetty-closer-2898-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1940502 INFO  (jetty-closer-2898-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1940503 INFO  (jetty-closer-2898-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1940505 INFO  (jetty-closer-2898-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1940506 INFO  (jetty-closer-2898-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1940507 INFO  (coreCloseExecutor-8440-thread-1) [    ] o.a.s.c.SolrCore [exitable_shard2_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@1775a95
   [junit4]   2> 1940508 INFO  (coreCloseExecutor-8441-thread-1) [    ] o.a.s.c.SolrCore [exitable_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@326d0f
   [junit4]   2> 1940508 INFO  (coreCloseExecutor-8440-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.exitable.shard2.replica_n2, tag=24599189
   [junit4]   2> 1940513 INFO  (coreCloseExecutor-8441-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.exitable.shard1.replica_n1, tag=3304719
   [junit4]   2> 1940519 INFO  (coreCloseExecutor-8441-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.exitable.shard1.leader, tag=3304719
   [junit4]   2> 1940526 INFO  (jetty-closer-2898-thread-2) [    ] o.a.s.c.Overseer Overseer (id=98481510949847045-127.0.0.1:54162_solr-n_0000000000) closing
   [junit4]   2> 1940526 INFO  (OverseerStateUpdate-98481510949847045-127.0.0.1:54162_solr-n_0000000000) [    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:54162_solr
   [junit4]   2> 1940530 WARN  (zkCallback-2908-thread-1) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1940531 INFO  (jetty-closer-2898-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@887d39{/solr,null,UNAVAILABLE}
   [junit4]   2> 1940531 INFO  (zkCallback-2909-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 1940532 INFO  (zkCallback-2909-thread-2) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54163_solr
   [junit4]   2> 1940534 INFO  (coreCloseExecutor-8440-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.exitable.shard2.leader, tag=24599189
   [junit4]   2> 1942038 WARN  (zkCallback-2909-thread-2) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1942038 INFO  (jetty-closer-2898-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@12d1bbc{/solr,null,UNAVAILABLE}
   [junit4]   2> 1942039 ERROR (SUITE-CloudExitableDirectoryReaderTest-seed#[2281E7508B9A74DA]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1942040 INFO  (SUITE-CloudExitableDirectoryReaderTest-seed#[2281E7508B9A74DA]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:54158 54158
   [junit4]   2> 1942041 INFO  (Thread-3941) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:54158 54158
   [junit4]   2> 1942042 WARN  (Thread-3941) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/aliases.json
   [junit4]   2> 2 /solr/security.json
   [junit4]   2> 2 /solr/configs/conf
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 11 /solr/collections/exitable/state.json
   [junit4]   2> 3 /solr/clusterstate.json
   [junit4]   2> 3 /solr/clusterprops.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/live_nodes
   [junit4]   2> 3 /solr/collections
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CloudExitableDirectoryReaderTest_2281E7508B9A74DA-001
   [junit4]   2> Aug 14, 2017 11:03:27 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {name=PostingsFormat(name=MockRandom), id=PostingsFormat(name=MockRandom)}, docValues:{}, maxPointsInLeafNode=880, maxMBSortInHeap=6.233291310204748, sim=RandomSimilarity(queryNorm=false): {}, locale=ar-001, timezone=Etc/GMT-3
   [junit4]   2> NOTE: Windows 10 10.0 x86/Oracle Corporation 9 (32-bit)/cpus=3,threads=1,free=193511184,total=467980288
   [junit4]   2> NOTE: All tests run in this JVM: [CollectionsAPIAsyncDistributedZkTest, DefaultValueUpdateProcessorTest, VersionInfoTest, TestDistribIDF, MoveReplicaHDFSUlogDirTest, BJQParserTest, TestCloudDeleteByQuery, TestLegacyField, TestFilteredDocIdSet, SegmentsInfoRequestHandlerTest, LargeFieldTest, CustomCollectionTest, SuggesterWFSTTest, CollectionStateFormat2Test, TestLeaderElectionZkExpiry, TestTolerantUpdateProcessorCloud, TestBlobHandler, DocValuesNotIndexedTest, DateMathParserTest, RuleEngineTest, BooleanFieldTest, TestCloudPseudoReturnFields, TestPHPSerializedResponseWriter, DirectUpdateHandlerOptimizeTest, ExitableDirectoryReaderTest, TestCollectionAPI, TestSlowCompositeReaderWrapper, BigEndianAscendingWordSerializerTest, TestConfigOverlay, TestLegacyFieldCache, OutOfBoxZkACLAndCredentialsProvidersTest, TlogReplayBufferedWhileIndexingTest, ConcurrentDeleteAndCreateCollectionTest, TestConfigSetsAPI, TestFieldCache, TestHdfsCloudBackupRestore, DirectUpdateHandlerTest, TestSubQueryTransformer, UUIDUpdateProcessorFallbackTest, TestCustomDocTransformer, CachingDirectoryFactoryTest, TestNonDefinedSimilarityFactory, SSLMigrationTest, NumericFieldsTest, PrimUtilsTest, TestFieldCollectionResource, CdcrRequestHandlerTest, RankQueryTest, RequestLoggingTest, RegexBoostProcessorTest, SolrCLIZkUtilsTest, BlobRepositoryCloudTest, LeaderFailureAfterFreshStartTest, TestJmxIntegration, ManagedSchemaRoundRobinCloudTest, HdfsWriteToMultipleCollectionsTest, TestCoreContainer, CursorPagingTest, SliceStateTest, TestManagedStopFilterFactory, TestDistributedSearch, TestRawResponseWriter, TestLegacyNumericUtils, TestUseDocValuesAsStored2, DirectSolrConnectionTest, TestTestInjection, ShufflingReplicaListTransformerTest, AtomicUpdateProcessorFactoryTest, ClassificationUpdateProcessorTest, SolrGraphiteReporterTest, TestJavabinTupleStreamParser, HdfsTlogReplayBufferedWhileIndexingTest, TestSha256AuthenticationProvider, GraphQueryTest, ClassificationUpdateProcessorFactoryTest, TestDocBasedVersionConstraints, TestInfoStreamLogging, ShardSplitTest, ScriptEngineTest, TestStreamBody, QueryElevationComponentTest, DistributedQueueTest, TestReqParamsAPI, UniqFieldsUpdateProcessorFactoryTest, TestIBSimilarityFactory, TestIndexingPerformance, TestPrepRecovery, TestGroupingSearch, TestImpersonationWithHadoopAuth, OverseerTaskQueueTest, TestExtendedDismaxParser, IndexSchemaTest, SimplePostToolTest, TestDFRSimilarityFactory, CdcrReplicationDistributedZkTest, TestBulkSchemaAPI, HdfsLockFactoryTest, TestNamedUpdateProcessors, TestXIncludeConfig, SortByFunctionTest, NotRequiredUniqueKeyTest, TestRTimerTree, URLClassifyProcessorTest, TestSystemCollAutoCreate, TestCursorMarkWithoutUniqueKey, ExplicitHLLTest, AnalysisAfterCoreReloadTest, TestRandomRequestDistribution, HealthCheckHandlerTest, TestInPlaceUpdatesStandalone, TestMaxTokenLenTokenizer, DistributedFacetPivotSmallTest, TestUpdate, TestHashQParserPlugin, DistributedFacetPivotLongTailTest, MinimalSchemaTest, HighlighterMaxOffsetTest, TestManagedSynonymGraphFilterFactory, ResourceLoaderTest, TestHdfsUpdateLog, TestLegacyNumericRangeQueryBuilder, TestRandomDVFaceting, TestShardHandlerFactory, ZkStateWriterTest, TestFuzzyAnalyzedSuggestions, TestSubQueryTransformerDistrib, RestartWhileUpdatingTest, OutputWriterTest, NoCacheHeaderTest, TestBlendedInfixSuggestions, SmileWriterTest, SolrIndexSplitterTest, TestFunctionQuery, PathHierarchyTokenizerFactoryTest, ResponseHeaderTest, TimeZoneUtilsTest, TestManagedResourceStorage, SolrCoreTest, RecoveryZkTest, TestConfigsApi, TestSolrCoreParser, SyncSliceTest, CircularListTest, TestApiFramework, AliasIntegrationTest, TestSchemalessBufferedUpdates, TestExactStatsCache, TestStressReorder, ClusterStateTest, TestPointFields, TestSchemaSimilarityResource, TestWriterPerf, XmlUpdateRequestHandlerTest, ClusterStateUpdateTest, DocExpirationUpdateProcessorFactoryTest, FieldAnalysisRequestHandlerTest, WrapperMergePolicyFactoryTest, RequestHandlersTest, TestHighlightDedupGrouping, SolrMetricManagerTest, UnloadDistributedZkTest, TestEmbeddedSolrServerSchemaAPI, DistanceFunctionTest, HighlighterTest, VMParamsZkACLAndCredentialsProvidersTest, FastVectorHighlighterTest, HdfsBasicDistributedZkTest, HdfsRecoveryZkTest, TestQuerySenderListener, LeaderElectionTest, TestMultiValuedNumericRangeQuery, TestSolrCloudWithKerberosAlt, TestXmlQParserPlugin, TestSegmentSorting, ChaosMonkeySafeLeaderTest, OverseerTest, LeaderElectionIntegrationTest, ShardRoutingTest, BasicZkTest, FullSolrCloudDistribCmdsTest, TestReplicationHandler, TestRandomFaceting, ZkSolrClientTest, TestZkChroot, ZkCLITest, AutoCommitTest, TestDistributedGrouping, TestFaceting, TestHashPartitioner, DistributedSpellCheckComponentTest, ZkControllerTest, TestJoin, TestStressVersions, HardAutoCommitTest, DistributedTermsComponentTest, TestRangeQuery, SimpleFacetsTest, SpellCheckComponentTest, PeerSyncTest, TestSort, TestFiltering, TestLazyCores, TestIndexSearcher, SoftAutoCommitTest, CoreAdminHandlerTest, SolrRequestParserTest, TestFoldingMultitermQuery, DocValuesTest, SuggesterTest, TestStressLucene, SpatialFilterTest, WordBreakSolrSpellCheckerTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, TestWordDelimiterFilterFactory, QueryEqualityTest, StatelessScriptUpdateProcessorFactoryTest, TestRemoteStreaming, TestSolrDeletionPolicy1, SolrInfoBeanTest, XsltUpdateRequestHandlerTest, CacheHeaderTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, DisMaxRequestHandlerTest, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, TestQueryTypes, PrimitiveFieldTypeTest, TestOmitPositions, TermsComponentTest, TestValueSourceCache, TermVectorComponentTest, RequiredFieldsTest, IndexSchemaRuntimeFieldTest, LoggingHandlerTest, SolrPluginUtilsTest, ReturnFieldsTest, UpdateRequestProcessorFactoryTest, TestCSVResponseWriter, MBeansHandlerTest, QueryParsingTest, JsonLoaderTest, BinaryUpdateRequestHandlerTest, CSVRequestHandlerTest, SearchHandlerTest, TestLFUCache, HighlighterConfigTest, UpdateParamsTest, AlternateDirectoryTest, TestQuerySenderNoQuery, CopyFieldTest, ResponseLogComponentTest, SolrIndexConfigTest, BadComponentTest, TestStressRecovery, TestMergePolicyConfig, TestSolrDeletionPolicy2, MultiTermTest, SampleTest, TestDocSet, TestBinaryField, TestSolrCoreProperties, TestPostingsSolrHighlighter, TestLuceneMatchVersion, TestPhraseSuggestions, SpellPossibilityIteratorTest, TestCharFilters, TestCodecSupport, TestSweetSpotSimilarityFactory, TestPerFieldSimilarity, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, TestFastWriter, TestFastOutputStream, TestFastLRUCache, ChaosMonkeyNothingIsSafeTest, BaseCdcrDistributedZkTest, CdcrReplicationHandlerTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, CloudExitableDirectoryReaderTest]
   [junit4] Completed [494/729 (1!)] on J0 in 6.00s, 1 test, 1 failure <<< FAILURES!

[...truncated 37328 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk1.8.0_144) - Build # 6823 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6823/
Java: 64bit/jdk1.8.0_144 -XX:-UseCompressedOops -XX:+UseSerialGC

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIAsyncDistributedZkTest.testAsyncRequests

Error Message:
DeleteCollection did not complete expected same:<COMPLETED> was not:<FAILED>

Stack Trace:
java.lang.AssertionError: DeleteCollection did not complete expected same:<COMPLETED> was not:<FAILED>
        at __randomizedtesting.SeedInfo.seed([7780120901B9DB52:93C42EBEA711958D]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotSame(Assert.java:641)
        at org.junit.Assert.assertSame(Assert.java:580)
        at org.apache.solr.cloud.CollectionsAPIAsyncDistributedZkTest.testAsyncRequests(CollectionsAPIAsyncDistributedZkTest.java:174)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader

Error Message:
Doc with id=1 not found in http://127.0.0.1:61232/ltx/forceleader_test_collection due to: Path not found: /id; rsp={doc=null}

Stack Trace:
java.lang.AssertionError: Doc with id=1 not found in http://127.0.0.1:61232/ltx/forceleader_test_collection due to: Path not found: /id; rsp={doc=null}
        at __randomizedtesting.SeedInfo.seed([7780120901B9DB52:911726C9383B2233]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:603)
        at org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:556)
        at org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:142)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 11401 lines...]
   [junit4] Suite: org.apache.solr.cloud.ForceLeaderTest
   [junit4]   2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\init-core-data-001
   [junit4]   2> 687625 INFO  (SUITE-ForceLeaderTest-seed#[7780120901B9DB52]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 687628 INFO  (SUITE-ForceLeaderTest-seed#[7780120901B9DB52]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 687628 INFO  (SUITE-ForceLeaderTest-seed#[7780120901B9DB52]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 687628 INFO  (SUITE-ForceLeaderTest-seed#[7780120901B9DB52]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /ltx/
   [junit4]   2> 687631 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 687632 INFO  (Thread-1541) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 687632 INFO  (Thread-1541) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 687638 ERROR (Thread-1541) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 687732 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.ZkTestServer start zk server on port:60612
   [junit4]   2> 687755 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 687759 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 687761 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 687764 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 687766 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 687768 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 687771 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 687773 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 687775 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 687778 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 687780 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 687784 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
   [junit4]   2> 687784 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15de20a88350001, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 688499 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 688499 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@442ec4b{/ltx,null,AVAILABLE}
   [junit4]   2> 688501 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@6f3e1269{HTTP/1.1,[http/1.1]}{127.0.0.1:60623}
   [junit4]   2> 688501 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.e.j.s.Server Started @696559ms
   [junit4]   2> 688501 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\tempDir-001/control/data, replicaType=NRT, hostContext=/ltx, hostPort=60622, coreRootDirectory=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\control-001\cores}
   [junit4]   2> 688502 ERROR (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 688502 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 688502 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 688502 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 688502 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-14T18:38:48.213Z
   [junit4]   2> 688509 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 688509 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\control-001\solr.xml
   [junit4]   2> 688520 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 688527 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60612/solr
   [junit4]   2> 688531 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15de20a88350003, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 688667 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60622_ltx    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 688667 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60622_ltx    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:60622_ltx
   [junit4]   2> 688668 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60622_ltx    ] o.a.s.c.Overseer Overseer (id=98483301734744068-127.0.0.1:60622_ltx-n_0000000000) starting
   [junit4]   2> 688678 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60622_ltx    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60622_ltx
   [junit4]   2> 688680 INFO  (OverseerStateUpdate-98483301734744068-127.0.0.1:60622_ltx-n_0000000000) [n:127.0.0.1:60622_ltx    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 688825 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60622_ltx    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1
   [junit4]   2> 688825 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60622_ltx    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1
   [junit4]   2> 688825 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60622_ltx    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1
   [junit4]   2> 688843 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60622_ltx    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\control-001\cores
   [junit4]   2> 688885 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 688885 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:60612/solr ready
   [junit4]   2> 688885 INFO  (SocketProxy-Acceptor-60622) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=60638,localport=60622], receiveBufferSize:65536
   [junit4]   2> 688890 INFO  (SocketProxy-Acceptor-60622) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=60623,localport=60639], receiveBufferSize=65536
   [junit4]   2> 688891 INFO  (qtp1981803108-6619) [n:127.0.0.1:60622_ltx    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:60622_ltx&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 688896 INFO  (OverseerThreadFactory-2624-thread-1-processing-n:127.0.0.1:60622_ltx) [n:127.0.0.1:60622_ltx    ] o.a.s.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 689004 INFO  (SocketProxy-Acceptor-60622) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=60643,localport=60622], receiveBufferSize:65536
   [junit4]   2> 689004 INFO  (SocketProxy-Acceptor-60622) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=60623,localport=60644], receiveBufferSize=65536
   [junit4]   2> 689007 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 689008 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 689119 INFO  (zkCallback-1081-thread-1-processing-n:127.0.0.1:60622_ltx) [n:127.0.0.1:60622_ltx    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 690037 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 690049 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 690179 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 690201 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 690201 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1
   [junit4]   2> 690202 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 690202 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\control-001\cores\control_collection_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\control-001\cores\control_collection_shard1_replica_n1\data\]
   [junit4]   2> 690204 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=30, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 690206 WARN  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 690304 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 690304 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 690305 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 690305 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 690307 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=97.5654296875, floorSegmentMB=1.01953125, forceMergeDeletesPctAllowed=11.836872422657862, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7217115328872558
   [junit4]   2> 690307 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@62c1727e[control_collection_shard1_replica_n1] main]
   [junit4]   2> 690309 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 690309 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 690310 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 690310 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575732830557700096
   [junit4]   2> 690313 INFO  (searcherExecutor-2627-thread-1-processing-n:127.0.0.1:60622_ltx x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@62c1727e[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 690322 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 690322 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 690322 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:60622/ltx/control_collection_shard1_replica_n1/
   [junit4]   2> 690322 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 690322 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:60622/ltx/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 690322 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 690328 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:60622/ltx/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 690434 INFO  (zkCallback-1081-thread-1-processing-n:127.0.0.1:60622_ltx) [n:127.0.0.1:60622_ltx    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 690482 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 690484 INFO  (qtp1981803108-6614) [n:127.0.0.1:60622_ltx c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1477
   [junit4]   2> 690488 INFO  (qtp1981803108-6619) [n:127.0.0.1:60622_ltx    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 690589 INFO  (zkCallback-1081-thread-1-processing-n:127.0.0.1:60622_ltx) [n:127.0.0.1:60622_ltx    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 690907 INFO  (OverseerCollectionConfigSetProcessor-98483301734744068-127.0.0.1:60622_ltx-n_0000000000) [n:127.0.0.1:60622_ltx    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 691490 INFO  (qtp1981803108-6619) [n:127.0.0.1:60622_ltx    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:60622_ltx&wt=javabin&version=2} status=0 QTime=2597
   [junit4]   2> 691504 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 691506 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:60612/solr ready
   [junit4]   2> 691506 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 691506 INFO  (SocketProxy-Acceptor-60622) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=60661,localport=60622], receiveBufferSize:65536
   [junit4]   2> 691507 INFO  (SocketProxy-Acceptor-60622) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=60623,localport=60662], receiveBufferSize=65536
   [junit4]   2> 691507 INFO  (qtp1981803108-6611) [n:127.0.0.1:60622_ltx    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 691512 INFO  (OverseerThreadFactory-2624-thread-2-processing-n:127.0.0.1:60622_ltx) [n:127.0.0.1:60622_ltx    ] o.a.s.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 691513 WARN  (OverseerThreadFactory-2624-thread-2-processing-n:127.0.0.1:60622_ltx) [n:127.0.0.1:60622_ltx    ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 691719 INFO  (qtp1981803108-6611) [n:127.0.0.1:60622_ltx    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 691720 INFO  (qtp1981803108-6611) [n:127.0.0.1:60622_ltx    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=212
   [junit4]   2> 692569 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\shard-1-001 of type TLOG
   [junit4]   2> 692569 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 692569 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5d6ee8d2{/ltx,null,AVAILABLE}
   [junit4]   2> 692569 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1a283040{HTTP/1.1,[http/1.1]}{127.0.0.1:60684}
   [junit4]   2> 692569 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.e.j.s.Server Started @700629ms
   [junit4]   2> 692569 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\tempDir-001/jetty1, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/ltx, hostPort=60683, coreRootDirectory=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\shard-1-001\cores}
   [junit4]   2> 692569 ERROR (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 692569 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 692569 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 692569 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 692569 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-14T18:38:52.280Z
   [junit4]   2> 692582 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 692582 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\shard-1-001\solr.xml
   [junit4]   2> 692602 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 692612 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60612/solr
   [junit4]   2> 692616 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15de20a88350008, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 692630 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60683_ltx    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 692635 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60683_ltx    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 692638 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60683_ltx    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60683_ltx
   [junit4]   2> 692642 INFO  (zkCallback-1081-thread-1-processing-n:127.0.0.1:60622_ltx) [n:127.0.0.1:60622_ltx    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 692643 INFO  (zkCallback-1093-thread-1-processing-n:127.0.0.1:60683_ltx) [n:127.0.0.1:60683_ltx    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 692644 INFO  (zkCallback-1088-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 692864 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60683_ltx    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1
   [junit4]   2> 692874 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60683_ltx    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1
   [junit4]   2> 692874 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60683_ltx    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1
   [junit4]   2> 692877 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60683_ltx    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\shard-1-001\cores
   [junit4]   2> 692901 INFO  (qtp1981803108-6610) [n:127.0.0.1:60622_ltx    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:60683_ltx&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 692904 INFO  (OverseerCollectionConfigSetProcessor-98483301734744068-127.0.0.1:60622_ltx-n_0000000000) [n:127.0.0.1:60622_ltx    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 692904 INFO  (OverseerThreadFactory-2624-thread-3-processing-n:127.0.0.1:60622_ltx) [n:127.0.0.1:60622_ltx    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:60683_ltx for creating new replica
   [junit4]   2> 692908 INFO  (SocketProxy-Acceptor-60683) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=60703,localport=60683], receiveBufferSize:65536
   [junit4]   2> 692908 INFO  (SocketProxy-Acceptor-60683) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=60684,localport=60704], receiveBufferSize=65536
   [junit4]   2> 692910 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 692911 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 693020 INFO  (zkCallback-1093-thread-1-processing-n:127.0.0.1:60683_ltx) [n:127.0.0.1:60683_ltx    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 693934 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 693945 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.s.IndexSchema [collection1_shard2_replica_t41] Schema name=test
   [junit4]   2> 694076 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 694100 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t41' using configuration from collection collection1, trusted=true
   [junit4]   2> 694100 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_t41' (registry 'solr.core.collection1.shard2.replica_t41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1
   [junit4]   2> 694101 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 694101 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.c.SolrCore [[collection1_shard2_replica_t41] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\shard-1-001\cores\collection1_shard2_replica_t41], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\shard-1-001\cores\collection1_shard2_replica_t41\data\]
   [junit4]   2> 694102 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=30, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 694104 WARN  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 694175 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 694175 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 694176 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 694176 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 694176 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=97.5654296875, floorSegmentMB=1.01953125, forceMergeDeletesPctAllowed=11.836872422657862, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7217115328872558
   [junit4]   2> 694177 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.s.SolrIndexSearcher Opening [Searcher@648df63c[collection1_shard2_replica_t41] main]
   [junit4]   2> 694178 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 694179 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 694179 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 694179 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575732834614640640
   [junit4]   2> 694180 INFO  (searcherExecutor-2638-thread-1-processing-n:127.0.0.1:60683_ltx x:collection1_shard2_replica_t41 s:shard2 c:collection1) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.c.SolrCore [collection1_shard2_replica_t41] Registered new searcher Searcher@648df63c[collection1_shard2_replica_t41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 694189 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 694189 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 694189 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:60683/ltx/collection1_shard2_replica_t41/
   [junit4]   2> 694189 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 694189 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.c.SyncStrategy http://127.0.0.1:60683/ltx/collection1_shard2_replica_t41/ has no replicas
   [junit4]   2> 694189 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 694189 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.c.ZkController collection1_shard2_replica_t41 stopping background replication from leader
   [junit4]   2> 694192 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:60683/ltx/collection1_shard2_replica_t41/ shard2
   [junit4]   2> 694299 INFO  (zkCallback-1093-thread-1-processing-n:127.0.0.1:60683_ltx) [n:127.0.0.1:60683_ltx    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 694347 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 694348 INFO  (qtp1240075242-6669) [n:127.0.0.1:60683_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t41] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1437
   [junit4]   2> 694351 INFO  (qtp1981803108-6610) [n:127.0.0.1:60622_ltx    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:60683_ltx&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2} status=0 QTime=1449
   [junit4]   2> 694453 INFO  (zkCallback-1093-thread-1-processing-n:127.0.0.1:60683_ltx) [n:127.0.0.1:60683_ltx    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 694907 INFO  (OverseerCollectionConfigSetProcessor-98483301734744068-127.0.0.1:60622_ltx-n_0000000000) [n:127.0.0.1:60622_ltx    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 695008 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\shard-2-001 of type TLOG
   [junit4]   2> 695009 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 695009 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6b88dc32{/ltx,null,AVAILABLE}
   [junit4]   2> 695011 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@74d10f2d{HTTP/1.1,[http/1.1]}{127.0.0.1:60710}
   [junit4]   2> 695011 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.e.j.s.Server Started @703069ms
   [junit4]   2> 695011 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\tempDir-001/jetty2, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/ltx, hostPort=60709, coreRootDirectory=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\shard-2-001\cores}
   [junit4]   2> 695012 ERROR (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 695012 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 695012 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 695012 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 695012 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-14T18:38:54.723Z
   [junit4]   2> 695017 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15de20a8835000a, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 695017 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 695017 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\shard-2-001\solr.xml
   [junit4]   2> 695026 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 695031 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60612/solr
   [junit4]   2> 695035 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15de20a8835000b, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 695046 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60709_ltx    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 695049 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60709_ltx    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 695052 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60709_ltx    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60709_ltx
   [junit4]   2> 695053 INFO  (zkCallback-1088-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 695054 INFO  (zkCallback-1081-thread-1-processing-n:127.0.0.1:60622_ltx) [n:127.0.0.1:60622_ltx    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 695055 INFO  (zkCallback-1099-thread-1-processing-n:127.0.0.1:60709_ltx) [n:127.0.0.1:60709_ltx    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 695055 INFO  (zkCallback-1093-thread-1-processing-n:127.0.0.1:60683_ltx) [n:127.0.0.1:60683_ltx    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 695198 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60709_ltx    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1
   [junit4]   2> 695207 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60709_ltx    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1
   [junit4]   2> 695207 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60709_ltx    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1
   [junit4]   2> 695209 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60709_ltx    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\shard-2-001\cores
   [junit4]   2> 695242 INFO  (SocketProxy-Acceptor-60709) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=60725,localport=60709], receiveBufferSize:65536
   [junit4]   2> 695242 INFO  (SocketProxy-Acceptor-60709) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=60710,localport=60726], receiveBufferSize=65536
   [junit4]   2> 695244 INFO  (qtp2054179248-6697) [n:127.0.0.1:60709_ltx    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:60709_ltx&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 695247 INFO  (OverseerThreadFactory-2624-thread-4-processing-n:127.0.0.1:60622_ltx) [n:127.0.0.1:60622_ltx    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:60709_ltx for creating new replica
   [junit4]   2> 695250 INFO  (SocketProxy-Acceptor-60709) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=60727,localport=60709], receiveBufferSize:65536
   [junit4]   2> 695250 INFO  (SocketProxy-Acceptor-60709) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=60710,localport=60728], receiveBufferSize=65536
   [junit4]   2> 695250 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 695251 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 695359 INFO  (zkCallback-1099-thread-1-processing-n:127.0.0.1:60709_ltx) [n:127.0.0.1:60709_ltx    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 695360 INFO  (zkCallback-1093-thread-1-processing-n:127.0.0.1:60683_ltx) [n:127.0.0.1:60683_ltx    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 696285 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 696299 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.s.IndexSchema [collection1_shard1_replica_t43] Schema name=test
   [junit4]   2> 696434 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 696463 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_t43' using configuration from collection collection1, trusted=true
   [junit4]   2> 696463 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_t43' (registry 'solr.core.collection1.shard1.replica_t43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1
   [junit4]   2> 696463 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 696465 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.c.SolrCore [[collection1_shard1_replica_t43] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\shard-2-001\cores\collection1_shard1_replica_t43], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\shard-2-001\cores\collection1_shard1_replica_t43\data\]
   [junit4]   2> 696466 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=30, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 696468 WARN  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 696551 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 696551 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 696554 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 696554 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 696555 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=97.5654296875, floorSegmentMB=1.01953125, forceMergeDeletesPctAllowed=11.836872422657862, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7217115328872558
   [junit4]   2> 696555 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.s.SolrIndexSearcher Opening [Searcher@67167f30[collection1_shard1_replica_t43] main]
   [junit4]   2> 696557 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 696558 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 696559 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 696559 INFO  (searcherExecutor-2649-thread-1-processing-n:127.0.0.1:60709_ltx x:collection1_shard1_replica_t43 s:shard1 c:collection1) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.c.SolrCore [collection1_shard1_replica_t43] Registered new searcher Searcher@67167f30[collection1_shard1_replica_t43] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 696559 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575732837110251520
   [junit4]   2> 696567 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 696567 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 696567 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:60709/ltx/collection1_shard1_replica_t43/
   [junit4]   2> 696567 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 696567 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.c.SyncStrategy http://127.0.0.1:60709/ltx/collection1_shard1_replica_t43/ has no replicas
   [junit4]   2> 696567 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 696567 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.c.ZkController collection1_shard1_replica_t43 stopping background replication from leader
   [junit4]   2> 696573 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:60709/ltx/collection1_shard1_replica_t43/ shard1
   [junit4]   2> 696677 INFO  (zkCallback-1093-thread-1-processing-n:127.0.0.1:60683_ltx) [n:127.0.0.1:60683_ltx    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 696677 INFO  (zkCallback-1099-thread-1-processing-n:127.0.0.1:60709_ltx) [n:127.0.0.1:60709_ltx    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 696724 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 696726 INFO  (qtp2054179248-6699) [n:127.0.0.1:60709_ltx c:collection1 s:shard1  x:collection1_shard1_replica_t43] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1475
   [junit4]   2> 696728 INFO  (qtp2054179248-6697) [n:127.0.0.1:60709_ltx    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:60709_ltx&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2} status=0 QTime=1484
   [junit4]   2> 696829 INFO  (zkCallback-1093-thread-1-processing-n:127.0.0.1:60683_ltx) [n:127.0.0.1:60683_ltx    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 696829 INFO  (zkCallback-1099-thread-1-processing-n:127.0.0.1:60709_ltx) [n:127.0.0.1:60709_ltx    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 697247 INFO  (OverseerCollectionConfigSetProcessor-98483301734744068-127.0.0.1:60622_ltx-n_0000000000) [n:127.0.0.1:60622_ltx    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 697527 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\shard-3-001 of type TLOG
   [junit4]   2> 697528 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 697529 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@558510ec{/ltx,null,AVAILABLE}
   [junit4]   2> 697530 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@42108c49{HTTP/1.1,[http/1.1]}{127.0.0.1:60790}
   [junit4]   2> 697530 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.e.j.s.Server Started @705588ms
   [junit4]   2> 697530 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\tempDir-001/jetty3, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/ltx, hostPort=60789, coreRootDirectory=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\shard-3-001\cores}
   [junit4]   2> 697530 ERROR (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 697530 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 697530 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 697530 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 697530 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-14T18:38:57.241Z
   [junit4]   2> 697600 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 697600 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\shard-3-001\solr.xml
   [junit4]   2> 697615 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 697622 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60612/solr
   [junit4]   2> 697671 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60789_ltx    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 697674 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60789_ltx    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 697695 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60789_ltx    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60789_ltx
   [junit4]   2> 697698 INFO  (zkCallback-1081-thread-1-processing-n:127.0.0.1:60622_ltx) [n:127.0.0.1:60622_ltx    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 697700 INFO  (zkCallback-1099-thread-1-processing-n:127.0.0.1:60709_ltx) [n:127.0.0.1:60709_ltx    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 697705 INFO  (zkCallback-1105-thread-1-processing-n:127.0.0.1:60789_ltx) [n:127.0.0.1:60789_ltx    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 697706 INFO  (zkCallback-1088-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 697706 INFO  (zkCallback-1093-thread-1-processing-n:127.0.0.1:60683_ltx) [n:127.0.0.1:60683_ltx    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 697826 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60789_ltx    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1
   [junit4]   2> 697837 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60789_ltx    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1
   [junit4]   2> 697838 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60789_ltx    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1
   [junit4]   2> 697839 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[7780120901B9DB52]) [n:127.0.0.1:60789_ltx    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\shard-3-001\cores
   [junit4]   2> 697899 INFO  (SocketProxy-Acceptor-60789) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=60802,localport=60789], receiveBufferSize:65536
   [junit4]   2> 697900 INFO  (SocketProxy-Acceptor-60789) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=60790,localport=60803], receiveBufferSize=65536
   [junit4]   2> 697903 INFO  (qtp615984946-6740) [n:127.0.0.1:60789_ltx    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:60789_ltx&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 697907 INFO  (OverseerThreadFactory-2624-thread-5-processing-n:127.0.0.1:60622_ltx) [n:127.0.0.1:60622_ltx    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:60789_ltx for creating new replica
   [junit4]   2> 697910 INFO  (SocketProxy-Acceptor-60789) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=60804,localport=60789], receiveBufferSize:65536
   [junit4]   2> 697910 INFO  (SocketProxy-Acceptor-60789) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=60790,localport=60805], receiveBufferSize=65536
   [junit4]   2> 697911 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 697912 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 698024 INFO  (zkCallback-1093-thread-1-processing-n:127.0.0.1:60683_ltx) [n:127.0.0.1:60683_ltx    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 698024 INFO  (zkCallback-1099-thread-1-processing-n:127.0.0.1:60709_ltx) [n:127.0.0.1:60709_ltx    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 698024 INFO  (zkCallback-1105-thread-1-processing-n:127.0.0.1:60789_ltx) [n:127.0.0.1:60789_ltx    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 698960 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 699031 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.s.IndexSchema [collection1_shard2_replica_t45] Schema name=test
   [junit4]   2> 699180 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 699217 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t45' using configuration from collection collection1, trusted=true
   [junit4]   2> 699217 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_t45' (registry 'solr.core.collection1.shard2.replica_t45') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2767e4c1
   [junit4]   2> 699218 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 699218 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.c.SolrCore [[collection1_shard2_replica_t45] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\shard-3-001\cores\collection1_shard2_replica_t45], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.ForceLeaderTest_7780120901B9DB52-001\shard-3-001\cores\collection1_shard2_replica_t45\data\]
   [junit4]   2> 699221 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=30, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 699223 WARN  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 699332 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 699332 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 699333 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 699333 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 699333 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=97.5654296875, floorSegmentMB=1.01953125, forceMergeDeletesPctAllowed=11.836872422657862, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7217115328872558
   [junit4]   2> 699333 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.s.SolrIndexSearcher Opening [Searcher@cc75944[collection1_shard2_replica_t45] main]
   [junit4]   2> 699345 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 699346 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 699347 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 699347 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575732840033681408
   [junit4]   2> 699348 INFO  (searcherExecutor-2660-thread-1-processing-n:127.0.0.1:60789_ltx x:collection1_shard2_replica_t45 s:shard2 c:collection1) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.c.SolrCore [collection1_shard2_replica_t45] Registered new searcher Searcher@cc75944[collection1_shard2_replica_t45] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 699360 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.c.ZkController Core needs to recover:collection1_shard2_replica_t45
   [junit4]   2> 699360 INFO  (updateExecutor-1102-thread-1-processing-n:127.0.0.1:60789_ltx x:collection1_shard2_replica_t45 s:shard2 c:collection1) [n:127.0.0.1:60789_ltx c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_t45] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 699361 INFO  (qtp615984946-6736) [n:127.0.0.1:60789_ltx c:collection1 s:shard2  x:collection1_shard2_replica_t45] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1449
   [junit4]   2> 699361 INFO  (recoveryExecutor-1103-thread-1-processing-n:127.0.0.1:60789_ltx x:collection1_shard2_replica_t45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:60789_ltx c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_t45] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 699362 INFO  (recoveryExecutor-1103-thread-1-processing-n:127.0.0.1:60789_ltx x:collection1_shard2_replica_t45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:60789_ltx c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_t45] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
   [junit4]   2> 699362 INFO  (recoveryExecutor-1103-thread-1-processing-n:127.0.0.1:60789_lt

[...truncated too long message...]

llection.testasynccollectioncreation.shard1_1.leader, tag=964136459
   [junit4]   2> 2054613 INFO  (coreCloseExecutor-8286-thread-2) [n:127.0.0.1:54282_solr c:testasynccollectioncreation s:shard1_0 r:core_node7 x:testasynccollectioncreation_shard1_0_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testasynccollectioncreation.shard1_0.replica_n5, tag=1534925933
   [junit4]   2> 2054613 INFO  (coreCloseExecutor-8286-thread-2) [n:127.0.0.1:54282_solr c:testasynccollectioncreation s:shard1_0 r:core_node7 x:testasynccollectioncreation_shard1_0_replica_n5] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@52f17383: rootName = solr_54282, domain = solr.core.testasynccollectioncreation.shard1_0.replica_n5, service url = null, agent id = null] for registry solr.core.testasynccollectioncreation.shard1_0.replica_n5 / null
   [junit4]   2> 2054613 INFO  (coreCloseExecutor-8286-thread-2) [n:127.0.0.1:54282_solr c:testasynccollectioncreation s:shard1_0 r:core_node7 x:testasynccollectioncreation_shard1_0_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testasynccollectioncreation.shard1_0.leader, tag=1534925933
   [junit4]   2> 2054617 WARN  (zkCallback-2903-thread-2-processing-n:127.0.0.1:54282_solr) [n:127.0.0.1:54282_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 2054617 INFO  (jetty-closer-2892-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4f0035a4{/solr,null,UNAVAILABLE}
   [junit4]   2> 2054618 ERROR (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[7780120901B9DB52]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2054618 INFO  (SUITE-CollectionsAPIAsyncDistributedZkTest-seed#[7780120901B9DB52]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:54278 54278
   [junit4]   2> 2054620 INFO  (Thread-4086) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:54278 54278
   [junit4]   2> 2054621 WARN  (Thread-4086) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/aliases.json
   [junit4]   2> 2 /solr/security.json
   [junit4]   2> 2 /solr/configs/conf1
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 48 /solr/collections/testasynccollectioncreation/state.json
   [junit4]   2> 34 /solr/collections/testAsyncOperations/state.json
   [junit4]   2> 3 /solr/clusterstate.json
   [junit4]   2> 3 /solr/clusterprops.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/live_nodes
   [junit4]   2> 3 /solr/collections
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CollectionsAPIAsyncDistributedZkTest_7780120901B9DB52-001
   [junit4]   2> Aug 14, 2017 7:01:34 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 3 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {id=FST50, _route_=FSTOrd50}, docValues:{}, maxPointsInLeafNode=342, maxMBSortInHeap=5.466455733489379, sim=RandomSimilarity(queryNorm=false): {}, locale=es-SV, timezone=Europe/Tallinn
   [junit4]   2> NOTE: Windows 10 10.0 amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=3,threads=1,free=287663664,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [V2ApiIntegrationTest, ImplicitSnitchTest, TestAddFieldRealTimeGet, ChaosMonkeySafeLeaderTest, TestWordDelimiterFilterFactory, OverseerModifyCollectionTest, LeaderElectionIntegrationTest, DocumentBuilderTest, TestRealTimeGet, ZkSolrClientTest, EchoParamsTest, TestMacros, PingRequestHandlerTest, SecurityConfHandlerTest, SolrCmdDistributorTest, HdfsUnloadDistributedZkTest, CdcrVersionReplicationTest, CleanupOldIndexTest, TestXmlQParserPlugin, PeerSyncWithIndexFingerprintCachingTest, LeaderFailoverAfterPartitionTest, CdcrReplicationHandlerTest, BadComponentTest, DistributedExpandComponentTest, TestSolrCoreParser, BlockJoinFacetRandomTest, SuggesterFSTTest, AsyncCallRequestStatusResponseTest, DocValuesMissingTest, TestConfigSetsAPIExclusivity, PluginInfoTest, SolrMetricReporterTest, RankQueryTest, TestConfigSetImmutable, CheckHdfsIndexTest, SchemaVersionSpecificBehaviorTest, TestTrieFacet, TestTolerantUpdateProcessorRandomCloud, TestLegacyTerms, TermsComponentTest, TestBlendedInfixSuggestions, OverriddenZkACLAndCredentialsProvidersTest, TestLMJelinekMercerSimilarityFactory, DocExpirationUpdateProcessorFactoryTest, TestConfigSets, TestStreamBody, TestSuggestSpellingConverter, HdfsDirectoryFactoryTest, ShowFileRequestHandlerTest, UUIDUpdateProcessorFallbackTest, TestSchemaSimilarityResource, TestRebalanceLeaders, TestPivotHelperCode, TestDocSet, TestCustomSort, CloudMLTQParserTest, TestCoreAdminApis, AutoScalingHandlerTest, MigrateRouteKeyTest, FullSolrCloudDistribCmdsTest, TestNoOpRegenerator, TestRTimerTree, TestDistributedGrouping, OpenExchangeRatesOrgProviderTest, TestDocumentBuilder, TestAnalyzedSuggestions, SimpleCollectionCreateDeleteTest, TestSearcherReuse, TestHdfsUpdateLog, TestCloudSchemaless, PKIAuthenticationIntegrationTest, SpellCheckComponentTest, DistribCursorPagingTest, TestSolrCoreSnapshots, TestNumericRangeQuery32, DistributedMLTComponentTest, JavabinLoaderTest, QueryEqualityTest, TestBulkSchemaAPI, CollectionTooManyReplicasTest, JsonLoaderTest, TestFieldCacheVsDocValues, DocumentAnalysisRequestHandlerTest, TestFilteredDocIdSet, TestQueryUtils, TestFastWriter, SolrShardReporterTest, TestLockTree, HdfsCollectionsAPIDistributedZkTest, ForceLeaderTest, SearchHandlerTest, CreateCollectionCleanupTest, FileBasedSpellCheckerTest, BadIndexSchemaTest, TestTrie, TestPostingsSolrHighlighter, TestReload, TestFieldCacheSort, TestSubQueryTransformer, CurrencyFieldTypeTest, DOMUtilTest, TestSolrIndexConfig, TestManagedSchemaAPI, TestCloudInspectUtil, DistributedFacetPivotSmallAdvancedTest, SuggesterTSTTest, TestLeaderElectionWithEmptyReplica, TestRandomFlRTGCloud, TermVectorComponentDistributedTest, TestRawResponseWriter, DistributedQueryElevationComponentTest, MoveReplicaHDFSTest, TestPerFieldSimilarity, OpenCloseCoreStressTest, TestConfigReload, TestHashQParserPlugin, AutoCommitTest, TestImplicitCoreProperties, TestClassicSimilarityFactory, TestSSLRandomization, AnalyticsQueryTest, FacetPivotSmallTest, ZkNodePropsTest, DistribJoinFromCollectionTest, RollingRestartTest, CollectionReloadTest, JvmMetricsTest, TestSQLHandlerNonCloud, TestNumericTokenStream, TestRecovery, TestNumericTerms64, LoggingHandlerTest, TestCoreDiscovery, TestDocBasedVersionConstraints, TestSolrConfigHandlerConcurrent, StatsComponentTest, TestCloudRecovery, SolrGangliaReporterTest, DirectSolrSpellCheckerTest, TestZkChroot, TestExclusionRuleCollectionAccess, TestManagedStopFilterFactory, HdfsRestartWhileUpdatingTest, SolrJmxReporterCloudTest, BJQParserTest, AssignBackwardCompatibilityTest, OverseerStatusTest, TestLegacyNumericRangeQueryBuilder, HttpPartitionTest, DistributedFacetPivotWhiteBoxTest, ManagedSchemaRoundRobinCloudTest, SharedFSAutoReplicaFailoverTest, MoveReplicaTest, BigEndianAscendingWordSerializerTest, TestCustomStream, TestInPlaceUpdatesStandalone, TestSolr4Spatial, PeerSyncReplicationTest, TestMaxTokenLenTokenizer, BooleanFieldTest, TestQueryWrapperFilter, TestPKIAuthenticationPlugin, TestFreeTextSuggestions, ZkStateReaderTest, TestBulkSchemaConcurrent, TestMissingGroups, TestFastLRUCache, ExternalFileFieldSortTest, TestReplicationHandlerBackup, TestSolrDeletionPolicy1, TestShortCircuitedRequests, SOLR749Test, MoreLikeThisHandlerTest, TestUnifiedSolrHighlighter, ChaosMonkeySafeLeaderWithPullReplicasTest, BasicDistributedZkTest, BasicDistributedZk2Test, SyncSliceTest, ShardRoutingTest, RecoveryZkTest, TestRandomFaceting, TestRandomDVFaceting, ZkCLITest, ShardRoutingCustomTest, TestFaceting, TestJoin, HardAutoCommitTest, DistributedTermsComponentTest, TestCoreContainer, TestGroupingSearch, QueryElevationComponentTest, TestFunctionQuery, TestBadConfig, SoftAutoCommitTest, SimplePostToolTest, AnalysisAfterCoreReloadTest, TestExtendedDismaxParser, CoreAdminHandlerTest, TestStressLucene, SpatialFilterTest, SuggesterWFSTTest, TestCSVLoader, PolyFieldTest, NoCacheHeaderTest, SolrCoreCheckLockOnStartupTest, FieldMutatingUpdateProcessorTest, DirectUpdateHandlerOptimizeTest, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, DocValuesMultiTest, DebugComponentTest, IndexBasedSpellCheckerTest, DisMaxRequestHandlerTest, TestQueryTypes, PrimitiveFieldTypeTest, TestOmitPositions, TestValueSourceCache, TermVectorComponentTest, TestSolrQueryParser, IndexSchemaRuntimeFieldTest, SolrPluginUtilsTest, UpdateRequestProcessorFactoryTest, MBeansHandlerTest, TestPartialUpdateDeduplication, UpdateParamsTest, SolrIndexConfigTest, MultiTermTest, SampleTest, TestBinaryField, TestSearchPerf, TestConfig, TestFuzzyAnalyzedSuggestions, TestLuceneMatchVersion, SynonymTokenizerTest, TestSweetSpotSimilarityFactory, TestBM25SimilarityFactory, TestFastOutputStream, PreAnalyzedFieldTest, RAMDirectoryFactoryTest, TestLRUCache, SystemInfoHandlerTest, CircularListTest, CursorPagingTest, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestSimpleTrackingShardHandler, AliasIntegrationTest, CdcrReplicationDistributedZkTest, CdcrRequestHandlerTest, CollectionsAPIAsyncDistributedZkTest]
   [junit4] Completed [499/730 (2!)] on J1 in 70.43s, 2 tests, 1 failure <<< FAILURES!

[...truncated 44047 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-master-Windows (32bit/jdk1.8.0_144) - Build # 6824 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6824/
Java: 32bit/jdk1.8.0_144 -server -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI

Error Message:
Something is broken in the assert for no shards using the same indexDir - probably something was changed in the attributes published in the MBean of SolrCore : {}

Stack Trace:
java.lang.AssertionError: Something is broken in the assert for no shards using the same indexDir - probably something was changed in the attributes published in the MBean of SolrCore : {}
        at __randomizedtesting.SeedInfo.seed([60CD8E0C8B9EC242:28B8FAB88DADEDD7]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.checkNoTwoShardsUseTheSameIndexDir(CollectionsAPIDistributedZkTest.java:581)
        at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:462)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 11772 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CollectionsAPIDistributedZkTest_60CD8E0C8B9EC242-001\init-core-data-001
   [junit4]   2> 1232100 WARN  (SUITE-CollectionsAPIDistributedZkTest-seed#[60CD8E0C8B9EC242]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=7 numCloses=7
   [junit4]   2> 1232100 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[60CD8E0C8B9EC242]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1232104 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[60CD8E0C8B9EC242]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 1232104 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[60CD8E0C8B9EC242]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1232106 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[60CD8E0C8B9EC242]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CollectionsAPIDistributedZkTest_60CD8E0C8B9EC242-001\tempDir-001
   [junit4]   2> 1232106 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[60CD8E0C8B9EC242]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1232107 INFO  (Thread-2268) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1232107 INFO  (Thread-2268) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1232111 ERROR (Thread-2268) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1232208 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[60CD8E0C8B9EC242]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:61343
   [junit4]   2> 1232216 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15de3caea690000, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1232220 INFO  (jetty-launcher-1698-thread-2) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1232221 INFO  (jetty-launcher-1698-thread-1) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1232222 INFO  (jetty-launcher-1698-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@377b23{/solr,null,AVAILABLE}
   [junit4]   2> 1232222 INFO  (jetty-launcher-1698-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@290be8{SSL,[ssl, http/1.1]}{127.0.0.1:61347}
   [junit4]   2> 1232223 INFO  (jetty-launcher-1698-thread-2) [    ] o.e.j.s.Server Started @1240071ms
   [junit4]   2> 1232223 INFO  (jetty-launcher-1698-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=61347}
   [junit4]   2> 1232223 ERROR (jetty-launcher-1698-thread-2) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1232223 INFO  (jetty-launcher-1698-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1232223 INFO  (jetty-launcher-1698-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1232223 INFO  (jetty-launcher-1698-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1232223 INFO  (jetty-launcher-1698-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-15T02:48:32.729Z
   [junit4]   2> 1232224 INFO  (jetty-launcher-1698-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@e7e271{/solr,null,AVAILABLE}
   [junit4]   2> 1232225 INFO  (jetty-launcher-1698-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@184e489{SSL,[ssl, http/1.1]}{127.0.0.1:61352}
   [junit4]   2> 1232225 INFO  (jetty-launcher-1698-thread-1) [    ] o.e.j.s.Server Started @1240073ms
   [junit4]   2> 1232225 INFO  (jetty-launcher-1698-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=61352}
   [junit4]   2> 1232225 ERROR (jetty-launcher-1698-thread-1) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1232225 INFO  (jetty-launcher-1698-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1232225 INFO  (jetty-launcher-1698-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1232225 INFO  (jetty-launcher-1698-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1232225 INFO  (jetty-launcher-1698-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-15T02:48:32.731Z
   [junit4]   2> 1232228 INFO  (jetty-launcher-1698-thread-3) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1232228 INFO  (jetty-launcher-1698-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@17cf528{/solr,null,AVAILABLE}
   [junit4]   2> 1232229 INFO  (jetty-launcher-1698-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@85dacd{SSL,[ssl, http/1.1]}{127.0.0.1:61357}
   [junit4]   2> 1232229 INFO  (jetty-launcher-1698-thread-3) [    ] o.e.j.s.Server Started @1240077ms
   [junit4]   2> 1232229 INFO  (jetty-launcher-1698-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=61357}
   [junit4]   2> 1232229 ERROR (jetty-launcher-1698-thread-3) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1232229 INFO  (jetty-launcher-1698-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1232229 INFO  (jetty-launcher-1698-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1232229 INFO  (jetty-launcher-1698-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1232229 INFO  (jetty-launcher-1698-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-15T02:48:32.735Z
   [junit4]   2> 1232232 INFO  (jetty-launcher-1698-thread-4) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1232232 INFO  (jetty-launcher-1698-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1eb3cb6{/solr,null,AVAILABLE}
   [junit4]   2> 1232233 INFO  (jetty-launcher-1698-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@137536a{SSL,[ssl, http/1.1]}{127.0.0.1:61362}
   [junit4]   2> 1232233 INFO  (jetty-launcher-1698-thread-4) [    ] o.e.j.s.Server Started @1240081ms
   [junit4]   2> 1232233 INFO  (jetty-launcher-1698-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=61362}
   [junit4]   2> 1232233 ERROR (jetty-launcher-1698-thread-4) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1232233 INFO  (jetty-launcher-1698-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1232233 INFO  (jetty-launcher-1698-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1232233 INFO  (jetty-launcher-1698-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1232233 INFO  (jetty-launcher-1698-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-15T02:48:32.739Z
   [junit4]   2> 1232250 INFO  (jetty-launcher-1698-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1232257 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15de3caea690003, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1232261 INFO  (jetty-launcher-1698-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:61343/solr
   [junit4]   2> 1232263 INFO  (jetty-launcher-1698-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1232270 INFO  (jetty-launcher-1698-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:61343/solr
   [junit4]   2> 1232273 INFO  (jetty-launcher-1698-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1232280 INFO  (jetty-launcher-1698-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:61343/solr
   [junit4]   2> 1232282 INFO  (jetty-launcher-1698-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1232289 INFO  (jetty-launcher-1698-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:61343/solr
   [junit4]   2> 1232487 INFO  (jetty-launcher-1698-thread-4) [n:127.0.0.1:61362_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1232487 INFO  (jetty-launcher-1698-thread-1) [n:127.0.0.1:61352_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1232487 INFO  (jetty-launcher-1698-thread-2) [n:127.0.0.1:61347_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1232487 INFO  (jetty-launcher-1698-thread-3) [n:127.0.0.1:61357_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1232488 INFO  (jetty-launcher-1698-thread-4) [n:127.0.0.1:61362_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:61362_solr
   [junit4]   2> 1232489 INFO  (jetty-launcher-1698-thread-1) [n:127.0.0.1:61352_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61352_solr
   [junit4]   2> 1232489 INFO  (jetty-launcher-1698-thread-2) [n:127.0.0.1:61347_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61347_solr
   [junit4]   2> 1232489 INFO  (jetty-launcher-1698-thread-3) [n:127.0.0.1:61357_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61357_solr
   [junit4]   2> 1232490 INFO  (jetty-launcher-1698-thread-4) [n:127.0.0.1:61362_solr    ] o.a.s.c.Overseer Overseer (id=98485227527667724-127.0.0.1:61362_solr-n_0000000000) starting
   [junit4]   2> 1232491 INFO  (zkCallback-1714-thread-1-processing-n:127.0.0.1:61352_solr) [n:127.0.0.1:61352_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1232492 INFO  (zkCallback-1719-thread-1-processing-n:127.0.0.1:61357_solr) [n:127.0.0.1:61357_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1232492 INFO  (zkCallback-1716-thread-1-processing-n:127.0.0.1:61347_solr) [n:127.0.0.1:61347_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1232593 INFO  (zkCallback-1720-thread-1-processing-n:127.0.0.1:61362_solr) [n:127.0.0.1:61362_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1232604 WARN  (jetty-launcher-1698-thread-1) [n:127.0.0.1:61352_solr    ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 1232604 WARN  (jetty-launcher-1698-thread-1) [n:127.0.0.1:61352_solr    ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 1232604 WARN  (jetty-launcher-1698-thread-1) [n:127.0.0.1:61352_solr    ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 1232605 INFO  (jetty-launcher-1698-thread-1) [n:127.0.0.1:61352_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CollectionsAPIDistributedZkTest_60CD8E0C8B9EC242-001\tempDir-001\node2
   [junit4]   2> 1232620 INFO  (jetty-launcher-1698-thread-4) [n:127.0.0.1:61362_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61362_solr
   [junit4]   2> 1232621 WARN  (jetty-launcher-1698-thread-3) [n:127.0.0.1:61357_solr    ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 1232621 WARN  (jetty-launcher-1698-thread-3) [n:127.0.0.1:61357_solr    ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 1232621 WARN  (jetty-launcher-1698-thread-3) [n:127.0.0.1:61357_solr    ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 1232623 INFO  (jetty-launcher-1698-thread-3) [n:127.0.0.1:61357_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CollectionsAPIDistributedZkTest_60CD8E0C8B9EC242-001\tempDir-001\node3
   [junit4]   2> 1232623 INFO  (zkCallback-1714-thread-1-processing-n:127.0.0.1:61352_solr) [n:127.0.0.1:61352_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1232624 INFO  (zkCallback-1716-thread-1-processing-n:127.0.0.1:61347_solr) [n:127.0.0.1:61347_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1232624 INFO  (zkCallback-1719-thread-1-processing-n:127.0.0.1:61357_solr) [n:127.0.0.1:61357_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1232624 INFO  (zkCallback-1720-thread-1-processing-n:127.0.0.1:61362_solr) [n:127.0.0.1:61362_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1232693 WARN  (jetty-launcher-1698-thread-2) [n:127.0.0.1:61347_solr    ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 1232693 WARN  (jetty-launcher-1698-thread-2) [n:127.0.0.1:61347_solr    ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 1232693 WARN  (jetty-launcher-1698-thread-2) [n:127.0.0.1:61347_solr    ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 1232694 INFO  (jetty-launcher-1698-thread-2) [n:127.0.0.1:61347_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CollectionsAPIDistributedZkTest_60CD8E0C8B9EC242-001\tempDir-001\node1
   [junit4]   2> 1232737 WARN  (jetty-launcher-1698-thread-4) [n:127.0.0.1:61362_solr    ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 1232737 WARN  (jetty-launcher-1698-thread-4) [n:127.0.0.1:61362_solr    ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 1232737 WARN  (jetty-launcher-1698-thread-4) [n:127.0.0.1:61362_solr    ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 1232737 INFO  (jetty-launcher-1698-thread-4) [n:127.0.0.1:61362_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CollectionsAPIDistributedZkTest_60CD8E0C8B9EC242-001\tempDir-001\node4
   [junit4]   2> 1232753 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[60CD8E0C8B9EC242]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1232754 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[60CD8E0C8B9EC242]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:61343/solr ready
   [junit4]   2> 1232780 INFO  (TEST-CollectionsAPIDistributedZkTest.deleteCollectionOnlyInZk-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.SolrTestCaseJ4 ###Starting deleteCollectionOnlyInZk
   [junit4]   2> 1232781 INFO  (TEST-CollectionsAPIDistributedZkTest.deleteCollectionOnlyInZk-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1232809 INFO  (qtp27335993-10924) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=onlyinzk&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1232816 INFO  (qtp27335993-10924) [n:127.0.0.1:61347_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=onlyinzk&action=DELETE&wt=javabin&version=2} status=0 QTime=6
   [junit4]   2> 1232817 INFO  (qtp27335993-10971) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1232817 INFO  (qtp27335993-10971) [n:127.0.0.1:61347_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1232817 INFO  (qtp27335993-10970) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=onlyinzk&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1232820 INFO  (OverseerCollectionConfigSetProcessor-98485227527667724-127.0.0.1:61362_solr-n_0000000000) [n:127.0.0.1:61362_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1232820 INFO  (OverseerThreadFactory-5021-thread-2-processing-n:127.0.0.1:61362_solr) [n:127.0.0.1:61362_solr    ] o.a.s.c.CreateCollectionCmd Create collection onlyinzk
   [junit4]   2> 1232927 INFO  (OverseerStateUpdate-98485227527667724-127.0.0.1:61362_solr-n_0000000000) [n:127.0.0.1:61362_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"onlyinzk",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"onlyinzk_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:61357/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1232929 INFO  (OverseerStateUpdate-98485227527667724-127.0.0.1:61362_solr-n_0000000000) [n:127.0.0.1:61362_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"onlyinzk",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"onlyinzk_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:61347/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1233137 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=onlyinzk_shard2_replica_n2&action=CREATE&numShards=2&collection=onlyinzk&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1233137 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1233158 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=onlyinzk_shard1_replica_n1&action=CREATE&numShards=2&collection=onlyinzk&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1233159 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1233264 INFO  (zkCallback-1716-thread-1-processing-n:127.0.0.1:61347_solr) [n:127.0.0.1:61347_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1233264 INFO  (zkCallback-1719-thread-1-processing-n:127.0.0.1:61357_solr) [n:127.0.0.1:61357_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1233266 INFO  (zkCallback-1719-thread-2-processing-n:127.0.0.1:61357_solr) [n:127.0.0.1:61357_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1233266 INFO  (zkCallback-1716-thread-2-processing-n:127.0.0.1:61347_solr) [n:127.0.0.1:61347_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1234160 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1234166 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.s.IndexSchema [onlyinzk_shard2_replica_n2] Schema name=minimal
   [junit4]   2> 1234170 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1234170 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'onlyinzk_shard2_replica_n2' using configuration from collection onlyinzk, trusted=true
   [junit4]   2> 1234170 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1234172 WARN  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 1234172 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1234172 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.SolrCore [[onlyinzk_shard2_replica_n2] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CollectionsAPIDistributedZkTest_60CD8E0C8B9EC242-001\tempDir-001\node1\onlyinzk_shard2_replica_n2], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CollectionsAPIDistributedZkTest_60CD8E0C8B9EC242-001\tempDir-001\node1\onlyinzk_shard2_replica_n2\data\]
   [junit4]   2> 1234178 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.s.IndexSchema [onlyinzk_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 1234180 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1234180 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'onlyinzk_shard1_replica_n1' using configuration from collection onlyinzk, trusted=true
   [junit4]   2> 1234180 WARN  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 1234180 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1234180 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.SolrCore [[onlyinzk_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CollectionsAPIDistributedZkTest_60CD8E0C8B9EC242-001\tempDir-001\node3\onlyinzk_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CollectionsAPIDistributedZkTest_60CD8E0C8B9EC242-001\tempDir-001\node3\onlyinzk_shard1_replica_n1\data\]
   [junit4]   2> 1234254 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1234254 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1234268 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1234268 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1234269 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1234269 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1234270 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1234270 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1234273 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@3a0e94[onlyinzk_shard2_replica_n2] main]
   [junit4]   2> 1234274 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@c420cc[onlyinzk_shard1_replica_n1] main]
   [junit4]   2> 1234274 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1234274 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1234275 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1234275 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1234275 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1234275 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1234276 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575763642712850432
   [junit4]   2> 1234276 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575763642712850432
   [junit4]   2> 1234276 INFO  (searcherExecutor-5026-thread-1-processing-n:127.0.0.1:61357_solr x:onlyinzk_shard1_replica_n1 s:shard1 c:onlyinzk r:core_node3) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.SolrCore [onlyinzk_shard1_replica_n1] Registered new searcher Searcher@c420cc[onlyinzk_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1234277 INFO  (searcherExecutor-5025-thread-1-processing-n:127.0.0.1:61347_solr x:onlyinzk_shard2_replica_n2 s:shard2 c:onlyinzk r:core_node4) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.SolrCore [onlyinzk_shard2_replica_n2] Registered new searcher Searcher@3a0e94[onlyinzk_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1234283 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1234283 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1234283 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:61347/solr/onlyinzk_shard2_replica_n2/
   [junit4]   2> 1234283 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1234283 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.SyncStrategy https://127.0.0.1:61347/solr/onlyinzk_shard2_replica_n2/ has no replicas
   [junit4]   2> 1234283 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1234283 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1234283 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1234283 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:61357/solr/onlyinzk_shard1_replica_n1/
   [junit4]   2> 1234283 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1234283 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:61357/solr/onlyinzk_shard1_replica_n1/ has no replicas
   [junit4]   2> 1234283 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1234287 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:61357/solr/onlyinzk_shard1_replica_n1/ shard1
   [junit4]   2> 1234288 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:61347/solr/onlyinzk_shard2_replica_n2/ shard2
   [junit4]   2> 1234391 INFO  (zkCallback-1716-thread-2-processing-n:127.0.0.1:61347_solr) [n:127.0.0.1:61347_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1234391 INFO  (zkCallback-1719-thread-2-processing-n:127.0.0.1:61357_solr) [n:127.0.0.1:61357_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1234391 INFO  (zkCallback-1719-thread-1-processing-n:127.0.0.1:61357_solr) [n:127.0.0.1:61357_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1234391 INFO  (zkCallback-1716-thread-1-processing-n:127.0.0.1:61347_solr) [n:127.0.0.1:61347_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1234440 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1234440 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1234442 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr c:onlyinzk s:shard2 r:core_node4 x:onlyinzk_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=onlyinzk_shard2_replica_n2&action=CREATE&numShards=2&collection=onlyinzk&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1306
   [junit4]   2> 1234447 INFO  (qtp14073861-10940) [n:127.0.0.1:61357_solr c:onlyinzk s:shard1 r:core_node3 x:onlyinzk_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=onlyinzk_shard1_replica_n1&action=CREATE&numShards=2&collection=onlyinzk&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1289
   [junit4]   2> 1234450 INFO  (qtp27335993-10970) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1234551 INFO  (zkCallback-1716-thread-1-processing-n:127.0.0.1:61347_solr) [n:127.0.0.1:61347_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1234551 INFO  (zkCallback-1716-thread-2-processing-n:127.0.0.1:61347_solr) [n:127.0.0.1:61347_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1234551 INFO  (zkCallback-1719-thread-2-processing-n:127.0.0.1:61357_solr) [n:127.0.0.1:61357_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1234551 INFO  (zkCallback-1719-thread-1-processing-n:127.0.0.1:61357_solr) [n:127.0.0.1:61357_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/onlyinzk/state.json] for collection [onlyinzk] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1234822 INFO  (OverseerCollectionConfigSetProcessor-98485227527667724-127.0.0.1:61362_solr-n_0000000000) [n:127.0.0.1:61362_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1235450 INFO  (qtp27335993-10970) [n:127.0.0.1:61347_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=onlyinzk&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2632
   [junit4]   2> 1235450 INFO  (qtp27335993-10926) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1235451 INFO  (qtp27335993-10926) [n:127.0.0.1:61347_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1235451 INFO  (TEST-CollectionsAPIDistributedZkTest.deleteCollectionOnlyInZk-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.SolrTestCaseJ4 ###Ending deleteCollectionOnlyInZk
   [junit4]   2> 1235479 INFO  (TEST-CollectionsAPIDistributedZkTest.deleteCollectionRemovesStaleZkCollectionsNode-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.SolrTestCaseJ4 ###Starting deleteCollectionRemovesStaleZkCollectionsNode
   [junit4]   2> 1235480 INFO  (TEST-CollectionsAPIDistributedZkTest.deleteCollectionRemovesStaleZkCollectionsNode-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1235481 INFO  (qtp27335993-10925) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=onlyinzk&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1235484 INFO  (OverseerThreadFactory-5021-thread-3-processing-n:127.0.0.1:61362_solr) [n:127.0.0.1:61362_solr    ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 1235486 INFO  (qtp14073861-10939) [n:127.0.0.1:61357_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.onlyinzk.shard1.replica_n1, tag=null
   [junit4]   2> 1235486 INFO  (qtp14073861-10939) [n:127.0.0.1:61357_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@12a093d: rootName = null, domain = solr.core.onlyinzk.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.onlyinzk.shard1.replica_n1 / null
   [junit4]   2> 1235486 INFO  (qtp14073861-10939) [n:127.0.0.1:61357_solr    ] o.a.s.c.SolrCore [onlyinzk_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1b3afe1
   [junit4]   2> 1235487 INFO  (qtp27335993-10924) [n:127.0.0.1:61347_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.onlyinzk.shard2.replica_n2, tag=null
   [junit4]   2> 1235487 INFO  (qtp27335993-10924) [n:127.0.0.1:61347_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@db7b75: rootName = null, domain = solr.core.onlyinzk.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.onlyinzk.shard2.replica_n2 / null
   [junit4]   2> 1235487 INFO  (qtp27335993-10924) [n:127.0.0.1:61347_solr    ] o.a.s.c.SolrCore [onlyinzk_shard2_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@162798d
   [junit4]   2> 1235488 INFO  (qtp14073861-10939) [n:127.0.0.1:61357_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.onlyinzk.shard1.replica_n1, tag=28553185
   [junit4]   2> 1235488 INFO  (qtp14073861-10939) [n:127.0.0.1:61357_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.onlyinzk.shard1.leader, tag=28553185
   [junit4]   2> 1235488 INFO  (qtp27335993-10924) [n:127.0.0.1:61347_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.onlyinzk.shard2.replica_n2, tag=23230861
   [junit4]   2> 1235488 INFO  (qtp27335993-10924) [n:127.0.0.1:61347_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.onlyinzk.shard2.leader, tag=23230861
   [junit4]   2> 1235491 INFO  (qtp14073861-10939) [n:127.0.0.1:61357_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=onlyinzk_shard1_replica_n1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=6
   [junit4]   2> 1235492 INFO  (qtp27335993-10924) [n:127.0.0.1:61347_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=onlyinzk_shard2_replica_n2&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=6
   [junit4]   2> 1236197 INFO  (qtp27335993-10925) [n:127.0.0.1:61347_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=onlyinzk&action=DELETE&wt=javabin&version=2} status=0 QTime=715
   [junit4]   2> 1236199 INFO  (qtp27335993-10971) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=out_of_sync_collection&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1236200 INFO  (OverseerCollectionConfigSetProcessor-98485227527667724-127.0.0.1:61362_solr-n_0000000000) [n:127.0.0.1:61362_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1236203 INFO  (qtp27335993-10971) [n:127.0.0.1:61347_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=out_of_sync_collection&action=DELETE&wt=javabin&version=2} status=0 QTime=4
   [junit4]   2> 1236204 INFO  (qtp27335993-10972) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1236204 INFO  (qtp27335993-10972) [n:127.0.0.1:61347_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1236204 INFO  (TEST-CollectionsAPIDistributedZkTest.deleteCollectionRemovesStaleZkCollectionsNode-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.SolrTestCaseJ4 ###Ending deleteCollectionRemovesStaleZkCollectionsNode
   [junit4]   2> 1236227 INFO  (TEST-CollectionsAPIDistributedZkTest.testSpecificConfigsets-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSpecificConfigsets
   [junit4]   2> 1236228 INFO  (TEST-CollectionsAPIDistributedZkTest.testSpecificConfigsets-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1236229 INFO  (qtp27335993-10931) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf2&name=withconfigset2&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1236232 INFO  (OverseerCollectionConfigSetProcessor-98485227527667724-127.0.0.1:61362_solr-n_0000000000) [n:127.0.0.1:61362_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1236232 INFO  (OverseerThreadFactory-5021-thread-5-processing-n:127.0.0.1:61362_solr) [n:127.0.0.1:61362_solr    ] o.a.s.c.CreateCollectionCmd Create collection withconfigset2
   [junit4]   2> 1236442 INFO  (OverseerStateUpdate-98485227527667724-127.0.0.1:61362_solr-n_0000000000) [n:127.0.0.1:61362_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"withconfigset2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"withconfigset2_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:61352/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1236680 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf2&newCollection=true&name=withconfigset2_shard1_replica_n1&action=CREATE&numShards=1&collection=withconfigset2&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1236681 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1236789 INFO  (zkCallback-1714-thread-1-processing-n:127.0.0.1:61352_solr) [n:127.0.0.1:61352_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withconfigset2/state.json] for collection [withconfigset2] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1236790 INFO  (zkCallback-1714-thread-2-processing-n:127.0.0.1:61352_solr) [n:127.0.0.1:61352_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withconfigset2/state.json] for collection [withconfigset2] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1237700 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1237704 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.s.IndexSchema [withconfigset2_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 1237708 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1237708 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'withconfigset2_shard1_replica_n1' using configuration from collection withconfigset2, trusted=true
   [junit4]   2> 1237708 WARN  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 1237708 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1237708 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.c.SolrCore [[withconfigset2_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CollectionsAPIDistributedZkTest_60CD8E0C8B9EC242-001\tempDir-001\node2\withconfigset2_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CollectionsAPIDistributedZkTest_60CD8E0C8B9EC242-001\tempDir-001\node2\withconfigset2_shard1_replica_n1\data\]
   [junit4]   2> 1237785 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1237785 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1237787 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1237787 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1237788 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@16fabd8[withconfigset2_shard1_replica_n1] main]
   [junit4]   2> 1237790 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf2
   [junit4]   2> 1237790 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf2
   [junit4]   2> 1237790 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1237790 INFO  (searcherExecutor-5035-thread-1-processing-n:127.0.0.1:61352_solr x:withconfigset2_shard1_replica_n1 s:shard1 c:withconfigset2 r:core_node2) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.c.SolrCore [withconfigset2_shard1_replica_n1] Registered new searcher Searcher@16fabd8[withconfigset2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1237791 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575763646398595072
   [junit4]   2> 1237795 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1237795 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1237795 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:61352/solr/withconfigset2_shard1_replica_n1/
   [junit4]   2> 1237795 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1237795 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:61352/solr/withconfigset2_shard1_replica_n1/ has no replicas
   [junit4]   2> 1237795 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1237798 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:61352/solr/withconfigset2_shard1_replica_n1/ shard1
   [junit4]   2> 1237903 INFO  (zkCallback-1714-thread-2-processing-n:127.0.0.1:61352_solr) [n:127.0.0.1:61352_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withconfigset2/state.json] for collection [withconfigset2] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1237904 INFO  (zkCallback-1714-thread-1-processing-n:127.0.0.1:61352_solr) [n:127.0.0.1:61352_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withconfigset2/state.json] for collection [withconfigset2] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1237950 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1237951 INFO  (qtp13661702-10922) [n:127.0.0.1:61352_solr c:withconfigset2 s:shard1 r:core_node2 x:withconfigset2_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf2&newCollection=true&name=withconfigset2_shard1_replica_n1&action=CREATE&numShards=1&collection=withconfigset2&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1271
   [junit4]   2> 1237953 INFO  (qtp27335993-10931) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1238055 INFO  (zkCallback-1714-thread-1-processing-n:127.0.0.1:61352_solr) [n:127.0.0.1:61352_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withconfigset2/state.json] for collection [withconfigset2] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1238055 INFO  (zkCallback-1714-thread-2-processing-n:127.0.0.1:61352_solr) [n:127.0.0.1:61352_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withconfigset2/state.json] for collection [withconfigset2] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1238234 INFO  (OverseerCollectionConfigSetProcessor-98485227527667724-127.0.0.1:61362_solr-n_0000000000) [n:127.0.0.1:61362_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000008 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1238955 INFO  (qtp27335993-10931) [n:127.0.0.1:61347_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf2&name=withconfigset2&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=2726
   [junit4]   2> 1238956 INFO  (TEST-CollectionsAPIDistributedZkTest.testSpecificConfigsets-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testSpecificConfigsets
   [junit4]   2> 1238980 INFO  (TEST-CollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testTooManyReplicas
   [junit4]   2> 1238981 INFO  (TEST-CollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1238983 INFO  (qtp27335993-10927) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=withconfigset2&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1238985 INFO  (OverseerThreadFactory-5021-thread-5-processing-n:127.0.0.1:61362_solr) [n:127.0.0.1:61362_solr    ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 1238987 INFO  (qtp13661702-10921) [n:127.0.0.1:61352_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.withconfigset2.shard1.replica_n1, tag=null
   [junit4]   2> 1238987 INFO  (qtp13661702-10921) [n:127.0.0.1:61352_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1449d9c: rootName = null, domain = solr.core.withconfigset2.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.withconfigset2.shard1.replica_n1 / null
   [junit4]   2> 1238987 INFO  (qtp13661702-10921) [n:127.0.0.1:61352_solr    ] o.a.s.c.SolrCore [withconfigset2_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@11e3d41
   [junit4]   2> 1238988 INFO  (qtp13661702-10921) [n:127.0.0.1:61352_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.withconfigset2.shard1.replica_n1, tag=18758977
   [junit4]   2> 1238988 INFO  (qtp13661702-10921) [n:127.0.0.1:61352_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.withconfigset2.shard1.leader, tag=18758977
   [junit4]   2> 1238992 INFO  (qtp13661702-10921) [n:127.0.0.1:61352_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=withconfigset2_shard1_replica_n1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=6
   [junit4]   2> 1239697 INFO  (qtp27335993-10927) [n:127.0.0.1:61347_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=withconfigset2&action=DELETE&wt=javabin&version=2} status=0 QTime=713
   [junit4]   2> 1239698 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=10&collection.configName=conf&name=collection&nrtReplicas=10&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1239700 INFO  (OverseerThreadFactory-5021-thread-5-processing-n:127.0.0.1:61362_solr) [n:127.0.0.1:61362_solr    ] o.a.s.c.CreateCollectionCmd Create collection collection
   [junit4]   2> 1239700 WARN  (OverseerThreadFactory-5021-thread-5-processing-n:127.0.0.1:61362_solr) [n:127.0.0.1:61362_solr    ] o.a.s.c.CreateCollectionCmd Specified number of replicas of 10 on collection collection is higher than the number of Solr instances currently live or live and part of your createNodeSet(4). It's unusual to run two replica of the same slice on the same Solr-instance.
   [junit4]   2> 1239700 INFO  (OverseerCollectionConfigSetProcessor-98485227527667724-127.0.0.1:61362_solr-n_0000000000) [n:127.0.0.1:61362_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000010 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1239701 ERROR (OverseerThreadFactory-5021-thread-5-processing-n:127.0.0.1:61362_solr) [n:127.0.0.1:61362_solr    ] o.a.s.c.OverseerCollectionMessageHandler Collection: collection operation: create failed:org.apache.solr.common.SolrException: Cannot create collection collection. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 4. This allows a maximum of 4 to be created. Value of numShards is 2, value of nrtReplicas is 10, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 20 shards to be created (higher than the allowed number)
   [junit4]   2> at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:157)
   [junit4]   2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:227)
   [junit4]   2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:464)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>
   [junit4]   2> 1239702 INFO  (qtp27335993-10928) [n:127.0.0.1:61347_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=10&collection.configName=conf&name=collection&nrtReplicas=10&action=CREATE&numShards=2&wt=javabin&version=2} status=400 QTime=4
   [junit4]   2> 1239703 INFO  (TEST-CollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testTooManyReplicas
   [junit4]   2> 1239728 INFO  (TEST-CollectionsAPIDistributedZkTest.testCreationAndDeletion-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCreationAndDeletion
   [junit4]   2> 1239729 INFO  (TEST-CollectionsAPIDistributedZkTest.testCreationAndDeletion-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1239731 INFO  (qtp27335993-10970) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=created_and_deleted&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1239732 INFO  (OverseerThreadFactory-5021-thread-5-processing-n:127.0.0.1:61362_solr) [n:127.0.0.1:61362_solr    ] o.a.s.c.CreateCollectionCmd Create collection created_and_deleted
   [junit4]   2> 1239733 INFO  (OverseerCollectionConfigSetProcessor-98485227527667724-127.0.0.1:61362_solr-n_0000000000) [n:127.0.0.1:61362_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000012 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1239939 INFO  (OverseerStateUpdate-98485227527667724-127.0.0.1:61362_solr-n_0000000000) [n:127.0.0.1:61362_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"created_and_deleted",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"created_and_deleted_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:61352/solr",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 1240141 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=created_and_deleted_shard1_replica_n1&action=CREATE&numShards=1&collection=created_and_deleted&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1240248 INFO  (zkCallback-1714-thread-1-processing-n:127.0.0.1:61352_solr) [n:127.0.0.1:61352_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/created_and_deleted/state.json] for collection [created_and_deleted] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1240248 INFO  (zkCallback-1714-thread-2-processing-n:127.0.0.1:61352_solr) [n:127.0.0.1:61352_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/created_and_deleted/state.json] for collection [created_and_deleted] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1241161 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1241165 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.s.IndexSchema [created_and_deleted_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 1241168 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1241168 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'created_and_deleted_shard1_replica_n1' using configuration from collection created_and_deleted, trusted=true
   [junit4]   2> 1241170 WARN  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 1241170 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1241170 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.c.SolrCore [[created_and_deleted_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CollectionsAPIDistributedZkTest_60CD8E0C8B9EC242-001\tempDir-001\node2\created_and_deleted_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CollectionsAPIDistributedZkTest_60CD8E0C8B9EC242-001\tempDir-001\node2\created_and_deleted_shard1_replica_n1\data\]
   [junit4]   2> 1241230 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1241230 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1241231 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1241232 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1241232 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@14d8f94[created_and_deleted_shard1_replica_n1] main]
   [junit4]   2> 1241233 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1241234 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1241234 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1241234 INFO  (searcherExecutor-5040-thread-1-processing-n:127.0.0.1:61352_solr x:created_and_deleted_shard1_replica_n1 s:shard1 c:created_and_deleted r:core_node2) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.c.SolrCore [created_and_deleted_shard1_replica_n1] Registered new searcher Searcher@14d8f94[created_and_deleted_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1241234 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575763650008842240
   [junit4]   2> 1241239 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1241239 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1241239 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:61352/solr/created_and_deleted_shard1_replica_n1/
   [junit4]   2> 1241239 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1241239 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:61352/solr/created_and_deleted_shard1_replica_n1/ has no replicas
   [junit4]   2> 1241239 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1241242 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:61352/solr/created_and_deleted_shard1_replica_n1/ shard1
   [junit4]   2> 1241345 INFO  (zkCallback-1714-thread-1-processing-n:127.0.0.1:61352_solr) [n:127.0.0.1:61352_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/created_and_deleted/state.json] for collection [created_and_deleted] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1241345 INFO  (zkCallback-1714-thread-2-processing-n:127.0.0.1:61352_solr) [n:127.0.0.1:61352_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/created_and_deleted/state.json] for collection [created_and_deleted] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1241395 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1241396 INFO  (qtp13661702-10920) [n:127.0.0.1:61352_solr c:created_and_deleted s:shard1 r:core_node2 x:created_and_deleted_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=created_and_deleted_shard1_replica_n1&action=CREATE&numShards=1&collection=created_and_deleted&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1255
   [junit4]   2> 1241398 INFO  (qtp27335993-10970) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1241499 INFO  (zkCallback-1714-thread-1-processing-n:127.0.0.1:61352_solr) [n:127.0.0.1:61352_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/created_and_deleted/state.json] for collection [created_and_deleted] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1241499 INFO  (zkCallback-1714-thread-2-processing-n:127.0.0.1:61352_solr) [n:127.0.0.1:61352_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/created_and_deleted/state.json] for collection [created_and_deleted] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1241734 INFO  (OverseerCollectionConfigSetProcessor-98485227527667724-127.0.0.1:61362_solr-n_0000000000) [n:127.0.0.1:61362_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000014 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1242399 INFO  (qtp27335993-10970) [n:127.0.0.1:61347_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=created_and_deleted&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=2668
   [junit4]   2> 1242400 INFO  (qtp27335993-10926) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1242400 INFO  (qtp27335993-10926) [n:127.0.0.1:61347_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1242401 INFO  (qtp27335993-10924) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=created_and_deleted&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1242402 INFO  (OverseerThreadFactory-5021-thread-5-processing-n:127.0.0.1:61362_solr) [n:127.0.0.1:61362_solr    ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 1242403 INFO  (qtp13661702-10919) [n:127.0.0.1:61352_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.created_and_deleted.shard1.replica_n1, tag=null
   [junit4]   2> 1242403 INFO  (qtp13661702-10919) [n:127.0.0.1:61352_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@16d932: rootName = null, domain = solr.core.created_and_deleted.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.created_and_deleted.shard1.replica_n1 / null
   [junit4]   2> 1242404 INFO  (qtp13661702-10919) [n:127.0.0.1:61352_solr    ] o.a.s.c.SolrCore [created_and_deleted_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@196f6ff
   [junit4]   2> 1242405 INFO  (qtp13661702-10919) [n:127.0.0.1:61352_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.created_and_deleted.shard1.replica_n1, tag=26670847
   [junit4]   2> 1242405 INFO  (qtp13661702-10919) [n:127.0.0.1:61352_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.created_and_deleted.shard1.leader, tag=26670847
   [junit4]   2> 1242409 INFO  (qtp13661702-10919) [n:127.0.0.1:61352_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=created_and_deleted_shard1_replica_n1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=5
   [junit4]   2> 1243113 INFO  (qtp27335993-10924) [n:127.0.0.1:61347_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=created_and_deleted&action=DELETE&wt=javabin&version=2} status=0 QTime=712
   [junit4]   2> 1243114 INFO  (qtp27335993-10925) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1243114 INFO  (qtp27335993-10925) [n:127.0.0.1:61347_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1243115 INFO  (TEST-CollectionsAPIDistributedZkTest.testCreationAndDeletion-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testCreationAndDeletion
   [junit4]   2> 1243137 INFO  (TEST-CollectionsAPIDistributedZkTest.testZeroNumShards-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testZeroNumShards
   [junit4]   2> 1243138 INFO  (TEST-CollectionsAPIDistributedZkTest.testZeroNumShards-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1243140 INFO  (qtp27335993-10971) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=10&collection.configName=conf&name=acollection&action=CREATE&numShards=0&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1243141 INFO  (OverseerThreadFactory-5021-thread-5-processing-n:127.0.0.1:61362_solr) [n:127.0.0.1:61362_solr    ] o.a.s.c.CreateCollectionCmd Create collection acollection
   [junit4]   2> 1243142 ERROR (OverseerThreadFactory-5021-thread-5-processing-n:127.0.0.1:61362_solr) [n:127.0.0.1:61362_solr    ] o.a.s.c.OverseerCollectionMessageHandler Collection: acollection operation: create failed:org.apache.solr.common.SolrException: numShards must be > 0
   [junit4]   2> at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:127)
   [junit4]   2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:227)
   [junit4]   2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:464)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>
   [junit4]   2> 1243142 INFO  (OverseerCollectionConfigSetProcessor-98485227527667724-127.0.0.1:61362_solr-n_0000000000) [n:127.0.0.1:61362_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000016 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1243144 INFO  (qtp27335993-10971) [n:127.0.0.1:61347_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=10&collection.configName=conf&name=acollection&action=CREATE&numShards=0&wt=javabin&version=2} status=400 QTime=4
   [junit4]   2> 1243144 INFO  (TEST-CollectionsAPIDistributedZkTest.testZeroNumShards-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testZeroNumShards
   [junit4]   2> 1243168 INFO  (TEST-CollectionsAPIDistributedZkTest.testMaxNodesPerShard-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testMaxNodesPerShard
   [junit4]   2> 1243169 INFO  (TEST-CollectionsAPIDistributedZkTest.testMaxNodesPerShard-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1243170 INFO  (qtp27335993-10972) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&name=oversharded&nrtReplicas=2&action=CREATE&numShards=3&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1243172 INFO  (OverseerThreadFactory-5021-thread-5-processing-n:127.0.0.1:61362_solr) [n:127.0.0.1:61362_solr    ] o.a.s.c.CreateCollectionCmd Create collection oversharded
   [junit4]   2> 1243172 ERROR (OverseerThreadFactory-5021-thread-5-processing-n:127.0.0.1:61362_solr) [n:127.0.0.1:61362_solr    ] o.a.s.c.OverseerCollectionMessageHandler Collection: oversharded operation: create failed:org.apache.solr.common.SolrException: Cannot create collection oversharded. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 4. This allows a maximum of 4 to be created. Value of numShards is 3, value of nrtReplicas is 2, value of tlogReplicas is 0 and value of pullReplicas is 0. This requires 6 shards to be created (higher than the allowed number)
   [junit4]   2> at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:157)
   [junit4]   2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:227)
   [junit4]   2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:464)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>
   [junit4]   2> 1243173 INFO  (OverseerCollectionConfigSetProcessor-98485227527667724-127.0.0.1:61362_solr-n_0000000000) [n:127.0.0.1:61362_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000018 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1243175 INFO  (qtp27335993-10972) [n:127.0.0.1:61347_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&name=oversharded&nrtReplicas=2&action=CREATE&numShards=3&wt=javabin&version=2} status=400 QTime=4
   [junit4]   2> 1243175 INFO  (TEST-CollectionsAPIDistributedZkTest.testMaxNodesPerShard-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testMaxNodesPerShard
   [junit4]   2> 1243202 INFO  (TEST-CollectionsAPIDistributedZkTest.testMissingRequiredParameters-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testMissingRequiredParameters
   [junit4]   2> 1243203 INFO  (TEST-CollectionsAPIDistributedZkTest.testMissingRequiredParameters-seed#[60CD8E0C8B9EC242]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1243205 INFO  (qtp27335993-10931) [n:127.0.0.1:61347_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params action=CREATE&numShards=2&wt=javabin&version=2 and sendToO

[...truncated too long message...]

vletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
   [junit4]   2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>
   [junit4]   2> 1349825 INFO  (qtp16524918-11228) [n:127.0.0.1:61362_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:61347_solr&onlyIfLeaderActive=true&core=addReplicaColl_shard1_replica_n2&coreNodeName=core_node13&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=400 QTime=2007
   [junit4]   2> 1350520 WARN  (zkCallback-1719-thread-12-processing-n:127.0.0.1:61357_solr) [n:127.0.0.1:61357_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1350520 INFO  (jetty-closer-1699-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@17cf528{/solr,null,UNAVAILABLE}
   [junit4]   2> 1350540 WARN  (zkCallback-1716-thread-8-processing-n:127.0.0.1:61347_solr) [n:127.0.0.1:61347_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1350540 INFO  (jetty-closer-1699-thread-3) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@377b23{/solr,null,UNAVAILABLE}
   [junit4]   2> 1350542 ERROR (SUITE-CollectionsAPIDistributedZkTest-seed#[60CD8E0C8B9EC242]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1350543 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[60CD8E0C8B9EC242]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:61343 61343
   [junit4]   2> 1350544 INFO  (Thread-2327) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:61343 61343
   [junit4]   2> 1350545 WARN  (Thread-2327) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 15 /solr/configs/conf
   [junit4]   2> 7 /solr/aliases.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 61 /solr/collections/addReplicaColl/state.json
   [junit4]   2> 24 /solr/collections/nodes_used_collection/state.json
   [junit4]   2> 15 /solr/collections/acollectionafterbaddelete/state.json
   [junit4]   2> 13 /solr/collections/halfdeletedcollection/state.json
   [junit4]   2> 7 /solr/clusterprops.json
   [junit4]   2> 7 /solr/clusterstate.json
   [junit4]   2> 5 /solr/collections/halfcollectionblocker2/state.json
   [junit4]   2> 4 /solr/collections/halfcollectionblocker/state.json
   [junit4]   2> 4 /solr/collections/awhollynewcollection_0/state.json
   [junit4]   2> 4 /solr/collections/halfcollection/state.json
   [junit4]   2> 2 /solr/collections/addReplicaColl/leader_elect/shard1/election/98485227527667729-core_node5-n_0000000000
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 285 /solr/collections
   [junit4]   2> 7 /solr/live_nodes
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CollectionsAPIDistributedZkTest_60CD8E0C8B9EC242-001
   [junit4]   2> Aug 15, 2017 2:50:31 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {id=FST50}, docValues:{}, maxPointsInLeafNode=1098, maxMBSortInHeap=7.126697798037536, sim=RandomSimilarity(queryNorm=true): {}, locale=es-US, timezone=SST
   [junit4]   2> NOTE: Windows 10 10.0 x86/Oracle Corporation 1.8.0_144 (32-bit)/cpus=3,threads=1,free=101440928,total=457179136
   [junit4]   2> NOTE: All tests run in this JVM: [CacheHeaderTest, TestWordDelimiterFilterFactory, DistribCursorPagingTest, CloudMLTQParserTest, DocValuesMultiTest, TestIndexSearcher, SolrCoreMetricManagerTest, BasicAuthIntegrationTest, TestScoreJoinQPScore, TestBulkSchemaAPI, RankQueryTest, StatsReloadRaceTest, TestNamedUpdateProcessors, SearchHandlerTest, AddBlockUpdateTest, HdfsUnloadDistributedZkTest, TestSimpleTrackingShardHandler, ForceLeaderTest, DistributedMLTComponentTest, TestFastLRUCache, TestExclusionRuleCollectionAccess, ClassificationUpdateProcessorFactoryTest, TestNumericRangeQuery32, RulesTest, TestJettySolrRunner, SolrGangliaReporterTest, TestNumericTokenStream, TestXmlQParserPlugin, UninvertDocValuesMergePolicyTest, TestPointFields, TestRecovery, TestPKIAuthenticationPlugin, DistribJoinFromCollectionTest, JvmMetricsTest, TestPivotHelperCode, CdcrBootstrapTest, TestMissingGroups, TestDocSet, TestHalfAndHalfDocValues, GraphQueryTest, TestSolrXml, SolrPluginUtilsTest, QueryEqualityTest, FieldMutatingUpdateProcessorTest, TestAnalyzedSuggestions, TestNonDefinedSimilarityFactory, TestSolrCoreSnapshots, TestCloudRecovery, DocumentBuilderTest, TestManagedResourceStorage, OpenExchangeRatesOrgProviderTest, TestMacros, CoreMergeIndexesAdminHandlerTest, TestFastOutputStream, TestManagedSynonymGraphFilterFactory, TestZkChroot, DebugComponentTest, FacetPivotSmallTest, TestRestoreCore, TestClassicSimilarityFactory, AssignBackwardCompatibilityTest, HdfsRecoverLeaseTest, NodeMutatorTest, CopyFieldTest, FileBasedSpellCheckerTest, TestPullReplica, DocExpirationUpdateProcessorFactoryTest, TestReplicationHandlerBackup, TestSolrDeletionPolicy1, PKIAuthenticationIntegrationTest, TestStreamBody, ZkStateReaderTest, TestSolrCloudWithSecureImpersonation, TestLockTree, UpdateRequestProcessorFactoryTest, SparseHLLTest, LeaderFailoverAfterPartitionTest, HdfsLockFactoryTest, TestSerializedLuceneMatchVersion, ParsingFieldUpdateProcessorsTest, HdfsChaosMonkeyNothingIsSafeTest, ChaosMonkeySafeLeaderTest, SolrIndexConfigTest, TestLegacyFieldReuse, SolrSlf4jReporterTest, SolrCoreCheckLockOnStartupTest, SmileWriterTest, CollectionsAPISolrJTest, HttpPartitionTest, TestCustomSort, TestLuceneMatchVersion, ExternalFileFieldSortTest, LukeRequestHandlerTest, CollectionReloadTest, DeleteShardTest, SampleTest, StatelessScriptUpdateProcessorFactoryTest, OpenCloseCoreStressTest, TestHashQParserPlugin, TestConfigSetsAPIExclusivity, BJQParserTest, LargeFieldTest, TestLRUCache, HdfsThreadLeakTest, ReplicaListTransformerTest, TestSolr4Spatial, TestGeoJSONResponseWriter, CollectionTooManyReplicasTest, TestLegacyFieldCache, SignatureUpdateProcessorFactoryTest, TestDistributedMissingSort, TestDistribDocBasedVersion, TestElisionMultitermQuery, TestCoreDiscovery, TestCoreAdminApis, PreAnalyzedFieldTest, TestCrossCoreJoin, TestClusterProperties, TriLevelCompositeIdRoutingTest, SaslZkACLProviderTest, TestReloadAndDeleteDocs, TestLeaderInitiatedRecoveryThread, DistributedExpandComponentTest, BooleanFieldTest, TestRealTimeGet, TestSimpleTextCodec, MoveReplicaTest, OverseerStatusTest, SuggesterFSTTest, TestSolrCloudWithDelegationTokens, LoggingHandlerTest, PluginInfoTest, TestRawResponseWriter, AutoCommitTest, TestSolrConfigHandlerConcurrent, TestBM25SimilarityFactory, TestFieldCache, TestRTimerTree, TestCustomStream, SolrJmxReporterCloudTest, SuggesterTSTTest, TestExactStatsCache, TestBackupRepositoryFactory, TestManagedSchemaAPI, AnalyticsQueryTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [333/730 (1!)] on J1 in 119.47s, 19 tests, 1 failure <<< FAILURES!

[...truncated 40717 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS-EA] Lucene-Solr-master-Windows (64bit/jdk-9-ea+181) - Build # 6825 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6825/
Java: 64bit/jdk-9-ea+181 -XX:-UseCompressedOops -XX:+UseParallelGC --illegal-access=deny

4 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:51696/collection1

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:51696/collection1
        at __randomizedtesting.SeedInfo.seed([F0939BF1EDEE7763:78C7A42B43121A9B]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:638)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1121)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
        at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:484)
        at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:463)
        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.commit(AbstractFullDistribZkTestBase.java:1583)
        at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:213)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.net.SocketTimeoutException: Read timed out
        at java.base/java.net.SocketInputStream.socketRead0(Native Method)
        at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.base/java.net.SocketInputStream.read(SocketInputStream.java:171)
        at java.base/java.net.SocketInputStream.read(SocketInputStream.java:141)
        at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139)
        at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155)
        at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
        at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
        at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165)
        at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:167)
        at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
        at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:526)
        ... 52 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest

Error Message:
9 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest:     1) Thread[id=16136, name=zkCallback-3346-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)         at java.base@9/java.lang.Thread.run(Thread.java:844)    2) Thread[id=16288, name=zkCallback-3346-thread-6, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)         at java.base@9/java.lang.Thread.run(Thread.java:844)    3) Thread[id=16286, name=zkCallback-3346-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)         at java.base@9/java.lang.Thread.run(Thread.java:844)    4) Thread[id=16135, name=TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[F0939BF1EDEE7763]-EventThread, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@9/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)         at java.base@9/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)         at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)    5) Thread[id=16285, name=zkCallback-3346-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)         at java.base@9/java.lang.Thread.run(Thread.java:844)    6) Thread[id=16133, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.base@9/java.lang.Thread.sleep(Native Method)         at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.base@9/java.lang.Thread.run(Thread.java:844)    7) Thread[id=16284, name=zkCallback-3346-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)         at java.base@9/java.lang.Thread.run(Thread.java:844)    8) Thread[id=16287, name=zkCallback-3346-thread-5, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)         at java.base@9/java.lang.Thread.run(Thread.java:844)    9) Thread[id=16134, name=TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[F0939BF1EDEE7763]-SendThread(127.0.0.1:51653), state=RUNNABLE, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.base@9/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)         at java.base@9/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)         at java.base@9/sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)         at java.base@9/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)         at java.base@9/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at java.base@9/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)         at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 9 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest:
   1) Thread[id=16136, name=zkCallback-3346-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.base@9/java.lang.Thread.run(Thread.java:844)
   2) Thread[id=16288, name=zkCallback-3346-thread-6, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.base@9/java.lang.Thread.run(Thread.java:844)
   3) Thread[id=16286, name=zkCallback-3346-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.base@9/java.lang.Thread.run(Thread.java:844)
   4) Thread[id=16135, name=TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[F0939BF1EDEE7763]-EventThread, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@9/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@9/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2062)
        at java.base@9/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:435)
        at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
   5) Thread[id=16285, name=zkCallback-3346-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.base@9/java.lang.Thread.run(Thread.java:844)
   6) Thread[id=16133, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.base@9/java.lang.Thread.sleep(Native Method)
        at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.base@9/java.lang.Thread.run(Thread.java:844)
   7) Thread[id=16284, name=zkCallback-3346-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.base@9/java.lang.Thread.run(Thread.java:844)
   8) Thread[id=16287, name=zkCallback-3346-thread-5, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.base@9/java.lang.Thread.run(Thread.java:844)
   9) Thread[id=16134, name=TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[F0939BF1EDEE7763]-SendThread(127.0.0.1:51653), state=RUNNABLE, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.base@9/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at java.base@9/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at java.base@9/sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at java.base@9/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at java.base@9/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at java.base@9/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
        at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
        at __randomizedtesting.SeedInfo.seed([F0939BF1EDEE7763]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=16136, name=zkCallback-3346-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)         at java.base@9/java.lang.Thread.run(Thread.java:844)    2) Thread[id=16288, name=zkCallback-3346-thread-6, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)         at java.base@9/java.lang.Thread.run(Thread.java:844)    3) Thread[id=16286, name=zkCallback-3346-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)         at java.base@9/java.lang.Thread.run(Thread.java:844)    4) Thread[id=16285, name=zkCallback-3346-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)         at java.base@9/java.lang.Thread.run(Thread.java:844)    5) Thread[id=16284, name=zkCallback-3346-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)         at java.base@9/java.lang.Thread.run(Thread.java:844)    6) Thread[id=16287, name=zkCallback-3346-thread-5, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)         at java.base@9/java.lang.Thread.run(Thread.java:844)    7) Thread[id=16134, name=TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[F0939BF1EDEE7763]-SendThread(127.0.0.1:51653), state=RUNNABLE, group=TGRP-ChaosMonkeyNothingIsSafeTest]         at java.base@9/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)         at java.base@9/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)         at java.base@9/sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)         at java.base@9/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)         at java.base@9/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at java.base@9/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)         at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=16136, name=zkCallback-3346-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.base@9/java.lang.Thread.run(Thread.java:844)
   2) Thread[id=16288, name=zkCallback-3346-thread-6, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.base@9/java.lang.Thread.run(Thread.java:844)
   3) Thread[id=16286, name=zkCallback-3346-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.base@9/java.lang.Thread.run(Thread.java:844)
   4) Thread[id=16285, name=zkCallback-3346-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.base@9/java.lang.Thread.run(Thread.java:844)
   5) Thread[id=16284, name=zkCallback-3346-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.base@9/java.lang.Thread.run(Thread.java:844)
   6) Thread[id=16287, name=zkCallback-3346-thread-5, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
        at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
        at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.base@9/java.lang.Thread.run(Thread.java:844)
   7) Thread[id=16134, name=TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[F0939BF1EDEE7763]-SendThread(127.0.0.1:51653), state=RUNNABLE, group=TGRP-ChaosMonkeyNothingIsSafeTest]
        at java.base@9/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
        at java.base@9/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
        at java.base@9/sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
        at java.base@9/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
        at java.base@9/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at java.base@9/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
        at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
        at __randomizedtesting.SeedInfo.seed([F0939BF1EDEE7763]:0)


FAILED:  org.apache.solr.cloud.HttpPartitionTest.test

Error Message:
Doc with id=1 not found in http://127.0.0.1:64495/_pna/collMinRf_1x3 due to: Path not found: /id; rsp={doc=null}

Stack Trace:
java.lang.AssertionError: Doc with id=1 not found in http://127.0.0.1:64495/_pna/collMinRf_1x3 due to: Path not found: /id; rsp={doc=null}
        at __randomizedtesting.SeedInfo.seed([F0939BF1EDEE7763:78C7A42B43121A9B]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:603)
        at org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:558)
        at org.apache.solr.cloud.HttpPartitionTest.testMinRf(HttpPartitionTest.java:249)
        at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:127)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 11562 lines...]
   [junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
   [junit4]   2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\init-core-data-001
   [junit4]   2> 1089930 WARN  (SUITE-HttpPartitionTest-seed#[F0939BF1EDEE7763]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
   [junit4]   2> 1089930 INFO  (SUITE-HttpPartitionTest-seed#[F0939BF1EDEE7763]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1089932 INFO  (SUITE-HttpPartitionTest-seed#[F0939BF1EDEE7763]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl="https://issues.apache.org/jira/browse/SOLR-5776")
   [junit4]   2> 1089932 INFO  (SUITE-HttpPartitionTest-seed#[F0939BF1EDEE7763]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1089932 INFO  (SUITE-HttpPartitionTest-seed#[F0939BF1EDEE7763]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_pna/
   [junit4]   2> 1089934 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1089935 INFO  (Thread-3229) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1089935 INFO  (Thread-3229) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1089939 ERROR (Thread-3229) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1090034 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.ZkTestServer start zk server on port:64429
   [junit4]   2> 1090049 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1090051 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1090053 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1090055 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1090056 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1090058 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1090060 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1090062 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1090064 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1090066 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1090068 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.AbstractZkTestCase put C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\core\src\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1090070 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
   [junit4]   2> 1090392 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1090392 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3ca86461{/_pna,null,AVAILABLE}
   [junit4]   2> 1090393 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@78764f0e{HTTP/1.1,[http/1.1]}{127.0.0.1:64437}
   [junit4]   2> 1090394 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.e.j.s.Server Started @1093217ms
   [junit4]   2> 1090394 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/_pna, solr.data.dir=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\tempDir-001/control/data, hostPort=64436, coreRootDirectory=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\control-001\cores}
   [junit4]   2> 1090394 ERROR (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1090394 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1090394 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1090394 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1090394 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-15T10:45:39.980997500Z
   [junit4]   2> 1090398 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1090399 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\control-001\solr.xml
   [junit4]   2> 1090402 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@4235a755, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1090405 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:64429/solr
   [junit4]   2> 1090517 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1090519 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:64436__pna
   [junit4]   2> 1090520 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.Overseer Overseer (id=98487103620710404-127.0.0.1:64436__pna-n_0000000000) starting
   [junit4]   2> 1090531 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64436__pna
   [junit4]   2> 1090533 INFO  (zkCallback-2002-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1090586 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4235a755
   [junit4]   2> 1090594 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4235a755
   [junit4]   2> 1090594 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4235a755
   [junit4]   2> 1090597 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\control-001\cores
   [junit4]   2> 1090616 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1090617 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:64429/solr ready
   [junit4]   2> 1090617 INFO  (SocketProxy-Acceptor-64436) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=64452,localport=64436], receiveBufferSize:65536
   [junit4]   2> 1090619 INFO  (SocketProxy-Acceptor-64436) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=64437,localport=64453], receiveBufferSize=65536
   [junit4]   2> 1090621 INFO  (qtp1077669954-12900) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:64436__pna&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1090623 INFO  (OverseerThreadFactory-5725-thread-1) [    ] o.a.s.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 1090732 INFO  (SocketProxy-Acceptor-64436) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=64454,localport=64436], receiveBufferSize:65536
   [junit4]   2> 1090732 INFO  (SocketProxy-Acceptor-64436) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=64437,localport=64455], receiveBufferSize=65536
   [junit4]   2> 1090732 INFO  (qtp1077669954-12902) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1090733 INFO  (qtp1077669954-12902) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1090841 INFO  (zkCallback-2002-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1091746 INFO  (qtp1077669954-12902) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1091756 INFO  (qtp1077669954-12902) [    ] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 1091848 INFO  (qtp1077669954-12902) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1091864 INFO  (qtp1077669954-12902) [    ] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 1091866 INFO  (qtp1077669954-12902) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4235a755
   [junit4]   2> 1091866 INFO  (qtp1077669954-12902) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1091866 INFO  (qtp1077669954-12902) [    ] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\control-001\cores\control_collection_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\control-001\cores\control_collection_shard1_replica_n1\data\]
   [junit4]   2> 1091868 INFO  (qtp1077669954-12902) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=25, maxMergedSegmentMB=18.58203125, floorSegmentMB=0.9287109375, forceMergeDeletesPctAllowed=26.30086539387429, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1091870 WARN  (qtp1077669954-12902) [    ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 1091938 INFO  (qtp1077669954-12902) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1091938 INFO  (qtp1077669954-12902) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1091942 INFO  (qtp1077669954-12902) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1091942 INFO  (qtp1077669954-12902) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1091943 INFO  (qtp1077669954-12902) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=36, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.11236373351931511]
   [junit4]   2> 1091943 INFO  (qtp1077669954-12902) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1b9347c[control_collection_shard1_replica_n1] main]
   [junit4]   2> 1091946 INFO  (qtp1077669954-12902) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1091946 INFO  (qtp1077669954-12902) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1091947 INFO  (qtp1077669954-12902) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1091949 INFO  (searcherExecutor-5728-thread-1) [    ] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@1b9347c[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1091949 INFO  (qtp1077669954-12902) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575793660039004160
   [junit4]   2> 1091956 INFO  (qtp1077669954-12902) [    ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1091956 INFO  (qtp1077669954-12902) [    ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1091957 INFO  (qtp1077669954-12902) [    ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:64436/_pna/control_collection_shard1_replica_n1/
   [junit4]   2> 1091957 INFO  (qtp1077669954-12902) [    ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1091957 INFO  (qtp1077669954-12902) [    ] o.a.s.c.SyncStrategy http://127.0.0.1:64436/_pna/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 1091957 INFO  (qtp1077669954-12902) [    ] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1091962 INFO  (qtp1077669954-12902) [    ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:64436/_pna/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 1092066 INFO  (zkCallback-2002-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1092115 INFO  (qtp1077669954-12902) [    ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1092117 INFO  (qtp1077669954-12902) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1384
   [junit4]   2> 1092124 INFO  (qtp1077669954-12900) [    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1092226 INFO  (zkCallback-2002-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1092630 INFO  (OverseerCollectionConfigSetProcessor-98487103620710404-127.0.0.1:64436__pna-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1093124 INFO  (qtp1077669954-12900) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:64436__pna&wt=javabin&version=2} status=0 QTime=2502
   [junit4]   2> 1093129 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1093130 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:64429/solr ready
   [junit4]   2> 1093130 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1093131 INFO  (SocketProxy-Acceptor-64436) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=64459,localport=64436], receiveBufferSize:65536
   [junit4]   2> 1093131 INFO  (SocketProxy-Acceptor-64436) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=64437,localport=64460], receiveBufferSize=65536
   [junit4]   2> 1093132 INFO  (qtp1077669954-12899) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1093134 INFO  (OverseerThreadFactory-5725-thread-2) [    ] o.a.s.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 1093135 WARN  (OverseerThreadFactory-5725-thread-2) [    ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 1093339 INFO  (qtp1077669954-12899) [    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1093339 INFO  (qtp1077669954-12899) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=207
   [junit4]   2> 1093711 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\shard-1-001 of type TLOG
   [junit4]   2> 1093711 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1093712 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1ea8254b{/_pna,null,AVAILABLE}
   [junit4]   2> 1093713 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@51edacdf{HTTP/1.1,[http/1.1]}{127.0.0.1:64462}
   [junit4]   2> 1093713 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.e.j.s.Server Started @1096536ms
   [junit4]   2> 1093713 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/_pna, solrconfig=solrconfig.xml, solr.data.dir=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\tempDir-001/jetty1, hostPort=64461, coreRootDirectory=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\shard-1-001\cores, replicaType=TLOG}
   [junit4]   2> 1093714 ERROR (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1093714 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1093714 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1093714 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1093714 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-15T10:45:43.300196600Z
   [junit4]   2> 1093718 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1093718 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\shard-1-001\solr.xml
   [junit4]   2> 1093722 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@4235a755, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1093726 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:64429/solr
   [junit4]   2> 1093729 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15de57fba450008, likely client has closed socket
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 1093740 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1093742 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1093745 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64461__pna
   [junit4]   2> 1093746 INFO  (zkCallback-2009-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1093747 INFO  (zkCallback-2002-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1093747 INFO  (zkCallback-2014-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1093789 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4235a755
   [junit4]   2> 1093796 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4235a755
   [junit4]   2> 1093797 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4235a755
   [junit4]   2> 1093798 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\shard-1-001\cores
   [junit4]   2> 1093817 INFO  (SocketProxy-Acceptor-64461) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=64474,localport=64461], receiveBufferSize:65536
   [junit4]   2> 1093818 INFO  (SocketProxy-Acceptor-64461) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=64462,localport=64475], receiveBufferSize=65536
   [junit4]   2> 1093819 INFO  (qtp325611886-12949) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:64461__pna&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1093821 INFO  (OverseerCollectionConfigSetProcessor-98487103620710404-127.0.0.1:64436__pna-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1093821 INFO  (OverseerThreadFactory-5725-thread-3) [    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:64461__pna for creating new replica
   [junit4]   2> 1093823 INFO  (SocketProxy-Acceptor-64461) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=64476,localport=64461], receiveBufferSize:65536
   [junit4]   2> 1093824 INFO  (SocketProxy-Acceptor-64461) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=64462,localport=64477], receiveBufferSize=65536
   [junit4]   2> 1093824 INFO  (qtp325611886-12951) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 1093825 INFO  (qtp325611886-12951) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1095231 INFO  (qtp325611886-12951) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1095243 INFO  (qtp325611886-12951) [    ] o.a.s.s.IndexSchema [collection1_shard2_replica_t41] Schema name=test
   [junit4]   2> 1095320 INFO  (qtp325611886-12951) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1095337 INFO  (qtp325611886-12951) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t41' using configuration from collection collection1, trusted=true
   [junit4]   2> 1095337 INFO  (qtp325611886-12951) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_t41' (registry 'solr.core.collection1.shard2.replica_t41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4235a755
   [junit4]   2> 1095338 INFO  (qtp325611886-12951) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1095338 INFO  (qtp325611886-12951) [    ] o.a.s.c.SolrCore [[collection1_shard2_replica_t41] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\shard-1-001\cores\collection1_shard2_replica_t41], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\shard-1-001\cores\collection1_shard2_replica_t41\data\]
   [junit4]   2> 1095340 INFO  (qtp325611886-12951) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=25, maxMergedSegmentMB=18.58203125, floorSegmentMB=0.9287109375, forceMergeDeletesPctAllowed=26.30086539387429, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1095342 WARN  (qtp325611886-12951) [    ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 1095390 INFO  (qtp325611886-12951) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1095391 INFO  (qtp325611886-12951) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1095392 INFO  (qtp325611886-12951) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1095392 INFO  (qtp325611886-12951) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1095393 INFO  (qtp325611886-12951) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=36, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.11236373351931511]
   [junit4]   2> 1095394 INFO  (qtp325611886-12951) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@4fb53bb7[collection1_shard2_replica_t41] main]
   [junit4]   2> 1095395 INFO  (qtp325611886-12951) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1095396 INFO  (qtp325611886-12951) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1095396 INFO  (qtp325611886-12951) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1095396 INFO  (qtp325611886-12951) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575793663653445632
   [junit4]   2> 1095396 INFO  (searcherExecutor-5739-thread-1) [    ] o.a.s.c.SolrCore [collection1_shard2_replica_t41] Registered new searcher Searcher@4fb53bb7[collection1_shard2_replica_t41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1095402 INFO  (qtp325611886-12951) [    ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1095402 INFO  (qtp325611886-12951) [    ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1095402 INFO  (qtp325611886-12951) [    ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:64461/_pna/collection1_shard2_replica_t41/
   [junit4]   2> 1095402 INFO  (qtp325611886-12951) [    ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1095402 INFO  (qtp325611886-12951) [    ] o.a.s.c.SyncStrategy http://127.0.0.1:64461/_pna/collection1_shard2_replica_t41/ has no replicas
   [junit4]   2> 1095402 INFO  (qtp325611886-12951) [    ] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1095403 INFO  (qtp325611886-12951) [    ] o.a.s.c.ZkController collection1_shard2_replica_t41 stopping background replication from leader
   [junit4]   2> 1095407 INFO  (qtp325611886-12951) [    ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:64461/_pna/collection1_shard2_replica_t41/ shard2
   [junit4]   2> 1095560 INFO  (qtp325611886-12951) [    ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1095561 INFO  (qtp325611886-12951) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1737
   [junit4]   2> 1095563 INFO  (qtp325611886-12949) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:64461__pna&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2} status=0 QTime=1745
   [junit4]   2> 1095823 INFO  (OverseerCollectionConfigSetProcessor-98487103620710404-127.0.0.1:64436__pna-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1095856 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\shard-2-001 of type TLOG
   [junit4]   2> 1095857 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 1095858 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@576c562c{/_pna,null,AVAILABLE}
   [junit4]   2> 1095858 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2f23949c{HTTP/1.1,[http/1.1]}{127.0.0.1:64480}
   [junit4]   2> 1095859 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.e.j.s.Server Started @1098681ms
   [junit4]   2> 1095859 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/_pna, solrconfig=solrconfig.xml, solr.data.dir=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\tempDir-001/jetty2, hostPort=64479, coreRootDirectory=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\shard-2-001\cores, replicaType=TLOG}
   [junit4]   2> 1095859 ERROR (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1095859 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1095859 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1095859 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null, Default config dir: null
   [junit4]   2> 1095859 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-08-15T10:45:45.445849800Z
   [junit4]   2> 1095863 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1095863 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\shard-2-001\solr.xml
   [junit4]   2> 1095866 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@4235a755, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1095868 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:64429/solr
   [junit4]   2> 1095879 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1095881 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1095883 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64479__pna
   [junit4]   2> 1095884 INFO  (zkCallback-2014-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1095885 INFO  (zkCallback-2002-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1095885 INFO  (zkCallback-2020-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1095885 INFO  (zkCallback-2009-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1095931 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4235a755
   [junit4]   2> 1095948 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4235a755
   [junit4]   2> 1095949 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4235a755
   [junit4]   2> 1095950 INFO  (TEST-HttpPartitionTest.test-seed#[F0939BF1EDEE7763]) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\shard-2-001\cores
   [junit4]   2> 1095975 INFO  (qtp325611886-12952) [    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:64479__pna&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1095977 INFO  (OverseerThreadFactory-5725-thread-4) [    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:64479__pna for creating new replica
   [junit4]   2> 1095979 INFO  (SocketProxy-Acceptor-64479) [    ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=64492,localport=64479], receiveBufferSize:65536
   [junit4]   2> 1095979 INFO  (SocketProxy-Acceptor-64479) [    ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=64480,localport=64493], receiveBufferSize=65536
   [junit4]   2> 1095981 INFO  (qtp600943090-12983) [    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 1095981 INFO  (qtp600943090-12983) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1096993 INFO  (qtp600943090-12983) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1097002 INFO  (qtp600943090-12983) [    ] o.a.s.s.IndexSchema [collection1_shard1_replica_t43] Schema name=test
   [junit4]   2> 1097077 INFO  (qtp600943090-12983) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1097096 INFO  (qtp600943090-12983) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_t43' using configuration from collection collection1, trusted=true
   [junit4]   2> 1097096 INFO  (qtp600943090-12983) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_t43' (registry 'solr.core.collection1.shard1.replica_t43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4235a755
   [junit4]   2> 1097097 INFO  (qtp600943090-12983) [    ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1097097 INFO  (qtp600943090-12983) [    ] o.a.s.c.SolrCore [[collection1_shard1_replica_t43] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\shard-2-001\cores\collection1_shard1_replica_t43], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_F0939BF1EDEE7763-001\shard-2-001\cores\collection1_shard1_replica_t43\data\]
   [junit4]   2> 1097099 INFO  (qtp600943090-12983) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=25, maxMergedSegmentMB=18.58203125, floorSegmentMB=0.9287109375, forceMergeDeletesPctAllowed=26.30086539387429, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1097101 WARN  (qtp600943090-12983) [    ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 1097161 INFO  (qtp600943090-12983) [    ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1097161 INFO  (qtp600943090-12983) [    ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1097163 INFO  (qtp600943090-12983) [    ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1097163 INFO  (qtp600943090-12983) [    ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1097164 INFO  (qtp600943090-12983) [    ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=36, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.11236373351931511]
   [junit4]   2> 1097165 INFO  (qtp600943090-12983) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@53e0a127[collection1_shard1_replica_t43] main]
   [junit4]   2> 1097166 INFO  (qtp600943090-12983) [    ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1097166 INFO  (qtp600943090-12983) [    ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1097167 INFO  (qtp600943090-12983) [    ] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1097168 INFO  (searcherExecutor-5750-thread-1) [    ] o.a.s.c.SolrCore [collection1_shard1_replica_t43] Registered new searcher Searcher@53e0a127[collection1_shard1_replica_t43] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1097168 INFO  (qtp600943090-12983) [    ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575793665511522304
   [junit4]   2> 1097174 INFO  (qtp600943090-12983) [    ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1097174 INFO  (qtp600943090-12983) [    ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1097174 INFO  (qtp600943090-12983) [    ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:64479/_pna/collection1_shard1_replica_t43/
   [junit4]   2> 1097174 INFO  (qtp600943090-12983) [    ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1097174 INFO  (qtp600943090-12983) [    ] o.a.s.c.SyncStrategy http://127.0.0.1:64479/_pna/collection1_shard1_replica_t43/ has no replicas
   [junit4]   2> 1097174 INFO  (qtp600943090-12983) [    ] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1097175 INFO  (qtp600943090-12983) [    ] o.a.s.c.ZkController collection1_shard1_replica_t43 stopping background replication from leader
   [junit4]   2> 1097178 INFO  (qtp600943090-12983) [    ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:64479/_pna/collection1_shard1_replica_t43/ shard1
   [junit4]   2> 1097330 INFO  (qtp600943090-12983) [    ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1097332 INFO  (qtp600943090-12983) [    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t4

[...truncated too long message...]

.transfer(SynchronousQueue.java:361)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
   [junit4]    >         at java.base@9/java.lang.Thread.run(Thread.java:844)
   [junit4]    >    9) Thread[id=16134, name=TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[F0939BF1EDEE7763]-SendThread(127.0.0.1:51653), state=RUNNABLE, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]    >         at java.base@9/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
   [junit4]    >         at java.base@9/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
   [junit4]    >         at java.base@9/sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
   [junit4]    >         at java.base@9/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
   [junit4]    >         at java.base@9/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at java.base@9/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
   [junit4]    >         at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
   [junit4]    > at __randomizedtesting.SeedInfo.seed([F0939BF1EDEE7763]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=16136, name=zkCallback-3346-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]    >         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
   [junit4]    >         at java.base@9/java.lang.Thread.run(Thread.java:844)
   [junit4]    >    2) Thread[id=16288, name=zkCallback-3346-thread-6, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]    >         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
   [junit4]    >         at java.base@9/java.lang.Thread.run(Thread.java:844)
   [junit4]    >    3) Thread[id=16286, name=zkCallback-3346-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]    >         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
   [junit4]    >         at java.base@9/java.lang.Thread.run(Thread.java:844)
   [junit4]    >    4) Thread[id=16285, name=zkCallback-3346-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]    >         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
   [junit4]    >         at java.base@9/java.lang.Thread.run(Thread.java:844)
   [junit4]    >    5) Thread[id=16284, name=zkCallback-3346-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]    >         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
   [junit4]    >         at java.base@9/java.lang.Thread.run(Thread.java:844)
   [junit4]    >    6) Thread[id=16287, name=zkCallback-3346-thread-5, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]    >         at java.base@9/jdk.internal.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.base@9/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:462)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)
   [junit4]    >         at java.base@9/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:937)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1091)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
   [junit4]    >         at java.base@9/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
   [junit4]    >         at java.base@9/java.lang.Thread.run(Thread.java:844)
   [junit4]    >    7) Thread[id=16134, name=TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[F0939BF1EDEE7763]-SendThread(127.0.0.1:51653), state=RUNNABLE, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]    >         at java.base@9/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll0(Native Method)
   [junit4]    >         at java.base@9/sun.nio.ch.WindowsSelectorImpl$SubSelector.poll(WindowsSelectorImpl.java:296)
   [junit4]    >         at java.base@9/sun.nio.ch.WindowsSelectorImpl$SubSelector.access$400(WindowsSelectorImpl.java:278)
   [junit4]    >         at java.base@9/sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:159)
   [junit4]    >         at java.base@9/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at java.base@9/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
   [junit4]    >         at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
   [junit4]    > at __randomizedtesting.SeedInfo.seed([F0939BF1EDEE7763]:0)
   [junit4] Completed [440/730 (2!)] on J0 in 73.16s, 1 test, 3 errors <<< FAILURES!

[...truncated 37375 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]