[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_172) - Build # 22850 - Unstable!

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_172) - Build # 22850 - Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/22850/
Java: 64bit/jdk1.8.0_172 -XX:-UseCompressedOops -XX:+UseG1GC

4 tests failed.
FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:32821/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:32821/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([D3FF2F40DCE2110D:5DF34F30A9FA9527]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:46103/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:46103/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([D3FF2F40DCE2110D:5DF34F30A9FA9527]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:38919/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:38919/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([D3FF2F40DCE2110D:5DF34F30A9FA9527]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:45105/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:45105/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([D3FF2F40DCE2110D:5DF34F30A9FA9527]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at sun.reflect.GeneratedMethodAccessor84.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 14286 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestPullReplicaErrorHandling
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_D3FF2F40DCE2110D-001/init-core-data-001
   [junit4]   2> 1953666 WARN  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=9 numCloses=9
   [junit4]   2> 1953666 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1953668 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 1953668 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1953668 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_D3FF2F40DCE2110D-001/tempDir-001
   [junit4]   2> 1953668 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1953668 INFO  (Thread-5087) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1953668 INFO  (Thread-5087) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1953671 ERROR (Thread-5087) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1953768 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:36551
   [junit4]   2> 1953770 INFO  (zkConnectionManagerCallback-6035-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1953773 INFO  (jetty-launcher-6032-thread-4) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 1953773 INFO  (jetty-launcher-6032-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 1953773 INFO  (jetty-launcher-6032-thread-3) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 1953773 INFO  (jetty-launcher-6032-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 1953773 INFO  (jetty-launcher-6032-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1953773 INFO  (jetty-launcher-6032-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-4) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@55902ae4{/solr,null,AVAILABLE}
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-3) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7006b952{/solr,null,AVAILABLE}
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@48b21580{/solr,null,AVAILABLE}
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@97cb26c{HTTP/1.1,[http/1.1]}{127.0.0.1:40645}
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@24f060bd{HTTP/1.1,[http/1.1]}{127.0.0.1:33021}
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-4) [    ] o.e.j.s.Server Started @1953834ms
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@25d71cb8{HTTP/1.1,[http/1.1]}{127.0.0.1:45295}
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-2) [    ] o.e.j.s.Server Started @1953834ms
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40645}
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-3) [    ] o.e.j.s.Server Started @1953834ms
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33021}
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45295}
   [junit4]   2> 1953775 ERROR (jetty-launcher-6032-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1953775 ERROR (jetty-launcher-6032-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1953775 INFO  (jetty-launcher-6032-thread-3) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1953775 INFO  (jetty-launcher-6032-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1953775 INFO  (jetty-launcher-6032-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1953775 INFO  (jetty-launcher-6032-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1953775 INFO  (jetty-launcher-6032-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T07:32:19.189Z
   [junit4]   2> 1953774 INFO  (jetty-launcher-6032-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@74064111{/solr,null,AVAILABLE}
   [junit4]   2> 1953775 INFO  (jetty-launcher-6032-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1953777 INFO  (jetty-launcher-6032-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1953777 INFO  (jetty-launcher-6032-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1953777 INFO  (jetty-launcher-6032-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1953777 INFO  (jetty-launcher-6032-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T07:32:19.191Z
   [junit4]   2> 1953777 ERROR (jetty-launcher-6032-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1953777 INFO  (jetty-launcher-6032-thread-4) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1953777 INFO  (jetty-launcher-6032-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1953777 INFO  (jetty-launcher-6032-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@21e100a3{HTTP/1.1,[http/1.1]}{127.0.0.1:37539}
   [junit4]   2> 1953777 INFO  (jetty-launcher-6032-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1953777 INFO  (jetty-launcher-6032-thread-1) [    ] o.e.j.s.Server Started @1953836ms
   [junit4]   2> 1953777 INFO  (jetty-launcher-6032-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1953777 INFO  (jetty-launcher-6032-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T07:32:19.191Z
   [junit4]   2> 1953777 INFO  (jetty-launcher-6032-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37539}
   [junit4]   2> 1953777 ERROR (jetty-launcher-6032-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1953777 INFO  (jetty-launcher-6032-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1953777 INFO  (jetty-launcher-6032-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1953777 INFO  (jetty-launcher-6032-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1953777 INFO  (jetty-launcher-6032-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1953777 INFO  (jetty-launcher-6032-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T07:32:19.191Z
   [junit4]   2> 1953778 INFO  (zkConnectionManagerCallback-6039-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1953778 INFO  (zkConnectionManagerCallback-6037-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1953778 INFO  (jetty-launcher-6032-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1953778 INFO  (zkConnectionManagerCallback-6041-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1953800 INFO  (zkConnectionManagerCallback-6043-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1953801 INFO  (jetty-launcher-6032-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1953801 INFO  (jetty-launcher-6032-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1953801 INFO  (jetty-launcher-6032-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1953853 INFO  (jetty-launcher-6032-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36551/solr
   [junit4]   2> 1953865 INFO  (zkConnectionManagerCallback-6047-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1953870 INFO  (jetty-launcher-6032-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36551/solr
   [junit4]   2> 1953881 INFO  (zkConnectionManagerCallback-6051-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1953886 INFO  (zkConnectionManagerCallback-6053-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1953890 INFO  (zkConnectionManagerCallback-6055-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1953899 INFO  (jetty-launcher-6032-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36551/solr
   [junit4]   2> 1953924 INFO  (jetty-launcher-6032-thread-2) [n:127.0.0.1:33021_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33021_solr
   [junit4]   2> 1953925 INFO  (jetty-launcher-6032-thread-3) [n:127.0.0.1:45295_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45295_solr
   [junit4]   2> 1953925 INFO  (zkCallback-6052-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1953925 INFO  (jetty-launcher-6032-thread-2) [n:127.0.0.1:33021_solr    ] o.a.s.c.Overseer Overseer (id=72216166038700039-127.0.0.1:33021_solr-n_0000000000) starting
   [junit4]   2> 1953925 INFO  (zkCallback-6054-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1953925 INFO  (zkConnectionManagerCallback-6063-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1953926 INFO  (zkConnectionManagerCallback-6065-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1953930 INFO  (jetty-launcher-6032-thread-1) [n:127.0.0.1:37539_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1953932 INFO  (jetty-launcher-6032-thread-1) [n:127.0.0.1:37539_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1953932 INFO  (jetty-launcher-6032-thread-1) [n:127.0.0.1:37539_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37539_solr
   [junit4]   2> 1953933 INFO  (zkConnectionManagerCallback-6070-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1953934 INFO  (zkCallback-6052-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1953934 INFO  (zkCallback-6054-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1953934 INFO  (jetty-launcher-6032-thread-2) [n:127.0.0.1:33021_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1953935 INFO  (jetty-launcher-6032-thread-2) [n:127.0.0.1:33021_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36551/solr ready
   [junit4]   2> 1953935 INFO  (OverseerStateUpdate-72216166038700039-127.0.0.1:33021_solr-n_0000000000) [n:127.0.0.1:33021_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:33021_solr
   [junit4]   2> 1953936 INFO  (jetty-launcher-6032-thread-2) [n:127.0.0.1:33021_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1953936 INFO  (jetty-launcher-6032-thread-2) [n:127.0.0.1:33021_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33021_solr
   [junit4]   2> 1953936 DEBUG (OverseerStateUpdate-72216166038700039-127.0.0.1:33021_solr-n_0000000000) [n:127.0.0.1:33021_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:37539_solr"} current state version: 0
   [junit4]   2> 1953936 DEBUG (OverseerStateUpdate-72216166038700039-127.0.0.1:33021_solr-n_0000000000) [n:127.0.0.1:33021_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:33021_solr"} current state version: 0
   [junit4]   2> 1953936 INFO  (zkCallback-6054-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1953936 INFO  (zkCallback-6052-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1953937 INFO  (zkCallback-6069-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1953937 INFO  (zkCallback-6064-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
   [junit4]   2> 1953949 INFO  (zkConnectionManagerCallback-6081-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1953950 INFO  (jetty-launcher-6032-thread-1) [n:127.0.0.1:37539_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1953950 INFO  (jetty-launcher-6032-thread-1) [n:127.0.0.1:37539_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36551/solr ready
   [junit4]   2> 1953953 INFO  (zkConnectionManagerCallback-6082-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1953954 INFO  (jetty-launcher-6032-thread-3) [n:127.0.0.1:45295_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1953954 INFO  (jetty-launcher-6032-thread-3) [n:127.0.0.1:45295_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36551/solr ready
   [junit4]   2> 1953957 INFO  (jetty-launcher-6032-thread-2) [n:127.0.0.1:33021_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1953961 INFO  (jetty-launcher-6032-thread-3) [n:127.0.0.1:45295_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1953961 INFO  (jetty-launcher-6032-thread-1) [n:127.0.0.1:37539_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1953984 INFO  (jetty-launcher-6032-thread-3) [n:127.0.0.1:45295_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45295.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1953987 INFO  (jetty-launcher-6032-thread-1) [n:127.0.0.1:37539_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37539.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1953987 INFO  (jetty-launcher-6032-thread-2) [n:127.0.0.1:33021_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33021.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1953991 INFO  (jetty-launcher-6032-thread-3) [n:127.0.0.1:45295_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45295.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1953992 INFO  (jetty-launcher-6032-thread-3) [n:127.0.0.1:45295_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45295.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1953992 INFO  (jetty-launcher-6032-thread-1) [n:127.0.0.1:37539_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37539.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1953992 INFO  (jetty-launcher-6032-thread-1) [n:127.0.0.1:37539_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37539.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1953992 INFO  (jetty-launcher-6032-thread-3) [n:127.0.0.1:45295_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_D3FF2F40DCE2110D-001/tempDir-001/node3/.
   [junit4]   2> 1953993 INFO  (jetty-launcher-6032-thread-1) [n:127.0.0.1:37539_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_D3FF2F40DCE2110D-001/tempDir-001/node1/.
   [junit4]   2> 1953994 INFO  (jetty-launcher-6032-thread-2) [n:127.0.0.1:33021_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33021.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1953994 INFO  (jetty-launcher-6032-thread-2) [n:127.0.0.1:33021_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33021.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1953995 INFO  (jetty-launcher-6032-thread-2) [n:127.0.0.1:33021_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_D3FF2F40DCE2110D-001/tempDir-001/node2/.
   [junit4]   2> 1954048 INFO  (jetty-launcher-6032-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36551/solr
   [junit4]   2> 1954049 INFO  (zkConnectionManagerCallback-6089-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1954051 INFO  (zkConnectionManagerCallback-6091-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1954054 INFO  (jetty-launcher-6032-thread-4) [n:127.0.0.1:40645_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1954056 INFO  (jetty-launcher-6032-thread-4) [n:127.0.0.1:40645_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1954056 INFO  (jetty-launcher-6032-thread-4) [n:127.0.0.1:40645_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40645_solr
   [junit4]   2> 1954056 DEBUG (OverseerStateUpdate-72216166038700039-127.0.0.1:33021_solr-n_0000000000) [n:127.0.0.1:33021_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:40645_solr"} current state version: 0
   [junit4]   2> 1954056 INFO  (zkCallback-6064-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954056 INFO  (zkCallback-6052-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954056 INFO  (zkCallback-6054-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954056 INFO  (zkCallback-6080-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954056 INFO  (zkCallback-6069-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954056 INFO  (zkCallback-6090-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954056 INFO  (zkCallback-6079-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954068 INFO  (zkConnectionManagerCallback-6098-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1954069 INFO  (jetty-launcher-6032-thread-4) [n:127.0.0.1:40645_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1954069 INFO  (jetty-launcher-6032-thread-4) [n:127.0.0.1:40645_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36551/solr ready
   [junit4]   2> 1954069 INFO  (jetty-launcher-6032-thread-4) [n:127.0.0.1:40645_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1954093 INFO  (jetty-launcher-6032-thread-4) [n:127.0.0.1:40645_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40645.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1954098 INFO  (jetty-launcher-6032-thread-4) [n:127.0.0.1:40645_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40645.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1954098 INFO  (jetty-launcher-6032-thread-4) [n:127.0.0.1:40645_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40645.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1954099 INFO  (jetty-launcher-6032-thread-4) [n:127.0.0.1:40645_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_D3FF2F40DCE2110D-001/tempDir-001/node4/.
   [junit4]   2> 1954122 INFO  (zkConnectionManagerCallback-6101-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1954125 INFO  (zkConnectionManagerCallback-6106-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1954126 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1954126 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36551/solr ready
   [junit4]   2> 1954128 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@25d71cb8{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1954128 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1631184602
   [junit4]   2> 1954129 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1954129 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@40b21c46: rootName = solr_45295, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@5ba4329c
   [junit4]   2> 1954134 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1954134 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7e5dccce: rootName = solr_45295, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@48bac951
   [junit4]   2> 1954136 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1954136 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4c65225d: rootName = solr_45295, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1c3cc9d
   [junit4]   2> 1954137 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:45295_solr
   [junit4]   2> 1954137 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1954137 DEBUG (OverseerStateUpdate-72216166038700039-127.0.0.1:33021_solr-n_0000000000) [n:127.0.0.1:33021_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:45295_solr"} current state version: 0
   [junit4]   2> 1954138 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1954138 INFO  (zkCallback-6064-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954138 INFO  (zkCallback-6054-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954139 INFO  (zkCallback-6069-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954139 INFO  (zkCallback-6052-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954139 INFO  (zkCallback-6079-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954139 INFO  (zkCallback-6080-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954139 INFO  (zkCallback-6097-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954139 INFO  (zkCallback-6090-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954139 INFO  (zkCallback-6105-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954142 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7006b952{/solr,null,UNAVAILABLE}
   [junit4]   2> 1954142 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1954143 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 1954143 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1954143 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1954143 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1954144 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@91487af{/solr,null,AVAILABLE}
   [junit4]   2> 1954144 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@33874f65{HTTP/1.1,[http/1.1]}{127.0.0.1:46829}
   [junit4]   2> 1954144 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.Server Started @1954203ms
   [junit4]   2> 1954144 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44301}
   [junit4]   2> 1954144 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1954144 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1954144 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1954144 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1954144 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1954144 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T07:32:19.558Z
   [junit4]   2> 1954145 INFO  (zkConnectionManagerCallback-6108-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1954145 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1954198 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36551/solr
   [junit4]   2> 1954199 INFO  (zkConnectionManagerCallback-6112-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1954201 INFO  (zkConnectionManagerCallback-6114-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1954204 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [n:127.0.0.1:44301_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1954206 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [n:127.0.0.1:44301_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1954206 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [n:127.0.0.1:44301_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44301_solr
   [junit4]   2> 1954206 DEBUG (OverseerStateUpdate-72216166038700039-127.0.0.1:33021_solr-n_0000000000) [n:127.0.0.1:33021_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:44301_solr"} current state version: 0
   [junit4]   2> 1954206 INFO  (zkCallback-6079-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954206 INFO  (zkCallback-6069-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954206 INFO  (zkCallback-6090-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954206 INFO  (zkCallback-6064-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954206 INFO  (zkCallback-6097-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954206 INFO  (zkCallback-6105-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954206 INFO  (zkCallback-6113-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954206 INFO  (zkCallback-6052-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954222 INFO  (zkConnectionManagerCallback-6121-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1954223 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [n:127.0.0.1:44301_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1954224 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [n:127.0.0.1:44301_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36551/solr ready
   [junit4]   2> 1954224 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [n:127.0.0.1:44301_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1954235 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [n:127.0.0.1:44301_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44301.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1954240 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [n:127.0.0.1:44301_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44301.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1954240 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [n:127.0.0.1:44301_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44301.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1954241 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [n:127.0.0.1:44301_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_D3FF2F40DCE2110D-001/tempDir-001/node3/.
   [junit4]   2> 1954265 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:46829/solr. Proxy: http://127.0.0.1:44301/solr
   [junit4]   2> 1954265 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@24f060bd{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1954266 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2088179227
   [junit4]   2> 1954266 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1954266 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7bcd74fa: rootName = solr_33021, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@511d1eb
   [junit4]   2> 1954271 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1954271 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2a4ba2f1: rootName = solr_33021, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@48bac951
   [junit4]   2> 1954273 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1954273 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2ec5614c: rootName = solr_33021, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1c3cc9d
   [junit4]   2> 1954274 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:33021_solr
   [junit4]   2> 1954275 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1954275 DEBUG (OverseerStateUpdate-72216166038700039-127.0.0.1:33021_solr-n_0000000000) [n:127.0.0.1:33021_solr    ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:33021_solr"} current state version: 0
   [junit4]   2> 1954275 INFO  (zkCallback-6064-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954275 INFO  (zkCallback-6069-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954275 INFO  (zkCallback-6097-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954275 INFO  (zkCallback-6113-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954275 INFO  (zkCallback-6090-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954275 INFO  (zkCallback-6105-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954275 INFO  (zkCallback-6052-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954275 INFO  (zkCallback-6079-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954276 INFO  (zkCallback-6120-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954276 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.Overseer Overseer (id=72216166038700039-127.0.0.1:33021_solr-n_0000000000) closing
   [junit4]   2> 1954276 INFO  (OverseerStateUpdate-72216166038700039-127.0.0.1:33021_solr-n_0000000000) [n:127.0.0.1:33021_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:33021_solr
   [junit4]   2> 1954276 WARN  (OverseerAutoScalingTriggerThread-72216166038700039-127.0.0.1:33021_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1954278 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@48b21580{/solr,null,UNAVAILABLE}
   [junit4]   2> 1954278 INFO  (zkCallback-6064-thread-1) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37539_solr
   [junit4]   2> 1954279 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1954279 INFO  (zkCallback-6064-thread-1) [n:127.0.0.1:37539_solr    ] o.a.s.c.Overseer Overseer (id=72216166038700042-127.0.0.1:37539_solr-n_0000000002) starting
   [junit4]   2> 1954279 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 1954279 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1954280 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1954280 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1954280 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@12357ba7{/solr,null,AVAILABLE}
   [junit4]   2> 1954280 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3ac49642{HTTP/1.1,[http/1.1]}{127.0.0.1:38061}
   [junit4]   2> 1954280 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.Server Started @1954339ms
   [junit4]   2> 1954280 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45105}
   [junit4]   2> 1954280 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1954280 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1954281 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1954281 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1954281 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1954281 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T07:32:19.695Z
   [junit4]   2> 1954281 INFO  (OverseerStateUpdate-72216166038700042-127.0.0.1:37539_solr-n_0000000002) [n:127.0.0.1:37539_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:37539_solr
   [junit4]   2> 1954281 INFO  (zkConnectionManagerCallback-6124-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1954282 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1954282 DEBUG (OverseerStateUpdate-72216166038700042-127.0.0.1:37539_solr-n_0000000002) [n:127.0.0.1:37539_solr    ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:40645_solr"} current state version: 0
   [junit4]   2> 1954282 DEBUG (OverseerStateUpdate-72216166038700042-127.0.0.1:37539_solr-n_0000000002) [n:127.0.0.1:37539_solr    ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:45295_solr"} current state version: 0
   [junit4]   2> 1954282 DEBUG (OverseerStateUpdate-72216166038700042-127.0.0.1:37539_solr-n_0000000002) [n:127.0.0.1:37539_solr    ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:44301_solr"} current state version: 0
   [junit4]   2> 1954282 DEBUG (OverseerStateUpdate-72216166038700042-127.0.0.1:37539_solr-n_0000000002) [n:127.0.0.1:37539_solr    ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:33021_solr"} current state version: 0
   [junit4]   2> 1954341 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36551/solr
   [junit4]   2> 1954341 INFO  (zkConnectionManagerCallback-6128-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1954343 INFO  (zkConnectionManagerCallback-6130-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1954345 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1954347 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1954347 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45105_solr
   [junit4]   2> 1954347 DEBUG (OverseerStateUpdate-72216166038700042-127.0.0.1:37539_solr-n_0000000002) [n:127.0.0.1:37539_solr    ] o.a.s.c.Overseer processMessage: queueSize: 5, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:45105_solr"} current state version: 0
   [junit4]   2> 1954347 INFO  (zkCallback-6064-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954347 INFO  (zkCallback-6113-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954347 INFO  (zkCallback-6105-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954347 INFO  (zkCallback-6079-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954347 INFO  (zkCallback-6090-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954347 INFO  (zkCallback-6097-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954347 INFO  (zkCallback-6120-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954347 INFO  (zkCallback-6129-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954359 INFO  (zkConnectionManagerCallback-6137-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1954360 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1954360 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36551/solr ready
   [junit4]   2> 1954361 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1954371 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45105.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1954376 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45105.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1954376 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45105.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1954377 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_D3FF2F40DCE2110D-001/tempDir-001/node2/.
   [junit4]   2> 1954401 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:38061/solr. Proxy: http://127.0.0.1:45105/solr
   [junit4]   2> 1954402 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@21e100a3{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1954403 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=280035260
   [junit4]   2> 1954403 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1954403 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6fd64e9a: rootName = solr_37539, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@6b561394
   [junit4]   2> 1954408 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1954408 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@15b5e29f: rootName = solr_37539, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@48bac951
   [junit4]   2> 1954410 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1954410 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7fc5108: rootName = solr_37539, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1c3cc9d
   [junit4]   2> 1954411 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:37539_solr
   [junit4]   2> 1954411 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1954412 DEBUG (OverseerStateUpdate-72216166038700042-127.0.0.1:37539_solr-n_0000000002) [n:127.0.0.1:37539_solr    ] o.a.s.c.Overseer processMessage: queueSize: 6, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:37539_solr"} current state version: 0
   [junit4]   2> 1954412 INFO  (zkCallback-6113-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954412 INFO  (zkCallback-6064-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954412 INFO  (zkCallback-6079-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954413 INFO  (zkCallback-6097-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954413 INFO  (zkCallback-6129-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954413 INFO  (zkCallback-6105-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954413 INFO  (zkCallback-6090-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954413 INFO  (zkCallback-6120-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954413 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.Overseer Overseer (id=72216166038700042-127.0.0.1:37539_solr-n_0000000002) closing
   [junit4]   2> 1954413 INFO  (zkCallback-6136-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 1954413 WARN  (OverseerAutoScalingTriggerThread-72216166038700042-127.0.0.1:37539_solr-n_0000000002) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1954413 INFO  (OverseerStateUpdate-72216166038700042-127.0.0.1:37539_solr-n_0000000002) [n:127.0.0.1:37539_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37539_solr
   [junit4]   2> 1954415 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@74064111{/solr,null,UNAVAILABLE}
   [junit4]   2> 1954415 INFO  (zkCallback-6090-thread-1) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40645_solr
   [junit4]   2> 1954416 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1954416 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 1954416 INFO  (zkCallback-6090-thread-1) [n:127.0.0.1:40645_solr    ] o.a.s.c.Overseer Overseer (id=72216166038700047-127.0.0.1:40645_solr-n_0000000003) starting
   [junit4]   2> 1954416 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1954416 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1954417 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1954417 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6b3556dd{/solr,null,AVAILABLE}
   [junit4]   2> 1954417 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5fd0615c{HTTP/1.1,[http/1.1]}{127.0.0.1:39641}
   [junit4]   2> 1954417 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.Server Started @1954476ms
   [junit4]   2> 1954417 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34385}
   [junit4]   2> 1954417 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1954418 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1954418 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1954418 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1954418 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1954418 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T07:32:19.832Z
   [junit4]   2> 1954419 INFO  (OverseerStateUpdate-72216166038700047-127.0.0.1:40645_solr-n_0000000003) [n:127.0.0.1:40645_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:40645_solr
   [junit4]   2> 1954419 DEBUG (OverseerStateUpdate-72216166038700047-127.0.0.1:40645_solr-n_0000000003) [n:127.0.0.1:40645_solr    ] o.a.s.c.Overseer processMessage: queueSize: 6, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:40645_solr"} current state version: 0
   [junit4]   2> 1954420 DEBUG (OverseerStateUpdate-72216166038700047-127.0.0.1:40645_solr-n_0000000003) [n:127.0.0.1:40645_solr    ] o.a.s.c.Overseer processMessage: queueSize: 6, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:45295_solr"} current state version: 0
   [junit4]   2> 1954420 DEBUG (OverseerStateUpdate-72216166038700047-127.0.0.1:40645_solr-n_0000000003) [n:127.0.0.1:40645_solr    ] o.a.s.c.Overseer processMessage: queueSize: 6, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:44301_solr"} current state version: 0
   [junit4]   2> 1954420 DEBUG (OverseerStateUpdate-72216166038700047-127.0.0.1:40645_solr-n_0000000003) [n:127.0.0.1:40645_solr    ] o.a.s.c.Overseer processMessage: queueSize: 6, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:33021_solr"} current state version: 0
   [junit4]   2> 1954420 DEBUG (OverseerStateUpdate-72216166038700047-127.0.0.1:40645_solr-n_0000000003) [n:127.0.0.1:40645_solr    ] o.a.s.c.Overseer processMessage: queueSize: 6, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:45105_solr"} current state version: 0
   [junit4]   2> 1954420 DEBUG (OverseerStateUpdate-72216166038700047-127.0.0.1:40645_solr-n_0000000003) [n:127.0.0.1:40645_solr    ] o.a.s.c.Overseer processMessage: queueSize: 6, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:37539_solr"} current state version: 0
   [junit4]   2> 1954421 INFO  (zkConnectionManagerCallback-6140-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1954421 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1954505 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36551/solr
   [junit4]   2> 1954506 INFO  (zkConnectionManagerCallback-6144-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1954507 INFO  (zkConnectionManagerCallback-6146-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1954509 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1954511 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1954511 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34385_solr
   [junit4]   2> 1954511 DEBUG (OverseerStateUpdate-72216166038700047-127.0.0.1:40645_solr-n_0000000003) [n:127.0.0.1:40645_solr    ] o.a.s.c.Overseer processMessage: queueSize: 7, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:34385_solr"} current state version: 0
   [junit4]   2> 1954511 INFO  (zkCallback-6129-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954511 INFO  (zkCallback-6113-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954511 INFO  (zkCallback-6105-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954511 INFO  (zkCallback-6120-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954511 INFO  (zkCallback-6097-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954511 INFO  (zkCallback-6136-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954511 INFO  (zkCallback-6090-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954512 INFO  (zkCallback-6145-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1954523 INFO  (zkConnectionManagerCallback-6153-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1954524 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1954524 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36551/solr ready
   [junit4]   2> 1954524 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1954535 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34385.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1954541 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34385.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1954541 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34385.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@320c9086
   [junit4]   2> 1954542 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_D3FF2F40DCE2110D-001/tempDir-001/node1/.
   [junit4]   2> 1954582 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:39641/solr. Proxy: http://127.0.0.1:34385/solr
   [junit4]   2> 1954583 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@97cb26c{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1954583 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=446839305
   [junit4]   2> 1954583 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1954583 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@723032d5: rootName = solr_40645, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@56c0f2e3
   [junit4]   2> 1954588 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1954589 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@795d19e9: rootName = solr_40645, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@48bac951
   [junit4]   2> 1954591 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1954591 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@558d8e66: rootName = solr_40645, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1c3cc9d
   [junit4]   2> 1954592 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:40645_solr
   [junit4]   2> 1954593 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1954593 DEBUG (OverseerStateUpdate-72216166038700047-127.0.0.1:40645_solr-n_0000000003) [n:127.0.0.1:40645_solr    ] o.a.s.c.Overseer processMessage: queueSize: 8, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:40645_solr"} curren

[...truncated too long message...]

  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
   [junit4]   2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:674)
   [junit4]   2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
   [junit4]   2> at org.eclipse.jetty.server.Server.handle(Server.java:531)
   [junit4]   2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
   [junit4]   2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
   [junit4]   2> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
   [junit4]   2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue/qn-
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   [junit4]   2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:786)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:398)
   [junit4]   2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:398)
   [junit4]   2> at org.apache.solr.cloud.ZkDistributedQueue.offer(ZkDistributedQueue.java:321)
   [junit4]   2> at org.apache.solr.cloud.ZkController.unregister(ZkController.java:1605)
   [junit4]   2> at org.apache.solr.cloud.ZkController.unregister(ZkController.java:1573)
   [junit4]   2> at org.apache.solr.core.CoreContainer.unload(CoreContainer.java:1524)
   [junit4]   2> ... 40 more
   [junit4]   2>
   [junit4]   2> 1269612 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1269612 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[D3FF2F40DCE2110D]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41891 41891
   [junit4]   2> 1270395 INFO  (Thread-105) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41891 41891
   [junit4]   2> 1270396 WARN  (Thread-105) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 18 /solr/aliases.json
   [junit4]   2> 17 /solr/clusterprops.json
   [junit4]   2> 9 /solr/collections/pull_replica_error_handling_test_pull_replica_disconnects_from_zoo_keeper/terms/shard1
   [junit4]   2> 9 /solr/security.json
   [junit4]   2> 9 /solr/configs/conf
   [junit4]   2> 8 /solr/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/terms/shard1
   [junit4]   2> 8 /solr/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/terms/shard2
   [junit4]   2> 7 /solr/collections/pull_replica_error_handling_test_cant_connect_to_leader/terms/shard1
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 18 /solr/clusterstate.json
   [junit4]   2> 10 /solr/clusterprops.json
   [junit4]   2> 8 /solr/collections/pull_replica_error_handling_test_pull_replica_disconnects_from_zoo_keeper/state.json
   [junit4]   2> 6 /solr/autoscaling.json
   [junit4]   2> 4 /solr/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/state.json
   [junit4]   2> 2 /solr/collections/pull_replica_error_handling_test_cant_connect_to_leader/state.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 18 /solr/collections
   [junit4]   2> 17 /solr/live_nodes
   [junit4]   2> 6 /solr/overseer/queue
   [junit4]   2> 6 /solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 6 /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 6 /solr/overseer/collection-queue-work
   [junit4]   2> 6 /solr/overseer/queue-work
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplicaErrorHandling_D3FF2F40DCE2110D-002
   [junit4]   2> Sep 13, 2018 8:38:23 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene80): {fieldName_s=PostingsFormat(name=Direct), id=Lucene50(blocksize=128)}, docValues:{}, maxPointsInLeafNode=1934, maxMBSortInHeap=5.28365128294604, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@3f79a027), locale=en-IN, timezone=Europe/Athens
   [junit4]   2> NOTE: Linux 4.15.0-32-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=8,threads=1,free=351215608,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [TestPullReplicaErrorHandling, TestPullReplicaErrorHandling]
   [junit4] Completed [5/5 (5!)] on J0 in 628.55s, 3 tests, 1 error <<< FAILURES!
   [junit4]
   [junit4]
   [junit4] Tests with failures [seed: D3FF2F40DCE2110D]:
   [junit4]   - org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader
   [junit4]   - org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader
   [junit4]   - org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader
   [junit4]   - org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader
   [junit4]   - org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader
   [junit4]
   [junit4]
   [junit4] JVM J0:     0.43 ..  1271.42 =  1270.99s
   [junit4] JVM J1:     0.42 ..   730.62 =   730.20s
   [junit4] JVM J2:     0.43 ..  1260.38 =  1259.95s
   [junit4] Execution time total: 21 minutes 11 seconds
   [junit4] Tests summary: 5 suites, 15 tests, 5 errors

BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1567: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1092: There were test failures: 5 suites, 15 tests, 5 errors [seed: D3FF2F40DCE2110D]

Total time: 21 minutes 12 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   5/5 failed: org.apache.solr.cloud.TestPullReplicaErrorHandling
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=4a5b914eaa8683009191748bf6c0b1be14d59661, workspace=/var/lib/jenkins/workspace/Lucene-Solr-master-Linux
[WARNINGS] Computing warning deltas based on reference build #22849
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-11-ea+28) - Build # 22851 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/22851/
Java: 64bit/jdk-11-ea+28 -XX:+UseCompressedOops -XX:+UseSerialGC

14 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.handler.extraction.ExtractingRequestHandlerTest

Error Message:
SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.

Stack Trace:
java.lang.AssertionError: SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.
        at __randomizedtesting.SeedInfo.seed([1E0494F6BD610A4C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertFalse(Assert.java:68)
        at org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:44)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.extraction.ExtractingRequestHandlerTest

Error Message:
SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.

Stack Trace:
java.lang.AssertionError: SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.
        at __randomizedtesting.SeedInfo.seed([1E0494F6BD610A4C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertFalse(Assert.java:68)
        at org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:44)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.extraction.ExtractingRequestHandlerTest

Error Message:
SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.

Stack Trace:
java.lang.AssertionError: SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.
        at __randomizedtesting.SeedInfo.seed([1E0494F6BD610A4C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertFalse(Assert.java:68)
        at org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:44)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.extraction.ExtractingRequestHandlerTest

Error Message:
SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.

Stack Trace:
java.lang.AssertionError: SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.
        at __randomizedtesting.SeedInfo.seed([1E0494F6BD610A4C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertFalse(Assert.java:68)
        at org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:44)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger

Error Message:
Timed out waiting for collection:testMetricTrigger to become active null Live Nodes: [127.0.0.1:38467_solr, 127.0.0.1:39645_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for collection:testMetricTrigger to become active
null
Live Nodes: [127.0.0.1:38467_solr, 127.0.0.1:39645_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([FC5959527B2FEFEA:46556EDD24C739A5]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger(MetricTriggerIntegrationTest.java:95)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger

Error Message:
Timed out waiting for collection:testMetricTrigger to become active null Live Nodes: [127.0.0.1:34991_solr, 127.0.0.1:38503_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for collection:testMetricTrigger to become active
null
Live Nodes: [127.0.0.1:34991_solr, 127.0.0.1:38503_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([FC5959527B2FEFEA:46556EDD24C739A5]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger(MetricTriggerIntegrationTest.java:95)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger

Error Message:
{srt=[CapturedEvent{timestamp=1536843674261000000, stage=AFTER_ACTION, actionName='compute', event={   "id":"8aa92b20490dbTyw3o735czvqe26da5nglw0oq",   "source":"metric_trigger",   "eventTime":2439346842276059,   "eventType":"METRIC",   "properties":{     "node":{"127.0.0.1:32995_solr":13779},     "__start__":1,     "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n3:INDEX.sizeInBytes",     "collection":"testMetricTrigger",     "shard":"shard1",     "preferredOperation":"addreplica",     "_enqueue_time_":2439351868656498,     "requestedOps":[{         "action":"ADDREPLICA",         "hints":{           "COLL_SHARD":[{               "first":"testMetricTrigger",               "second":"shard1"}],           "SRC_NODE":["127.0.0.1:32995_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica, method=GET, params.action=ADDREPLICA, params.collection=testMetricTrigger, params.shard=shard1, params.node=127.0.0.1:34435_solr, params.type=NRT}], properties.BEFORE_ACTION=[compute], source=metric_trigger, properties.AFTER_ACTION=[compute]}, config={   "beforeAction":[],   "trigger":"metric_trigger",   "stage":[     "FAILED",     "SUCCEEDED"],   "afterAction":[     "compute",     "execute",     "test"],   "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536843679276000000, stage=AFTER_ACTION, actionName='execute', event={   "id":"8aa92b20490dbTyw3o735czvqe26da5nglw0oq",   "source":"metric_trigger",   "eventTime":2439346842276059,   "eventType":"METRIC",   "properties":{     "node":{"127.0.0.1:32995_solr":13779},     "__start__":1,     "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n3:INDEX.sizeInBytes",     "collection":"testMetricTrigger",     "shard":"shard1",     "preferredOperation":"addreplica",     "_enqueue_time_":2439351868656498,     "requestedOps":[{         "action":"ADDREPLICA",         "hints":{           "COLL_SHARD":[{               "first":"testMetricTrigger",               "second":"shard1"}],           "SRC_NODE":["127.0.0.1:32995_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica, method=GET, params.action=ADDREPLICA, params.async=metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/0, params.waitForFinalState=true, params.collection=testMetricTrigger, params.shard=shard1, params.node=127.0.0.1:34435_solr, params.type=NRT}], properties.BEFORE_ACTION=[compute, execute], properties.responses=[{responseHeader={status=0,QTime=1},success={127.0.0.1:34435_solr={responseHeader={status=0,QTime=0}}},metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647={responseHeader={status=0,QTime=0},STATUS=completed,Response=TaskId: metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647 webapp=null path=/admin/cores params={async=metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647&qt=/admin/cores&coreNodeName=core_node12&collection.configName=conf&name=testMetricTrigger_shard1_replica_n11&action=CREATE&collection=testMetricTrigger&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=0},status={state=completed,msg=found [metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/0] in completed tasks}}], source=metric_trigger, properties.AFTER_ACTION=[compute, execute]}, config={   "beforeAction":[],   "trigger":"metric_trigger",   "stage":[     "FAILED",     "SUCCEEDED"],   "afterAction":[     "compute",     "execute",     "test"],   "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536843679276000000, stage=AFTER_ACTION, actionName='test', event={   "id":"8aa92b20490dbTyw3o735czvqe26da5nglw0oq",   "source":"metric_trigger",   "eventTime":2439346842276059,   "eventType":"METRIC",   "properties":{     "node":{"127.0.0.1:32995_solr":13779},     "__start__":1,     "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n3:INDEX.sizeInBytes",     "collection":"testMetricTrigger",     "shard":"shard1",     "preferredOperation":"addreplica",     "_enqueue_time_":2439351868656498,     "requestedOps":[{         "action":"ADDREPLICA",         "hints":{           "COLL_SHARD":[{               "first":"testMetricTrigger",               "second":"shard1"}],           "SRC_NODE":["127.0.0.1:32995_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica, method=GET, params.action=ADDREPLICA, params.async=metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/0, params.waitForFinalState=true, params.collection=testMetricTrigger, params.shard=shard1, params.node=127.0.0.1:34435_solr, params.type=NRT}], properties.BEFORE_ACTION=[compute, execute, test], properties.responses=[{responseHeader={status=0,QTime=1},success={127.0.0.1:34435_solr={responseHeader={status=0,QTime=0}}},metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647={responseHeader={status=0,QTime=0},STATUS=completed,Response=TaskId: metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647 webapp=null path=/admin/cores params={async=metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647&qt=/admin/cores&coreNodeName=core_node12&collection.configName=conf&name=testMetricTrigger_shard1_replica_n11&action=CREATE&collection=testMetricTrigger&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=0},status={state=completed,msg=found [metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/0] in completed tasks}}], source=metric_trigger, properties.AFTER_ACTION=[compute, execute, test]}, config={   "beforeAction":[],   "trigger":"metric_trigger",   "stage":[     "FAILED",     "SUCCEEDED"],   "afterAction":[     "compute",     "execute",     "test"],   "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536843679277000000, stage=SUCCEEDED, actionName='null', event={   "id":"8aa92b20490dbTyw3o735czvqe26da5nglw0oq",   "source":"metric_trigger",   "eventTime":2439346842276059,   "eventType":"METRIC",   "properties":{     "node":{"127.0.0.1:32995_solr":13779},     "__start__":1,     "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n3:INDEX.sizeInBytes",     "collection":"testMetricTrigger",     "shard":"shard1",     "preferredOperation":"addreplica",     "_enqueue_time_":2439351868656498,     "requestedOps":[{         "action":"ADDREPLICA",         "hints":{           "COLL_SHARD":[{               "first":"testMetricTrigger",               "second":"shard1"}],           "SRC_NODE":["127.0.0.1:32995_solr"]}}]}}, context={}, config={   "beforeAction":[],   "trigger":"metric_trigger",   "stage":[     "FAILED",     "SUCCEEDED"],   "afterAction":[     "compute",     "execute",     "test"],   "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536844580042000000, stage=AFTER_ACTION, actionName='compute', event={   "id":"8ab659666f220Tyw3o735czvqe26da5nglw0r3",   "source":"metric_trigger",   "eventTime":2440252617060896,   "eventType":"METRIC",   "properties":{     "node":{"127.0.0.1:45851_solr":13602},     "__start__":1,     "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n1:INDEX.sizeInBytes",     "collection":"testMetricTrigger",     "shard":"shard1",     "preferredOperation":"movereplica",     "_enqueue_time_":2440257635924181,     "requestedOps":[{         "action":"MOVEREPLICA",         "hints":{           "COLL_SHARD":[{               "first":"testMetricTrigger",               "second":"shard1"}],           "SRC_NODE":["127.0.0.1:45851_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica, method=GET, params.action=MOVEREPLICA, params.collection=testMetricTrigger, params.targetNode=127.0.0.1:46761_solr, params.inPlaceMove=true, params.replica=core_node3}], properties.BEFORE_ACTION=[compute], source=metric_trigger, properties.AFTER_ACTION=[compute]}, config={   "beforeAction":[],   "trigger":"metric_trigger",   "stage":[     "FAILED",     "SUCCEEDED"],   "afterAction":[     "compute",     "execute",     "test"],   "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536844585053000000, stage=AFTER_ACTION, actionName='execute', event={   "id":"8ab659666f220Tyw3o735czvqe26da5nglw0r3",   "source":"metric_trigger",   "eventTime":2440252617060896,   "eventType":"METRIC",   "properties":{     "node":{"127.0.0.1:45851_solr":13602},     "__start__":1,     "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n1:INDEX.sizeInBytes",     "collection":"testMetricTrigger",     "shard":"shard1",     "preferredOperation":"movereplica",     "_enqueue_time_":2440257635924181,     "requestedOps":[{         "action":"MOVEREPLICA",         "hints":{           "COLL_SHARD":[{               "first":"testMetricTrigger",               "second":"shard1"}],           "SRC_NODE":["127.0.0.1:45851_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica, method=GET, params.action=MOVEREPLICA, params.async=metric_trigger/8ab659666f220Tyw3o735czvqe26da5nglw0r3/0, params.waitForFinalState=true, params.collection=testMetricTrigger, params.targetNode=127.0.0.1:46761_solr, params.inPlaceMove=true, params.replica=core_node3}], properties.BEFORE_ACTION=[compute, execute], properties.responses=[{responseHeader={status=0,QTime=1},success=MOVEREPLICA action completed successfully, moved replica=testMetricTrigger_shard1_replica_n1 at node=127.0.0.1:45851_solr to replica=testMetricTrigger_shard1_replica_n9 at node=127.0.0.1:46761_solr,status={state=completed,msg=found [metric_trigger/8ab659666f220Tyw3o735czvqe26da5nglw0r3/0] in completed tasks}}], source=metric_trigger, properties.AFTER_ACTION=[compute, execute]}, config={   "beforeAction":[],   "trigger":"metric_trigger",   "stage":[     "FAILED",     "SUCCEEDED"],   "afterAction":[     "compute",     "execute",     "test"],   "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536844585053000000, stage=AFTER_ACTION, actionName='test', event={   "id":"8ab659666f220Tyw3o735czvqe26da5nglw0r3",   "source":"metric_trigger",   "eventTime":2440252617060896,   "eventType":"METRIC",   "properties":{     "node":{"127.0.0.1:45851_solr":13602},     "__start__":1,     "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n1:INDEX.sizeInBytes",     "collection":"testMetricTrigger",     "shard":"shard1",     "preferredOperation":"movereplica",     "_enqueue_time_":2440257635924181,     "requestedOps":[{         "action":"MOVEREPLICA",         "hints":{           "COLL_SHARD":[{               "first":"testMetricTrigger",               "second":"shard1"}],           "SRC_NODE":["127.0.0.1:45851_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica, method=GET, params.action=MOVEREPLICA, params.async=metric_trigger/8ab659666f220Tyw3o735czvqe26da5nglw0r3/0, params.waitForFinalState=true, params.collection=testMetricTrigger, params.targetNode=127.0.0.1:46761_solr, params.inPlaceMove=true, params.replica=core_node3}], properties.BEFORE_ACTION=[compute, execute, test], properties.responses=[{responseHeader={status=0,QTime=1},success=MOVEREPLICA action completed successfully, moved replica=testMetricTrigger_shard1_replica_n1 at node=127.0.0.1:45851_solr to replica=testMetricTrigger_shard1_replica_n9 at node=127.0.0.1:46761_solr,status={state=completed,msg=found [metric_trigger/8ab659666f220Tyw3o735czvqe26da5nglw0r3/0] in completed tasks}}], source=metric_trigger, properties.AFTER_ACTION=[compute, execute, test]}, config={   "beforeAction":[],   "trigger":"metric_trigger",   "stage":[     "FAILED",     "SUCCEEDED"],   "afterAction":[     "compute",     "execute",     "test"],   "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536844585054000000, stage=SUCCEEDED, actionName='null', event={   "id":"8ab659666f220Tyw3o735czvqe26da5nglw0r3",   "source":"metric_trigger",   "eventTime":2440252617060896,   "eventType":"METRIC",   "properties":{     "node":{"127.0.0.1:45851_solr":13602},     "__start__":1,     "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n1:INDEX.sizeInBytes",     "collection":"testMetricTrigger",     "shard":"shard1",     "preferredOperation":"movereplica",     "_enqueue_time_":2440257635924181,     "requestedOps":[{         "action":"MOVEREPLICA",         "hints":{           "COLL_SHARD":[{               "first":"testMetricTrigger",               "second":"shard1"}],           "SRC_NODE":["127.0.0.1:45851_solr"]}}]}}, context={}, config={   "beforeAction":[],   "trigger":"metric_trigger",   "stage":[     "FAILED",     "SUCCEEDED"],   "afterAction":[     "compute",     "execute",     "test"],   "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}]} expected:<4> but was:<8>

Stack Trace:
java.lang.AssertionError: {srt=[CapturedEvent{timestamp=1536843674261000000, stage=AFTER_ACTION, actionName='compute', event={
  "id":"8aa92b20490dbTyw3o735czvqe26da5nglw0oq",
  "source":"metric_trigger",
  "eventTime":2439346842276059,
  "eventType":"METRIC",
  "properties":{
    "node":{"127.0.0.1:32995_solr":13779},
    "__start__":1,
    "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n3:INDEX.sizeInBytes",
    "collection":"testMetricTrigger",
    "shard":"shard1",
    "preferredOperation":"addreplica",
    "_enqueue_time_":2439351868656498,
    "requestedOps":[{
        "action":"ADDREPLICA",
        "hints":{
          "COLL_SHARD":[{
              "first":"testMetricTrigger",
              "second":"shard1"}],
          "SRC_NODE":["127.0.0.1:32995_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica, method=GET, params.action=ADDREPLICA, params.collection=testMetricTrigger, params.shard=shard1, params.node=127.0.0.1:34435_solr, params.type=NRT}], properties.BEFORE_ACTION=[compute], source=metric_trigger, properties.AFTER_ACTION=[compute]}, config={
  "beforeAction":[],
  "trigger":"metric_trigger",
  "stage":[
    "FAILED",
    "SUCCEEDED"],
  "afterAction":[
    "compute",
    "execute",
    "test"],
  "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536843679276000000, stage=AFTER_ACTION, actionName='execute', event={
  "id":"8aa92b20490dbTyw3o735czvqe26da5nglw0oq",
  "source":"metric_trigger",
  "eventTime":2439346842276059,
  "eventType":"METRIC",
  "properties":{
    "node":{"127.0.0.1:32995_solr":13779},
    "__start__":1,
    "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n3:INDEX.sizeInBytes",
    "collection":"testMetricTrigger",
    "shard":"shard1",
    "preferredOperation":"addreplica",
    "_enqueue_time_":2439351868656498,
    "requestedOps":[{
        "action":"ADDREPLICA",
        "hints":{
          "COLL_SHARD":[{
              "first":"testMetricTrigger",
              "second":"shard1"}],
          "SRC_NODE":["127.0.0.1:32995_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica, method=GET, params.action=ADDREPLICA, params.async=metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/0, params.waitForFinalState=true, params.collection=testMetricTrigger, params.shard=shard1, params.node=127.0.0.1:34435_solr, params.type=NRT}], properties.BEFORE_ACTION=[compute, execute], properties.responses=[{responseHeader={status=0,QTime=1},success={127.0.0.1:34435_solr={responseHeader={status=0,QTime=0}}},metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647={responseHeader={status=0,QTime=0},STATUS=completed,Response=TaskId: metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647 webapp=null path=/admin/cores params={async=metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647&qt=/admin/cores&coreNodeName=core_node12&collection.configName=conf&name=testMetricTrigger_shard1_replica_n11&action=CREATE&collection=testMetricTrigger&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=0},status={state=completed,msg=found [metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/0] in completed tasks}}], source=metric_trigger, properties.AFTER_ACTION=[compute, execute]}, config={
  "beforeAction":[],
  "trigger":"metric_trigger",
  "stage":[
    "FAILED",
    "SUCCEEDED"],
  "afterAction":[
    "compute",
    "execute",
    "test"],
  "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536843679276000000, stage=AFTER_ACTION, actionName='test', event={
  "id":"8aa92b20490dbTyw3o735czvqe26da5nglw0oq",
  "source":"metric_trigger",
  "eventTime":2439346842276059,
  "eventType":"METRIC",
  "properties":{
    "node":{"127.0.0.1:32995_solr":13779},
    "__start__":1,
    "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n3:INDEX.sizeInBytes",
    "collection":"testMetricTrigger",
    "shard":"shard1",
    "preferredOperation":"addreplica",
    "_enqueue_time_":2439351868656498,
    "requestedOps":[{
        "action":"ADDREPLICA",
        "hints":{
          "COLL_SHARD":[{
              "first":"testMetricTrigger",
              "second":"shard1"}],
          "SRC_NODE":["127.0.0.1:32995_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica, method=GET, params.action=ADDREPLICA, params.async=metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/0, params.waitForFinalState=true, params.collection=testMetricTrigger, params.shard=shard1, params.node=127.0.0.1:34435_solr, params.type=NRT}], properties.BEFORE_ACTION=[compute, execute, test], properties.responses=[{responseHeader={status=0,QTime=1},success={127.0.0.1:34435_solr={responseHeader={status=0,QTime=0}}},metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647={responseHeader={status=0,QTime=0},STATUS=completed,Response=TaskId: metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647 webapp=null path=/admin/cores params={async=metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647&qt=/admin/cores&coreNodeName=core_node12&collection.configName=conf&name=testMetricTrigger_shard1_replica_n11&action=CREATE&collection=testMetricTrigger&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=0},status={state=completed,msg=found [metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/0] in completed tasks}}], source=metric_trigger, properties.AFTER_ACTION=[compute, execute, test]}, config={
  "beforeAction":[],
  "trigger":"metric_trigger",
  "stage":[
    "FAILED",
    "SUCCEEDED"],
  "afterAction":[
    "compute",
    "execute",
    "test"],
  "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536843679277000000, stage=SUCCEEDED, actionName='null', event={
  "id":"8aa92b20490dbTyw3o735czvqe26da5nglw0oq",
  "source":"metric_trigger",
  "eventTime":2439346842276059,
  "eventType":"METRIC",
  "properties":{
    "node":{"127.0.0.1:32995_solr":13779},
    "__start__":1,
    "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n3:INDEX.sizeInBytes",
    "collection":"testMetricTrigger",
    "shard":"shard1",
    "preferredOperation":"addreplica",
    "_enqueue_time_":2439351868656498,
    "requestedOps":[{
        "action":"ADDREPLICA",
        "hints":{
          "COLL_SHARD":[{
              "first":"testMetricTrigger",
              "second":"shard1"}],
          "SRC_NODE":["127.0.0.1:32995_solr"]}}]}}, context={}, config={
  "beforeAction":[],
  "trigger":"metric_trigger",
  "stage":[
    "FAILED",
    "SUCCEEDED"],
  "afterAction":[
    "compute",
    "execute",
    "test"],
  "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536844580042000000, stage=AFTER_ACTION, actionName='compute', event={
  "id":"8ab659666f220Tyw3o735czvqe26da5nglw0r3",
  "source":"metric_trigger",
  "eventTime":2440252617060896,
  "eventType":"METRIC",
  "properties":{
    "node":{"127.0.0.1:45851_solr":13602},
    "__start__":1,
    "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n1:INDEX.sizeInBytes",
    "collection":"testMetricTrigger",
    "shard":"shard1",
    "preferredOperation":"movereplica",
    "_enqueue_time_":2440257635924181,
    "requestedOps":[{
        "action":"MOVEREPLICA",
        "hints":{
          "COLL_SHARD":[{
              "first":"testMetricTrigger",
              "second":"shard1"}],
          "SRC_NODE":["127.0.0.1:45851_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica, method=GET, params.action=MOVEREPLICA, params.collection=testMetricTrigger, params.targetNode=127.0.0.1:46761_solr, params.inPlaceMove=true, params.replica=core_node3}], properties.BEFORE_ACTION=[compute], source=metric_trigger, properties.AFTER_ACTION=[compute]}, config={
  "beforeAction":[],
  "trigger":"metric_trigger",
  "stage":[
    "FAILED",
    "SUCCEEDED"],
  "afterAction":[
    "compute",
    "execute",
    "test"],
  "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536844585053000000, stage=AFTER_ACTION, actionName='execute', event={
  "id":"8ab659666f220Tyw3o735czvqe26da5nglw0r3",
  "source":"metric_trigger",
  "eventTime":2440252617060896,
  "eventType":"METRIC",
  "properties":{
    "node":{"127.0.0.1:45851_solr":13602},
    "__start__":1,
    "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n1:INDEX.sizeInBytes",
    "collection":"testMetricTrigger",
    "shard":"shard1",
    "preferredOperation":"movereplica",
    "_enqueue_time_":2440257635924181,
    "requestedOps":[{
        "action":"MOVEREPLICA",
        "hints":{
          "COLL_SHARD":[{
              "first":"testMetricTrigger",
              "second":"shard1"}],
          "SRC_NODE":["127.0.0.1:45851_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica, method=GET, params.action=MOVEREPLICA, params.async=metric_trigger/8ab659666f220Tyw3o735czvqe26da5nglw0r3/0, params.waitForFinalState=true, params.collection=testMetricTrigger, params.targetNode=127.0.0.1:46761_solr, params.inPlaceMove=true, params.replica=core_node3}], properties.BEFORE_ACTION=[compute, execute], properties.responses=[{responseHeader={status=0,QTime=1},success=MOVEREPLICA action completed successfully, moved replica=testMetricTrigger_shard1_replica_n1 at node=127.0.0.1:45851_solr to replica=testMetricTrigger_shard1_replica_n9 at node=127.0.0.1:46761_solr,status={state=completed,msg=found [metric_trigger/8ab659666f220Tyw3o735czvqe26da5nglw0r3/0] in completed tasks}}], source=metric_trigger, properties.AFTER_ACTION=[compute, execute]}, config={
  "beforeAction":[],
  "trigger":"metric_trigger",
  "stage":[
    "FAILED",
    "SUCCEEDED"],
  "afterAction":[
    "compute",
    "execute",
    "test"],
  "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536844585053000000, stage=AFTER_ACTION, actionName='test', event={
  "id":"8ab659666f220Tyw3o735czvqe26da5nglw0r3",
  "source":"metric_trigger",
  "eventTime":2440252617060896,
  "eventType":"METRIC",
  "properties":{
    "node":{"127.0.0.1:45851_solr":13602},
    "__start__":1,
    "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n1:INDEX.sizeInBytes",
    "collection":"testMetricTrigger",
    "shard":"shard1",
    "preferredOperation":"movereplica",
    "_enqueue_time_":2440257635924181,
    "requestedOps":[{
        "action":"MOVEREPLICA",
        "hints":{
          "COLL_SHARD":[{
              "first":"testMetricTrigger",
              "second":"shard1"}],
          "SRC_NODE":["127.0.0.1:45851_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica, method=GET, params.action=MOVEREPLICA, params.async=metric_trigger/8ab659666f220Tyw3o735czvqe26da5nglw0r3/0, params.waitForFinalState=true, params.collection=testMetricTrigger, params.targetNode=127.0.0.1:46761_solr, params.inPlaceMove=true, params.replica=core_node3}], properties.BEFORE_ACTION=[compute, execute, test], properties.responses=[{responseHeader={status=0,QTime=1},success=MOVEREPLICA action completed successfully, moved replica=testMetricTrigger_shard1_replica_n1 at node=127.0.0.1:45851_solr to replica=testMetricTrigger_shard1_replica_n9 at node=127.0.0.1:46761_solr,status={state=completed,msg=found [metric_trigger/8ab659666f220Tyw3o735czvqe26da5nglw0r3/0] in completed tasks}}], source=metric_trigger, properties.AFTER_ACTION=[compute, execute, test]}, config={
  "beforeAction":[],
  "trigger":"metric_trigger",
  "stage":[
    "FAILED",
    "SUCCEEDED"],
  "afterAction":[
    "compute",
    "execute",
    "test"],
  "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536844585054000000, stage=SUCCEEDED, actionName='null', event={
  "id":"8ab659666f220Tyw3o735czvqe26da5nglw0r3",
  "source":"metric_trigger",
  "eventTime":2440252617060896,
  "eventType":"METRIC",
  "properties":{
    "node":{"127.0.0.1:45851_solr":13602},
    "__start__":1,
    "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n1:INDEX.sizeInBytes",
    "collection":"testMetricTrigger",
    "shard":"shard1",
    "preferredOperation":"movereplica",
    "_enqueue_time_":2440257635924181,
    "requestedOps":[{
        "action":"MOVEREPLICA",
        "hints":{
          "COLL_SHARD":[{
              "first":"testMetricTrigger",
              "second":"shard1"}],
          "SRC_NODE":["127.0.0.1:45851_solr"]}}]}}, context={}, config={
  "beforeAction":[],
  "trigger":"metric_trigger",
  "stage":[
    "FAILED",
    "SUCCEEDED"],
  "afterAction":[
    "compute",
    "execute",
    "test"],
  "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}]} expected:<4> but was:<8>
        at __randomizedtesting.SeedInfo.seed([FC5959527B2FEFEA:46556EDD24C739A5]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger(MetricTriggerIntegrationTest.java:157)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger

Error Message:
Timed out waiting for collection:testMetricTrigger to become active null Live Nodes: [127.0.0.1:34535_solr, 127.0.0.1:40005_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for collection:testMetricTrigger to become active
null
Live Nodes: [127.0.0.1:34535_solr, 127.0.0.1:40005_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([FC5959527B2FEFEA:46556EDD24C739A5]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger(MetricTriggerIntegrationTest.java:95)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger

Error Message:
Timed out waiting for collection:testMetricTrigger to become active null Live Nodes: [127.0.0.1:38491_solr, 127.0.0.1:45759_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for collection:testMetricTrigger to become active
null
Live Nodes: [127.0.0.1:38491_solr, 127.0.0.1:45759_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([FC5959527B2FEFEA:46556EDD24C739A5]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger(MetricTriggerIntegrationTest.java:95)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:33219/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33219/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([FC5959527B2FEFEA:725539220E376BC0]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:34339/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:34339/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([FC5959527B2FEFEA:725539220E376BC0]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:46673/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:46673/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([FC5959527B2FEFEA:725539220E376BC0]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:46727/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:46727/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([FC5959527B2FEFEA:725539220E376BC0]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:40379/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:40379/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([FC5959527B2FEFEA:725539220E376BC0]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at jdk.internal.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)




Build Log:
[...truncated 14128 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.MetricTriggerIntegrationTest_FC5959527B2FEFEA-001/init-core-data-001
   [junit4]   2> 1766257 WARN  (SUITE-MetricTriggerIntegrationTest-seed#[FC5959527B2FEFEA]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
   [junit4]   2> 1766257 INFO  (SUITE-MetricTriggerIntegrationTest-seed#[FC5959527B2FEFEA]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1766258 INFO  (SUITE-MetricTriggerIntegrationTest-seed#[FC5959527B2FEFEA]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
   [junit4]   2> 1766258 INFO  (SUITE-MetricTriggerIntegrationTest-seed#[FC5959527B2FEFEA]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1766258 INFO  (SUITE-MetricTriggerIntegrationTest-seed#[FC5959527B2FEFEA]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.MetricTriggerIntegrationTest_FC5959527B2FEFEA-001/tempDir-001
   [junit4]   2> 1766258 INFO  (SUITE-MetricTriggerIntegrationTest-seed#[FC5959527B2FEFEA]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1766259 INFO  (Thread-3720) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1766259 INFO  (Thread-3720) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1766260 ERROR (Thread-3720) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1766359 INFO  (SUITE-MetricTriggerIntegrationTest-seed#[FC5959527B2FEFEA]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:33829
   [junit4]   2> 1766360 INFO  (zkConnectionManagerCallback-10529-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1766364 INFO  (jetty-launcher-10526-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 11+28
   [junit4]   2> 1766364 INFO  (jetty-launcher-10526-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 11+28
   [junit4]   2> 1766365 INFO  (jetty-launcher-10526-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1766365 INFO  (jetty-launcher-10526-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1766365 INFO  (jetty-launcher-10526-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1766365 INFO  (jetty-launcher-10526-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1766365 INFO  (jetty-launcher-10526-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1766365 INFO  (jetty-launcher-10526-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1766365 INFO  (jetty-launcher-10526-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@a751483{/solr,null,

[...truncated too long message...]

erCnxn Unable to read additional data from client sessionid 0x100916ef7bb001a, likely client has closed socket
   [junit4]   2> 1274684 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100916ef7bb001d, likely client has closed socket
   [junit4]   2> 1274684 INFO  (zkCallback-278-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1274684 INFO  (zkCallback-271-thread-3) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1274684 INFO  (zkCallback-223-thread-6) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1274684 INFO  (zkCallback-239-thread-7) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1274688 INFO  (jetty-closer-143-thread-4) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@770a063e{/solr,null,UNAVAILABLE}
   [junit4]   2> 1274688 INFO  (jetty-closer-143-thread-4) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1276184 WARN  (zkCallback-246-thread-2) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1276184 WARN  (zkCallback-255-thread-4) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1276184 WARN  (zkCallback-230-thread-2) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 1276184 INFO  (jetty-closer-143-thread-3) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4c3a2a80{/solr,null,UNAVAILABLE}
   [junit4]   2> 1276185 INFO  (jetty-closer-143-thread-3) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1276185 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100916ef7bb0023, likely client has closed socket
   [junit4]   2> 1276186 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100916ef7bb0015, likely client has closed socket
   [junit4]   2> 1276186 INFO  (jetty-closer-143-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@592655ba{/solr,null,UNAVAILABLE}
   [junit4]   2> 1276187 INFO  (jetty-closer-143-thread-2) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1276187 INFO  (jetty-closer-143-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@62300756{/solr,null,UNAVAILABLE}
   [junit4]   2> 1276188 INFO  (jetty-closer-143-thread-1) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1276188 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[FC5959527B2FEFEA]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1276188 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[FC5959527B2FEFEA]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41371 41371
   [junit4]   2> 1276233 INFO  (Thread-98) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41371 41371
   [junit4]   2> 1276234 WARN  (Thread-98) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 18 /solr/aliases.json
   [junit4]   2> 17 /solr/clusterprops.json
   [junit4]   2> 9 /solr/collections/pull_replica_error_handling_test_pull_replica_disconnects_from_zoo_keeper/terms/shard1
   [junit4]   2> 9 /solr/security.json
   [junit4]   2> 8 /solr/collections/pull_replica_error_handling_test_cant_connect_to_leader/terms/shard1
   [junit4]   2> 8 /solr/configs/conf
   [junit4]   2> 7 /solr/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/terms/shard2
   [junit4]   2> 6 /solr/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/terms/shard1
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 18 /solr/clusterstate.json
   [junit4]   2> 10 /solr/clusterprops.json
   [junit4]   2> 7 /solr/collections/pull_replica_error_handling_test_pull_replica_disconnects_from_zoo_keeper/state.json
   [junit4]   2> 4 /solr/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/state.json
   [junit4]   2> 4 /solr/overseer_elect/election/72217499826585621-127.0.0.1:46117_solr-n_0000000004
   [junit4]   2> 3 /solr/overseer_elect/election/72217499826585610-127.0.0.1:36779_solr-n_0000000001
   [junit4]   2> 3 /solr/autoscaling.json
   [junit4]   2> 2 /solr/collections/pull_replica_error_handling_test_cant_connect_to_leader/state.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 18 /solr/collections
   [junit4]   2> 17 /solr/live_nodes
   [junit4]   2> 3 /solr/overseer/queue
   [junit4]   2> 3 /solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 3 /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 3 /solr/overseer/collection-queue-work
   [junit4]   2> 3 /solr/overseer/queue-work
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplicaErrorHandling_FC5959527B2FEFEA-002
   [junit4]   2> Sep 13, 2018 1:22:00 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 3 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=16012, maxDocsPerChunk=4, blockSize=975), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=16012, blockSize=975)), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@7deea640), locale=ff-SN, timezone=PLT
   [junit4]   2> NOTE: Linux 4.15.0-32-generic amd64/Oracle Corporation 11 (64-bit)/cpus=8,threads=1,free=385394456,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [TestPullReplicaErrorHandling, TestPullReplicaErrorHandling]
   [junit4] Completed [10/10 (9!)] on J0 in 630.44s, 3 tests, 1 error <<< FAILURES!
   [junit4]
   [junit4]
   [junit4] Tests with failures [seed: FC5959527B2FEFEA]:
   [junit4]   - org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger
   [junit4]   - org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger
   [junit4]   - org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader
   [junit4]   - org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader
   [junit4]   - org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader
   [junit4]   - org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger
   [junit4]   - org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger
   [junit4]   - org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader
   [junit4]   - org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader
   [junit4]
   [junit4]
   [junit4] JVM J0:     0.87 ..  1278.29 =  1277.42s
   [junit4] JVM J1:     0.87 ..  1271.09 =  1270.22s
   [junit4] JVM J2:     0.87 ..  1080.31 =  1079.44s
   [junit4] Execution time total: 21 minutes 18 seconds
   [junit4] Tests summary: 10 suites, 20 tests, 5 errors, 4 failures

BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1567: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1092: There were test failures: 10 suites, 20 tests, 5 errors, 4 failures [seed: FC5959527B2FEFEA]

Total time: 21 minutes 20 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   4/5 failed: org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest
[repro]   5/5 failed: org.apache.solr.cloud.TestPullReplicaErrorHandling
[repro]   5/5 failed: org.apache.solr.handler.extraction.ExtractingRequestHandlerTest
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=4a5b914eaa8683009191748bf6c0b1be14d59661, workspace=/var/lib/jenkins/workspace/Lucene-Solr-master-Linux
[WARNINGS] Computing warning deltas based on reference build #22850
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-11-ea+28) - Build # 22851 - Still Unstable!

david.w.smiley@gmail.com
Woops; I now fixed the assertFalse that was meant to be assumeFalse in ExtractingRequestHandlerTest.beforeClass

On Thu, Sep 13, 2018 at 9:30 AM Policeman Jenkins Server <[hidden email]> wrote:
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/22851/
Java: 64bit/jdk-11-ea+28 -XX:+UseCompressedOops -XX:+UseSerialGC

14 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.handler.extraction.ExtractingRequestHandlerTest

Error Message:
SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.

Stack Trace:
java.lang.AssertionError: SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.
        at __randomizedtesting.SeedInfo.seed([1E0494F6BD610A4C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertFalse(Assert.java:68)
        at org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:44)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.extraction.ExtractingRequestHandlerTest

Error Message:
SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.

Stack Trace:
java.lang.AssertionError: SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.
        at __randomizedtesting.SeedInfo.seed([1E0494F6BD610A4C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertFalse(Assert.java:68)
        at org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:44)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.extraction.ExtractingRequestHandlerTest

Error Message:
SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.

Stack Trace:
java.lang.AssertionError: SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.
        at __randomizedtesting.SeedInfo.seed([1E0494F6BD610A4C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertFalse(Assert.java:68)
        at org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:44)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.extraction.ExtractingRequestHandlerTest

Error Message:
SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.

Stack Trace:
java.lang.AssertionError: SOLR-12759 JDK 11 (1st release) and Tika 1.x can result in extracting dates in a bad format.
        at __randomizedtesting.SeedInfo.seed([1E0494F6BD610A4C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertFalse(Assert.java:68)
        at org.apache.solr.handler.extraction.ExtractingRequestHandlerTest.beforeClass(ExtractingRequestHandlerTest.java:44)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger

Error Message:
Timed out waiting for collection:testMetricTrigger to become active null Live Nodes: [127.0.0.1:38467_solr, 127.0.0.1:39645_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for collection:testMetricTrigger to become active
null
Live Nodes: [127.0.0.1:38467_solr, 127.0.0.1:39645_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([FC5959527B2FEFEA:46556EDD24C739A5]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger(MetricTriggerIntegrationTest.java:95)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger

Error Message:
Timed out waiting for collection:testMetricTrigger to become active null Live Nodes: [127.0.0.1:34991_solr, 127.0.0.1:38503_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for collection:testMetricTrigger to become active
null
Live Nodes: [127.0.0.1:34991_solr, 127.0.0.1:38503_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([FC5959527B2FEFEA:46556EDD24C739A5]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger(MetricTriggerIntegrationTest.java:95)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger

Error Message:
{srt=[CapturedEvent{timestamp=1536843674261000000, stage=AFTER_ACTION, actionName='compute', event={   "id":"8aa92b20490dbTyw3o735czvqe26da5nglw0oq",   "source":"metric_trigger",   "eventTime":2439346842276059,   "eventType":"METRIC",   "properties":{     "node":{"127.0.0.1:32995_solr":13779},     "__start__":1,     "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n3:INDEX.sizeInBytes",     "collection":"testMetricTrigger",     "shard":"shard1",     "preferredOperation":"addreplica",     "_enqueue_time_":2439351868656498,     "requestedOps":[{         "action":"ADDREPLICA",         "hints":{           "COLL_SHARD":[{               "first":"testMetricTrigger",               "second":"shard1"}],           "SRC_NODE":["127.0.0.1:32995_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica, method=GET, params.action=ADDREPLICA, params.collection=testMetricTrigger, params.shard=shard1, params.node=127.0.0.1:34435_solr, params.type=NRT}], properties.BEFORE_ACTION=[compute], source=metric_trigger, properties.AFTER_ACTION=[compute]}, config={   "beforeAction":[],   "trigger":"metric_trigger",   "stage":[     "FAILED",     "SUCCEEDED"],   "afterAction":[     "compute",     "execute",     "test"],   "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536843679276000000, stage=AFTER_ACTION, actionName='execute', event={   "id":"8aa92b20490dbTyw3o735czvqe26da5nglw0oq",   "source":"metric_trigger",   "eventTime":2439346842276059,   "eventType":"METRIC",   "properties":{     "node":{"127.0.0.1:32995_solr":13779},     "__start__":1,     "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n3:INDEX.sizeInBytes",     "collection":"testMetricTrigger",     "shard":"shard1",     "preferredOperation":"addreplica",     "_enqueue_time_":2439351868656498,     "requestedOps":[{         "action":"ADDREPLICA",         "hints":{           "COLL_SHARD":[{               "first":"testMetricTrigger",               "second":"shard1"}],           "SRC_NODE":["127.0.0.1:32995_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica, method=GET, params.action=ADDREPLICA, params.async=metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/0, params.waitForFinalState=true, params.collection=testMetricTrigger, params.shard=shard1, params.node=127.0.0.1:34435_solr, params.type=NRT}], properties.BEFORE_ACTION=[compute, execute], properties.responses=[{responseHeader={status=0,QTime=1},success={127.0.0.1:34435_solr={responseHeader={status=0,QTime=0}}},metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647={responseHeader={status=0,QTime=0},STATUS=completed,Response=TaskId: metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647 webapp=null path=/admin/cores params={async=metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647&qt=/admin/cores&coreNodeName=core_node12&collection.configName=conf&name=testMetricTrigger_shard1_replica_n11&action=CREATE&collection=testMetricTrigger&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=0},status={state=completed,msg=found [metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/0] in completed tasks}}], source=metric_trigger, properties.AFTER_ACTION=[compute, execute]}, config={   "beforeAction":[],   "trigger":"metric_trigger",   "stage":[     "FAILED",     "SUCCEEDED"],   "afterAction":[     "compute",     "execute",     "test"],   "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536843679276000000, stage=AFTER_ACTION, actionName='test', event={   "id":"8aa92b20490dbTyw3o735czvqe26da5nglw0oq",   "source":"metric_trigger",   "eventTime":2439346842276059,   "eventType":"METRIC",   "properties":{     "node":{"127.0.0.1:32995_solr":13779},     "__start__":1,     "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n3:INDEX.sizeInBytes",     "collection":"testMetricTrigger",     "shard":"shard1",     "preferredOperation":"addreplica",     "_enqueue_time_":2439351868656498,     "requestedOps":[{         "action":"ADDREPLICA",         "hints":{           "COLL_SHARD":[{               "first":"testMetricTrigger",               "second":"shard1"}],           "SRC_NODE":["127.0.0.1:32995_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica, method=GET, params.action=ADDREPLICA, params.async=metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/0, params.waitForFinalState=true, params.collection=testMetricTrigger, params.shard=shard1, params.node=127.0.0.1:34435_solr, params.type=NRT}], properties.BEFORE_ACTION=[compute, execute, test], properties.responses=[{responseHeader={status=0,QTime=1},success={127.0.0.1:34435_solr={responseHeader={status=0,QTime=0}}},metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647={responseHeader={status=0,QTime=0},STATUS=completed,Response=TaskId: metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647 webapp=null path=/admin/cores params={async=metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647&qt=/admin/cores&coreNodeName=core_node12&collection.configName=conf&name=testMetricTrigger_shard1_replica_n11&action=CREATE&collection=testMetricTrigger&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=0},status={state=completed,msg=found [metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/0] in completed tasks}}], source=metric_trigger, properties.AFTER_ACTION=[compute, execute, test]}, config={   "beforeAction":[],   "trigger":"metric_trigger",   "stage":[     "FAILED",     "SUCCEEDED"],   "afterAction":[     "compute",     "execute",     "test"],   "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536843679277000000, stage=SUCCEEDED, actionName='null', event={   "id":"8aa92b20490dbTyw3o735czvqe26da5nglw0oq",   "source":"metric_trigger",   "eventTime":2439346842276059,   "eventType":"METRIC",   "properties":{     "node":{"127.0.0.1:32995_solr":13779},     "__start__":1,     "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n3:INDEX.sizeInBytes",     "collection":"testMetricTrigger",     "shard":"shard1",     "preferredOperation":"addreplica",     "_enqueue_time_":2439351868656498,     "requestedOps":[{         "action":"ADDREPLICA",         "hints":{           "COLL_SHARD":[{               "first":"testMetricTrigger",               "second":"shard1"}],           "SRC_NODE":["127.0.0.1:32995_solr"]}}]}}, context={}, config={   "beforeAction":[],   "trigger":"metric_trigger",   "stage":[     "FAILED",     "SUCCEEDED"],   "afterAction":[     "compute",     "execute",     "test"],   "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536844580042000000, stage=AFTER_ACTION, actionName='compute', event={   "id":"8ab659666f220Tyw3o735czvqe26da5nglw0r3",   "source":"metric_trigger",   "eventTime":2440252617060896,   "eventType":"METRIC",   "properties":{     "node":{"127.0.0.1:45851_solr":13602},     "__start__":1,     "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n1:INDEX.sizeInBytes",     "collection":"testMetricTrigger",     "shard":"shard1",     "preferredOperation":"movereplica",     "_enqueue_time_":2440257635924181,     "requestedOps":[{         "action":"MOVEREPLICA",         "hints":{           "COLL_SHARD":[{               "first":"testMetricTrigger",               "second":"shard1"}],           "SRC_NODE":["127.0.0.1:45851_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica, method=GET, params.action=MOVEREPLICA, params.collection=testMetricTrigger, params.targetNode=127.0.0.1:46761_solr, params.inPlaceMove=true, params.replica=core_node3}], properties.BEFORE_ACTION=[compute], source=metric_trigger, properties.AFTER_ACTION=[compute]}, config={   "beforeAction":[],   "trigger":"metric_trigger",   "stage":[     "FAILED",     "SUCCEEDED"],   "afterAction":[     "compute",     "execute",     "test"],   "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536844585053000000, stage=AFTER_ACTION, actionName='execute', event={   "id":"8ab659666f220Tyw3o735czvqe26da5nglw0r3",   "source":"metric_trigger",   "eventTime":2440252617060896,   "eventType":"METRIC",   "properties":{     "node":{"127.0.0.1:45851_solr":13602},     "__start__":1,     "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n1:INDEX.sizeInBytes",     "collection":"testMetricTrigger",     "shard":"shard1",     "preferredOperation":"movereplica",     "_enqueue_time_":2440257635924181,     "requestedOps":[{         "action":"MOVEREPLICA",         "hints":{           "COLL_SHARD":[{               "first":"testMetricTrigger",               "second":"shard1"}],           "SRC_NODE":["127.0.0.1:45851_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica, method=GET, params.action=MOVEREPLICA, params.async=metric_trigger/8ab659666f220Tyw3o735czvqe26da5nglw0r3/0, params.waitForFinalState=true, params.collection=testMetricTrigger, params.targetNode=127.0.0.1:46761_solr, params.inPlaceMove=true, params.replica=core_node3}], properties.BEFORE_ACTION=[compute, execute], properties.responses=[{responseHeader={status=0,QTime=1},success=MOVEREPLICA action completed successfully, moved replica=testMetricTrigger_shard1_replica_n1 at node=127.0.0.1:45851_solr to replica=testMetricTrigger_shard1_replica_n9 at node=127.0.0.1:46761_solr,status={state=completed,msg=found [metric_trigger/8ab659666f220Tyw3o735czvqe26da5nglw0r3/0] in completed tasks}}], source=metric_trigger, properties.AFTER_ACTION=[compute, execute]}, config={   "beforeAction":[],   "trigger":"metric_trigger",   "stage":[     "FAILED",     "SUCCEEDED"],   "afterAction":[     "compute",     "execute",     "test"],   "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536844585053000000, stage=AFTER_ACTION, actionName='test', event={   "id":"8ab659666f220Tyw3o735czvqe26da5nglw0r3",   "source":"metric_trigger",   "eventTime":2440252617060896,   "eventType":"METRIC",   "properties":{     "node":{"127.0.0.1:45851_solr":13602},     "__start__":1,     "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n1:INDEX.sizeInBytes",     "collection":"testMetricTrigger",     "shard":"shard1",     "preferredOperation":"movereplica",     "_enqueue_time_":2440257635924181,     "requestedOps":[{         "action":"MOVEREPLICA",         "hints":{           "COLL_SHARD":[{               "first":"testMetricTrigger",               "second":"shard1"}],           "SRC_NODE":["127.0.0.1:45851_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica, method=GET, params.action=MOVEREPLICA, params.async=metric_trigger/8ab659666f220Tyw3o735czvqe26da5nglw0r3/0, params.waitForFinalState=true, params.collection=testMetricTrigger, params.targetNode=127.0.0.1:46761_solr, params.inPlaceMove=true, params.replica=core_node3}], properties.BEFORE_ACTION=[compute, execute, test], properties.responses=[{responseHeader={status=0,QTime=1},success=MOVEREPLICA action completed successfully, moved replica=testMetricTrigger_shard1_replica_n1 at node=127.0.0.1:45851_solr to replica=testMetricTrigger_shard1_replica_n9 at node=127.0.0.1:46761_solr,status={state=completed,msg=found [metric_trigger/8ab659666f220Tyw3o735czvqe26da5nglw0r3/0] in completed tasks}}], source=metric_trigger, properties.AFTER_ACTION=[compute, execute, test]}, config={   "beforeAction":[],   "trigger":"metric_trigger",   "stage":[     "FAILED",     "SUCCEEDED"],   "afterAction":[     "compute",     "execute",     "test"],   "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536844585054000000, stage=SUCCEEDED, actionName='null', event={   "id":"8ab659666f220Tyw3o735czvqe26da5nglw0r3",   "source":"metric_trigger",   "eventTime":2440252617060896,   "eventType":"METRIC",   "properties":{     "node":{"127.0.0.1:45851_solr":13602},     "__start__":1,     "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n1:INDEX.sizeInBytes",     "collection":"testMetricTrigger",     "shard":"shard1",     "preferredOperation":"movereplica",     "_enqueue_time_":2440257635924181,     "requestedOps":[{         "action":"MOVEREPLICA",         "hints":{           "COLL_SHARD":[{               "first":"testMetricTrigger",               "second":"shard1"}],           "SRC_NODE":["127.0.0.1:45851_solr"]}}]}}, context={}, config={   "beforeAction":[],   "trigger":"metric_trigger",   "stage":[     "FAILED",     "SUCCEEDED"],   "afterAction":[     "compute",     "execute",     "test"],   "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}]} expected:<4> but was:<8>

Stack Trace:
java.lang.AssertionError: {srt=[CapturedEvent{timestamp=1536843674261000000, stage=AFTER_ACTION, actionName='compute', event={
  "id":"8aa92b20490dbTyw3o735czvqe26da5nglw0oq",
  "source":"metric_trigger",
  "eventTime":2439346842276059,
  "eventType":"METRIC",
  "properties":{
    "node":{"127.0.0.1:32995_solr":13779},
    "__start__":1,
    "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n3:INDEX.sizeInBytes",
    "collection":"testMetricTrigger",
    "shard":"shard1",
    "preferredOperation":"addreplica",
    "_enqueue_time_":2439351868656498,
    "requestedOps":[{
        "action":"ADDREPLICA",
        "hints":{
          "COLL_SHARD":[{
              "first":"testMetricTrigger",
              "second":"shard1"}],
          "SRC_NODE":["127.0.0.1:32995_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica, method=GET, params.action=ADDREPLICA, params.collection=testMetricTrigger, params.shard=shard1, params.node=127.0.0.1:34435_solr, params.type=NRT}], properties.BEFORE_ACTION=[compute], source=metric_trigger, properties.AFTER_ACTION=[compute]}, config={
  "beforeAction":[],
  "trigger":"metric_trigger",
  "stage":[
    "FAILED",
    "SUCCEEDED"],
  "afterAction":[
    "compute",
    "execute",
    "test"],
  "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536843679276000000, stage=AFTER_ACTION, actionName='execute', event={
  "id":"8aa92b20490dbTyw3o735czvqe26da5nglw0oq",
  "source":"metric_trigger",
  "eventTime":2439346842276059,
  "eventType":"METRIC",
  "properties":{
    "node":{"127.0.0.1:32995_solr":13779},
    "__start__":1,
    "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n3:INDEX.sizeInBytes",
    "collection":"testMetricTrigger",
    "shard":"shard1",
    "preferredOperation":"addreplica",
    "_enqueue_time_":2439351868656498,
    "requestedOps":[{
        "action":"ADDREPLICA",
        "hints":{
          "COLL_SHARD":[{
              "first":"testMetricTrigger",
              "second":"shard1"}],
          "SRC_NODE":["127.0.0.1:32995_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica, method=GET, params.action=ADDREPLICA, params.async=metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/0, params.waitForFinalState=true, params.collection=testMetricTrigger, params.shard=shard1, params.node=127.0.0.1:34435_solr, params.type=NRT}], properties.BEFORE_ACTION=[compute, execute], properties.responses=[{responseHeader={status=0,QTime=1},success={127.0.0.1:34435_solr={responseHeader={status=0,QTime=0}}},metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647={responseHeader={status=0,QTime=0},STATUS=completed,Response=TaskId: metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647 webapp=null path=/admin/cores params={async=metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647&qt=/admin/cores&coreNodeName=core_node12&collection.configName=conf&name=testMetricTrigger_shard1_replica_n11&action=CREATE&collection=testMetricTrigger&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=0},status={state=completed,msg=found [metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/0] in completed tasks}}], source=metric_trigger, properties.AFTER_ACTION=[compute, execute]}, config={
  "beforeAction":[],
  "trigger":"metric_trigger",
  "stage":[
    "FAILED",
    "SUCCEEDED"],
  "afterAction":[
    "compute",
    "execute",
    "test"],
  "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536843679276000000, stage=AFTER_ACTION, actionName='test', event={
  "id":"8aa92b20490dbTyw3o735czvqe26da5nglw0oq",
  "source":"metric_trigger",
  "eventTime":2439346842276059,
  "eventType":"METRIC",
  "properties":{
    "node":{"127.0.0.1:32995_solr":13779},
    "__start__":1,
    "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n3:INDEX.sizeInBytes",
    "collection":"testMetricTrigger",
    "shard":"shard1",
    "preferredOperation":"addreplica",
    "_enqueue_time_":2439351868656498,
    "requestedOps":[{
        "action":"ADDREPLICA",
        "hints":{
          "COLL_SHARD":[{
              "first":"testMetricTrigger",
              "second":"shard1"}],
          "SRC_NODE":["127.0.0.1:32995_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica, method=GET, params.action=ADDREPLICA, params.async=metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/0, params.waitForFinalState=true, params.collection=testMetricTrigger, params.shard=shard1, params.node=127.0.0.1:34435_solr, params.type=NRT}], properties.BEFORE_ACTION=[compute, execute, test], properties.responses=[{responseHeader={status=0,QTime=1},success={127.0.0.1:34435_solr={responseHeader={status=0,QTime=0}}},metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647={responseHeader={status=0,QTime=0},STATUS=completed,Response=TaskId: metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647 webapp=null path=/admin/cores params={async=metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/02439352000123647&qt=/admin/cores&coreNodeName=core_node12&collection.configName=conf&name=testMetricTrigger_shard1_replica_n11&action=CREATE&collection=testMetricTrigger&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=0},status={state=completed,msg=found [metric_trigger/8aa92b20490dbTyw3o735czvqe26da5nglw0oq/0] in completed tasks}}], source=metric_trigger, properties.AFTER_ACTION=[compute, execute, test]}, config={
  "beforeAction":[],
  "trigger":"metric_trigger",
  "stage":[
    "FAILED",
    "SUCCEEDED"],
  "afterAction":[
    "compute",
    "execute",
    "test"],
  "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536843679277000000, stage=SUCCEEDED, actionName='null', event={
  "id":"8aa92b20490dbTyw3o735czvqe26da5nglw0oq",
  "source":"metric_trigger",
  "eventTime":2439346842276059,
  "eventType":"METRIC",
  "properties":{
    "node":{"127.0.0.1:32995_solr":13779},
    "__start__":1,
    "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n3:INDEX.sizeInBytes",
    "collection":"testMetricTrigger",
    "shard":"shard1",
    "preferredOperation":"addreplica",
    "_enqueue_time_":2439351868656498,
    "requestedOps":[{
        "action":"ADDREPLICA",
        "hints":{
          "COLL_SHARD":[{
              "first":"testMetricTrigger",
              "second":"shard1"}],
          "SRC_NODE":["127.0.0.1:32995_solr"]}}]}}, context={}, config={
  "beforeAction":[],
  "trigger":"metric_trigger",
  "stage":[
    "FAILED",
    "SUCCEEDED"],
  "afterAction":[
    "compute",
    "execute",
    "test"],
  "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536844580042000000, stage=AFTER_ACTION, actionName='compute', event={
  "id":"8ab659666f220Tyw3o735czvqe26da5nglw0r3",
  "source":"metric_trigger",
  "eventTime":2440252617060896,
  "eventType":"METRIC",
  "properties":{
    "node":{"127.0.0.1:45851_solr":13602},
    "__start__":1,
    "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n1:INDEX.sizeInBytes",
    "collection":"testMetricTrigger",
    "shard":"shard1",
    "preferredOperation":"movereplica",
    "_enqueue_time_":2440257635924181,
    "requestedOps":[{
        "action":"MOVEREPLICA",
        "hints":{
          "COLL_SHARD":[{
              "first":"testMetricTrigger",
              "second":"shard1"}],
          "SRC_NODE":["127.0.0.1:45851_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica, method=GET, params.action=MOVEREPLICA, params.collection=testMetricTrigger, params.targetNode=127.0.0.1:46761_solr, params.inPlaceMove=true, params.replica=core_node3}], properties.BEFORE_ACTION=[compute], source=metric_trigger, properties.AFTER_ACTION=[compute]}, config={
  "beforeAction":[],
  "trigger":"metric_trigger",
  "stage":[
    "FAILED",
    "SUCCEEDED"],
  "afterAction":[
    "compute",
    "execute",
    "test"],
  "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536844585053000000, stage=AFTER_ACTION, actionName='execute', event={
  "id":"8ab659666f220Tyw3o735czvqe26da5nglw0r3",
  "source":"metric_trigger",
  "eventTime":2440252617060896,
  "eventType":"METRIC",
  "properties":{
    "node":{"127.0.0.1:45851_solr":13602},
    "__start__":1,
    "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n1:INDEX.sizeInBytes",
    "collection":"testMetricTrigger",
    "shard":"shard1",
    "preferredOperation":"movereplica",
    "_enqueue_time_":2440257635924181,
    "requestedOps":[{
        "action":"MOVEREPLICA",
        "hints":{
          "COLL_SHARD":[{
              "first":"testMetricTrigger",
              "second":"shard1"}],
          "SRC_NODE":["127.0.0.1:45851_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica, method=GET, params.action=MOVEREPLICA, params.async=metric_trigger/8ab659666f220Tyw3o735czvqe26da5nglw0r3/0, params.waitForFinalState=true, params.collection=testMetricTrigger, params.targetNode=127.0.0.1:46761_solr, params.inPlaceMove=true, params.replica=core_node3}], properties.BEFORE_ACTION=[compute, execute], properties.responses=[{responseHeader={status=0,QTime=1},success=MOVEREPLICA action completed successfully, moved replica=testMetricTrigger_shard1_replica_n1 at node=127.0.0.1:45851_solr to replica=testMetricTrigger_shard1_replica_n9 at node=127.0.0.1:46761_solr,status={state=completed,msg=found [metric_trigger/8ab659666f220Tyw3o735czvqe26da5nglw0r3/0] in completed tasks}}], source=metric_trigger, properties.AFTER_ACTION=[compute, execute]}, config={
  "beforeAction":[],
  "trigger":"metric_trigger",
  "stage":[
    "FAILED",
    "SUCCEEDED"],
  "afterAction":[
    "compute",
    "execute",
    "test"],
  "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536844585053000000, stage=AFTER_ACTION, actionName='test', event={
  "id":"8ab659666f220Tyw3o735czvqe26da5nglw0r3",
  "source":"metric_trigger",
  "eventTime":2440252617060896,
  "eventType":"METRIC",
  "properties":{
    "node":{"127.0.0.1:45851_solr":13602},
    "__start__":1,
    "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n1:INDEX.sizeInBytes",
    "collection":"testMetricTrigger",
    "shard":"shard1",
    "preferredOperation":"movereplica",
    "_enqueue_time_":2440257635924181,
    "requestedOps":[{
        "action":"MOVEREPLICA",
        "hints":{
          "COLL_SHARD":[{
              "first":"testMetricTrigger",
              "second":"shard1"}],
          "SRC_NODE":["127.0.0.1:45851_solr"]}}]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica, method=GET, params.action=MOVEREPLICA, params.async=metric_trigger/8ab659666f220Tyw3o735czvqe26da5nglw0r3/0, params.waitForFinalState=true, params.collection=testMetricTrigger, params.targetNode=127.0.0.1:46761_solr, params.inPlaceMove=true, params.replica=core_node3}], properties.BEFORE_ACTION=[compute, execute, test], properties.responses=[{responseHeader={status=0,QTime=1},success=MOVEREPLICA action completed successfully, moved replica=testMetricTrigger_shard1_replica_n1 at node=127.0.0.1:45851_solr to replica=testMetricTrigger_shard1_replica_n9 at node=127.0.0.1:46761_solr,status={state=completed,msg=found [metric_trigger/8ab659666f220Tyw3o735czvqe26da5nglw0r3/0] in completed tasks}}], source=metric_trigger, properties.AFTER_ACTION=[compute, execute, test]}, config={
  "beforeAction":[],
  "trigger":"metric_trigger",
  "stage":[
    "FAILED",
    "SUCCEEDED"],
  "afterAction":[
    "compute",
    "execute",
    "test"],
  "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}, CapturedEvent{timestamp=1536844585054000000, stage=SUCCEEDED, actionName='null', event={
  "id":"8ab659666f220Tyw3o735czvqe26da5nglw0r3",
  "source":"metric_trigger",
  "eventTime":2440252617060896,
  "eventType":"METRIC",
  "properties":{
    "node":{"127.0.0.1:45851_solr":13602},
    "__start__":1,
    "metric":"metrics:solr.core.testMetricTrigger.shard1.replica_n1:INDEX.sizeInBytes",
    "collection":"testMetricTrigger",
    "shard":"shard1",
    "preferredOperation":"movereplica",
    "_enqueue_time_":2440257635924181,
    "requestedOps":[{
        "action":"MOVEREPLICA",
        "hints":{
          "COLL_SHARD":[{
              "first":"testMetricTrigger",
              "second":"shard1"}],
          "SRC_NODE":["127.0.0.1:45851_solr"]}}]}}, context={}, config={
  "beforeAction":[],
  "trigger":"metric_trigger",
  "stage":[
    "FAILED",
    "SUCCEEDED"],
  "afterAction":[
    "compute",
    "execute",
    "test"],
  "class":"org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest$TestTriggerListener"}, message='null'}]} expected:<4> but was:<8>
        at __randomizedtesting.SeedInfo.seed([FC5959527B2FEFEA:46556EDD24C739A5]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger(MetricTriggerIntegrationTest.java:157)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger

Error Message:
Timed out waiting for collection:testMetricTrigger to become active null Live Nodes: [127.0.0.1:34535_solr, 127.0.0.1:40005_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for collection:testMetricTrigger to become active
null
Live Nodes: [127.0.0.1:34535_solr, 127.0.0.1:40005_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([FC5959527B2FEFEA:46556EDD24C739A5]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger(MetricTriggerIntegrationTest.java:95)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger

Error Message:
Timed out waiting for collection:testMetricTrigger to become active null Live Nodes: [127.0.0.1:38491_solr, 127.0.0.1:45759_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Timed out waiting for collection:testMetricTrigger to become active
null
Live Nodes: [127.0.0.1:38491_solr, 127.0.0.1:45759_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([FC5959527B2FEFEA:46556EDD24C739A5]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.autoscaling.MetricTriggerIntegrationTest.testMetricTrigger(MetricTriggerIntegrationTest.java:95)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:33219/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33219/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([FC5959527B2FEFEA:725539220E376BC0]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
     
--
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_172) - Build # 22852 - Still Unstable!

Policeman Jenkins Server-2
In reply to this post by Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/22852/
Java: 64bit/jdk1.8.0_172 -XX:+UseCompressedOops -XX:+UseSerialGC

4 tests failed.
FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:36395/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:36395/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([4DC797584DF48098:C3CBF72838EC04B2]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:45279/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:45279/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([4DC797584DF48098:C3CBF72838EC04B2]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:35455/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:35455/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([4DC797584DF48098:C3CBF72838EC04B2]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader

Error Message:
Error from server at http://127.0.0.1:41869/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:41869/solr: Could not find collection : pull_replica_error_handling_test_cant_connect_to_leader
        at __randomizedtesting.SeedInfo.seed([4DC797584DF48098:C3CBF72838EC04B2]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplicaErrorHandling.tearDown(TestPullReplicaErrorHandling.java:128)
        at sun.reflect.GeneratedMethodAccessor30.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12989 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestPullReplicaErrorHandling
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_4DC797584DF48098-001/init-core-data-001
   [junit4]   2> 211328 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_4DC797584DF48098-001/tempDir-001
   [junit4]   2> 211328 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 211328 INFO  (Thread-457) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 211328 INFO  (Thread-457) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 211329 ERROR (Thread-457) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 211428 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:44373
   [junit4]   2> 211430 INFO  (zkConnectionManagerCallback-694-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211433 INFO  (jetty-launcher-691-thread-4) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 211433 INFO  (jetty-launcher-691-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 211434 INFO  (jetty-launcher-691-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 211434 INFO  (jetty-launcher-691-thread-3) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 211436 INFO  (jetty-launcher-691-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 211436 INFO  (jetty-launcher-691-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 211436 INFO  (jetty-launcher-691-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 211436 INFO  (jetty-launcher-691-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@79e0e72c{/solr,null,AVAILABLE}
   [junit4]   2> 211436 INFO  (jetty-launcher-691-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 211436 INFO  (jetty-launcher-691-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 211436 INFO  (jetty-launcher-691-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 211436 INFO  (jetty-launcher-691-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 211436 INFO  (jetty-launcher-691-thread-4) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 211436 INFO  (jetty-launcher-691-thread-3) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 211436 INFO  (jetty-launcher-691-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 211436 INFO  (jetty-launcher-691-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 211436 INFO  (jetty-launcher-691-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3f4b6219{/solr,null,AVAILABLE}
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@67806987{/solr,null,AVAILABLE}
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2d119162{/solr,null,AVAILABLE}
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@55e7672c{HTTP/1.1,[http/1.1]}{127.0.0.1:41811}
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-1) [    ] o.e.j.s.Server Started @211478ms
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41811}
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@230f84df{HTTP/1.1,[http/1.1]}{127.0.0.1:40667}
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1c687b39{HTTP/1.1,[http/1.1]}{127.0.0.1:43405}
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-3) [    ] o.e.j.s.Server Started @211478ms
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-4) [    ] o.e.j.s.Server Started @211478ms
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1d8dac70{HTTP/1.1,[http/1.1]}{127.0.0.1:46055}
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40667}
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43405}
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-2) [    ] o.e.j.s.Server Started @211478ms
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46055}
   [junit4]   2> 211437 ERROR (jetty-launcher-691-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 211437 ERROR (jetty-launcher-691-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-3) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 211437 ERROR (jetty-launcher-691-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 211437 ERROR (jetty-launcher-691-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-4) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T16:00:44.729Z
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T16:00:44.729Z
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T16:00:44.729Z
   [junit4]   2> 211437 INFO  (jetty-launcher-691-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T16:00:44.729Z
   [junit4]   2> 211439 INFO  (zkConnectionManagerCallback-698-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211439 INFO  (zkConnectionManagerCallback-696-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211439 INFO  (jetty-launcher-691-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 211439 INFO  (jetty-launcher-691-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 211439 INFO  (zkConnectionManagerCallback-701-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211440 INFO  (zkConnectionManagerCallback-702-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211440 INFO  (jetty-launcher-691-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 211440 INFO  (jetty-launcher-691-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 211456 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1009209e6090004, likely client has closed socket
   [junit4]   2> 211528 INFO  (jetty-launcher-691-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44373/solr
   [junit4]   2> 211528 INFO  (jetty-launcher-691-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44373/solr
   [junit4]   2> 211535 INFO  (zkConnectionManagerCallback-709-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211539 INFO  (zkConnectionManagerCallback-710-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211547 INFO  (zkConnectionManagerCallback-714-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211548 INFO  (zkConnectionManagerCallback-713-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211565 INFO  (jetty-launcher-691-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44373/solr
   [junit4]   2> 211570 INFO  (zkConnectionManagerCallback-722-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211581 INFO  (zkConnectionManagerCallback-724-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211588 INFO  (jetty-launcher-691-thread-1) [n:127.0.0.1:41811_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41811_solr
   [junit4]   2> 211588 INFO  (jetty-launcher-691-thread-1) [n:127.0.0.1:41811_solr    ] o.a.s.c.Overseer Overseer (id=72218165249638410-127.0.0.1:41811_solr-n_0000000000) starting
   [junit4]   2> 211595 INFO  (zkConnectionManagerCallback-731-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211596 INFO  (jetty-launcher-691-thread-1) [n:127.0.0.1:41811_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44373/solr ready
   [junit4]   2> 211599 INFO  (jetty-launcher-691-thread-3) [n:127.0.0.1:40667_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40667_solr
   [junit4]   2> 211599 INFO  (OverseerStateUpdate-72218165249638410-127.0.0.1:41811_solr-n_0000000000) [n:127.0.0.1:41811_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:41811_solr
   [junit4]   2> 211601 INFO  (jetty-launcher-691-thread-2) [n:127.0.0.1:46055_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 211602 INFO  (jetty-launcher-691-thread-2) [n:127.0.0.1:46055_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 211602 INFO  (jetty-launcher-691-thread-2) [n:127.0.0.1:46055_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46055_solr
   [junit4]   2> 211608 INFO  (zkCallback-730-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 211608 INFO  (zkCallback-712-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 211608 INFO  (OverseerStateUpdate-72218165249638410-127.0.0.1:41811_solr-n_0000000000) [n:127.0.0.1:41811_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 211609 INFO  (jetty-launcher-691-thread-1) [n:127.0.0.1:41811_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 211609 INFO  (jetty-launcher-691-thread-1) [n:127.0.0.1:41811_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41811_solr
   [junit4]   2> 211612 INFO  (zkCallback-723-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 211616 INFO  (zkCallback-711-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 211616 INFO  (zkCallback-712-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 211616 INFO  (zkCallback-730-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 211618 INFO  (jetty-launcher-691-thread-1) [n:127.0.0.1:41811_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 211616 INFO  (zkCallback-711-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 211623 INFO  (zkConnectionManagerCallback-738-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211624 INFO  (jetty-launcher-691-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44373/solr
   [junit4]   2> 211624 INFO  (jetty-launcher-691-thread-3) [n:127.0.0.1:40667_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 211624 INFO  (jetty-launcher-691-thread-3) [n:127.0.0.1:40667_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44373/solr ready
   [junit4]   2> 211625 INFO  (zkConnectionManagerCallback-743-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211625 INFO  (jetty-launcher-691-thread-3) [n:127.0.0.1:40667_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 211627 INFO  (zkConnectionManagerCallback-746-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211627 INFO  (jetty-launcher-691-thread-2) [n:127.0.0.1:46055_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 211629 INFO  (jetty-launcher-691-thread-2) [n:127.0.0.1:46055_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44373/solr ready
   [junit4]   2> 211629 INFO  (jetty-launcher-691-thread-2) [n:127.0.0.1:46055_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 211629 INFO  (zkConnectionManagerCallback-749-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211634 INFO  (jetty-launcher-691-thread-4) [n:127.0.0.1:43405_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 211638 INFO  (jetty-launcher-691-thread-4) [n:127.0.0.1:43405_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 211638 INFO  (jetty-launcher-691-thread-4) [n:127.0.0.1:43405_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43405_solr
   [junit4]   2> 211639 INFO  (zkCallback-730-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 211639 INFO  (zkCallback-712-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 211639 INFO  (zkCallback-723-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 211639 INFO  (zkCallback-711-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 211661 INFO  (zkCallback-748-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 211662 INFO  (zkCallback-742-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 211662 INFO  (zkCallback-737-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 211667 INFO  (jetty-launcher-691-thread-2) [n:127.0.0.1:46055_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46055.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 211668 INFO  (jetty-launcher-691-thread-3) [n:127.0.0.1:40667_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40667.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 211668 INFO  (jetty-launcher-691-thread-1) [n:127.0.0.1:41811_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41811.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 211673 INFO  (zkConnectionManagerCallback-757-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211674 INFO  (jetty-launcher-691-thread-4) [n:127.0.0.1:43405_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 211674 INFO  (jetty-launcher-691-thread-4) [n:127.0.0.1:43405_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44373/solr ready
   [junit4]   2> 211674 INFO  (jetty-launcher-691-thread-2) [n:127.0.0.1:46055_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46055.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 211675 INFO  (jetty-launcher-691-thread-2) [n:127.0.0.1:46055_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46055.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 211675 INFO  (jetty-launcher-691-thread-4) [n:127.0.0.1:43405_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 211675 INFO  (jetty-launcher-691-thread-3) [n:127.0.0.1:40667_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40667.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 211675 INFO  (jetty-launcher-691-thread-2) [n:127.0.0.1:46055_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_4DC797584DF48098-001/tempDir-001/node2/.
   [junit4]   2> 211675 INFO  (jetty-launcher-691-thread-3) [n:127.0.0.1:40667_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40667.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 211676 INFO  (jetty-launcher-691-thread-3) [n:127.0.0.1:40667_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_4DC797584DF48098-001/tempDir-001/node3/.
   [junit4]   2> 211676 INFO  (jetty-launcher-691-thread-1) [n:127.0.0.1:41811_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41811.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 211676 INFO  (jetty-launcher-691-thread-1) [n:127.0.0.1:41811_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41811.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 211677 INFO  (jetty-launcher-691-thread-1) [n:127.0.0.1:41811_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_4DC797584DF48098-001/tempDir-001/node1/.
   [junit4]   2> 211689 INFO  (jetty-launcher-691-thread-4) [n:127.0.0.1:43405_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43405.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 211693 INFO  (jetty-launcher-691-thread-4) [n:127.0.0.1:43405_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43405.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 211693 INFO  (jetty-launcher-691-thread-4) [n:127.0.0.1:43405_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43405.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 211694 INFO  (jetty-launcher-691-thread-4) [n:127.0.0.1:43405_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_4DC797584DF48098-001/tempDir-001/node4/.
   [junit4]   2> 211759 INFO  (zkConnectionManagerCallback-760-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211763 INFO  (zkConnectionManagerCallback-765-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211764 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 211765 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44373/solr ready
   [junit4]   2> 211767 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@55e7672c{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 211767 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=543554466
   [junit4]   2> 211767 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 211768 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2f41b3fd: rootName = solr_41811, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@52fb854a
   [junit4]   2> 211772 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 211772 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5c4ced64: rootName = solr_41811, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2ad4affc
   [junit4]   2> 211774 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 211774 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@11398828: rootName = solr_41811, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2b994eac
   [junit4]   2> 211775 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:41811_solr
   [junit4]   2> 211775 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 211775 INFO  (zkCallback-730-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 211775 INFO  (zkCallback-748-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 211775 INFO  (zkCallback-711-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 211775 INFO  (zkCallback-742-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 211775 INFO  (zkCallback-723-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 211775 INFO  (zkCallback-712-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 211776 INFO  (zkCallback-737-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 211776 INFO  (zkCallback-756-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 211776 INFO  (zkCallback-764-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 211776 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.Overseer Overseer (id=72218165249638410-127.0.0.1:41811_solr-n_0000000000) closing
   [junit4]   2> 211776 INFO  (OverseerStateUpdate-72218165249638410-127.0.0.1:41811_solr-n_0000000000) [n:127.0.0.1:41811_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:41811_solr
   [junit4]   2> 211776 WARN  (OverseerAutoScalingTriggerThread-72218165249638410-127.0.0.1:41811_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 211779 INFO  (zkCallback-712-thread-1) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40667_solr
   [junit4]   2> 211779 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@79e0e72c{/solr,null,UNAVAILABLE}
   [junit4]   2> 211779 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 211780 INFO  (zkCallback-712-thread-1) [n:127.0.0.1:40667_solr    ] o.a.s.c.Overseer Overseer (id=72218165249638407-127.0.0.1:40667_solr-n_0000000001) starting
   [junit4]   2> 211780 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 211781 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 211781 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 211781 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 211781 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7a0cdba8{/solr,null,AVAILABLE}
   [junit4]   2> 211781 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7afce27f{HTTP/1.1,[http/1.1]}{127.0.0.1:35255}
   [junit4]   2> 211781 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.Server Started @211822ms
   [junit4]   2> 211781 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41869}
   [junit4]   2> 211781 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 211782 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 211782 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 211782 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 211782 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 211782 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T16:00:45.074Z
   [junit4]   2> 211782 INFO  (OverseerStateUpdate-72218165249638407-127.0.0.1:40667_solr-n_0000000001) [n:127.0.0.1:40667_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:40667_solr
   [junit4]   2> 211783 INFO  (zkConnectionManagerCallback-767-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211784 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 211830 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44373/solr
   [junit4]   2> 211832 INFO  (zkConnectionManagerCallback-771-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211834 INFO  (zkConnectionManagerCallback-773-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211837 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [n:127.0.0.1:41869_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 211838 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [n:127.0.0.1:41869_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 211838 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [n:127.0.0.1:41869_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41869_solr
   [junit4]   2> 211839 INFO  (zkCallback-748-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 211839 INFO  (zkCallback-756-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 211839 INFO  (zkCallback-712-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 211839 INFO  (zkCallback-711-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 211839 INFO  (zkCallback-737-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 211839 INFO  (zkCallback-742-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 211839 INFO  (zkCallback-764-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 211839 INFO  (zkCallback-772-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 211851 INFO  (zkConnectionManagerCallback-780-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211853 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [n:127.0.0.1:41869_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 211853 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [n:127.0.0.1:41869_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44373/solr ready
   [junit4]   2> 211854 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [n:127.0.0.1:41869_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 211865 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [n:127.0.0.1:41869_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41869.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 211870 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [n:127.0.0.1:41869_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41869.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 211870 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [n:127.0.0.1:41869_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41869.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 211871 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [n:127.0.0.1:41869_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_4DC797584DF48098-001/tempDir-001/node1/.
   [junit4]   2> 211924 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:35255/solr. Proxy: http://127.0.0.1:41869/solr
   [junit4]   2> 211925 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@230f84df{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 211925 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=79869363
   [junit4]   2> 211925 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 211926 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@166e6f3e: rootName = solr_40667, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@33cc4f85
   [junit4]   2> 211931 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 211931 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@77f4cdf9: rootName = solr_40667, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2ad4affc
   [junit4]   2> 211934 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 211934 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3ad33bb0: rootName = solr_40667, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2b994eac
   [junit4]   2> 211935 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:40667_solr
   [junit4]   2> 211935 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 211936 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 211936 INFO  (zkCallback-764-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 211936 INFO  (zkCallback-742-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 211936 INFO  (zkCallback-748-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 211936 INFO  (zkCallback-737-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 211936 INFO  (zkCallback-712-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 211936 INFO  (zkCallback-711-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 211936 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.Overseer Overseer (id=72218165249638407-127.0.0.1:40667_solr-n_0000000001) closing
   [junit4]   2> 211936 INFO  (OverseerStateUpdate-72218165249638407-127.0.0.1:40667_solr-n_0000000001) [n:127.0.0.1:40667_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40667_solr
   [junit4]   2> 211936 WARN  (OverseerAutoScalingTriggerThread-72218165249638407-127.0.0.1:40667_solr-n_0000000001) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 211944 INFO  (zkCallback-772-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 211944 INFO  (zkCallback-756-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 211946 INFO  (zkCallback-779-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 211948 INFO  (zkCallback-711-thread-2) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46055_solr
   [junit4]   2> 211948 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@67806987{/solr,null,UNAVAILABLE}
   [junit4]   2> 211948 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 211949 INFO  (zkCallback-711-thread-2) [n:127.0.0.1:46055_solr    ] o.a.s.c.Overseer Overseer (id=72218165249638408-127.0.0.1:46055_solr-n_0000000002) starting
   [junit4]   2> 211949 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 211959 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 211959 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 211959 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 211960 INFO  (OverseerStateUpdate-72218165249638408-127.0.0.1:46055_solr-n_0000000002) [n:127.0.0.1:46055_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:46055_solr
   [junit4]   2> 211964 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@71ebe83a{/solr,null,AVAILABLE}
   [junit4]   2> 211964 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@209546ac{HTTP/1.1,[http/1.1]}{127.0.0.1:46757}
   [junit4]   2> 211964 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.Server Started @212005ms
   [junit4]   2> 211964 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42679}
   [junit4]   2> 211965 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 211965 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 211965 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 211965 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 211965 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 211965 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T16:00:45.257Z
   [junit4]   2> 211976 INFO  (zkConnectionManagerCallback-783-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 211977 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 212043 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44373/solr
   [junit4]   2> 212045 INFO  (zkConnectionManagerCallback-787-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 212047 INFO  (zkConnectionManagerCallback-789-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 212050 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 212052 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 212052 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42679_solr
   [junit4]   2> 212053 INFO  (zkCallback-764-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 212053 INFO  (zkCallback-748-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 212053 INFO  (zkCallback-711-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 212053 INFO  (zkCallback-788-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 212053 INFO  (zkCallback-772-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 212053 INFO  (zkCallback-742-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 212055 INFO  (zkCallback-779-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 212055 INFO  (zkCallback-756-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 212065 INFO  (zkConnectionManagerCallback-796-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 212066 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 212066 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44373/solr ready
   [junit4]   2> 212067 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 212082 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42679.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 212089 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42679.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 212090 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42679.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 212091 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_4DC797584DF48098-001/tempDir-001/node3/.
   [junit4]   2> 212127 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:46757/solr. Proxy: http://127.0.0.1:42679/solr
   [junit4]   2> 212128 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@1d8dac70{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 212128 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1933925672
   [junit4]   2> 212128 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 212129 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@21efe0f6: rootName = solr_46055, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@4e5b4bd5
   [junit4]   2> 212133 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 212133 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3340179f: rootName = solr_46055, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2ad4affc
   [junit4]   2> 212135 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 212135 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1364149e: rootName = solr_46055, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2b994eac
   [junit4]   2> 212136 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:46055_solr
   [junit4]   2> 212137 INFO  (zkCallback-748-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 212137 INFO  (zkCallback-764-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 212137 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 212137 INFO  (zkCallback-711-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 212143 INFO  (zkCallback-788-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 212143 INFO  (zkCallback-779-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 212143 INFO  (zkCallback-772-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 212143 INFO  (zkCallback-742-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 212143 INFO  (zkCallback-756-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 212144 INFO  (zkCallback-795-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 212148 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.Overseer Overseer (id=72218165249638408-127.0.0.1:46055_solr-n_0000000002) closing
   [junit4]   2> 212148 INFO  (OverseerStateUpdate-72218165249638408-127.0.0.1:46055_solr-n_0000000002) [n:127.0.0.1:46055_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:46055_solr
   [junit4]   2> 212148 WARN  (OverseerAutoScalingTriggerThread-72218165249638408-127.0.0.1:46055_solr-n_0000000002) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 212157 INFO  (zkCallback-748-thread-1) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43405_solr
   [junit4]   2> 212158 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2d119162{/solr,null,UNAVAILABLE}
   [junit4]   2> 212158 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 212159 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 212160 INFO  (zkCallback-748-thread-1) [n:127.0.0.1:43405_solr    ] o.a.s.c.Overseer Overseer (id=72218165249638415-127.0.0.1:43405_solr-n_0000000003) starting
   [junit4]   2> 212165 INFO  (OverseerStateUpdate-72218165249638415-127.0.0.1:43405_solr-n_0000000003) [n:127.0.0.1:43405_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:43405_solr
   [junit4]   2> 212167 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 212167 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 212167 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 212168 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@afb0523{/solr,null,AVAILABLE}
   [junit4]   2> 212168 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3b8a63a6{HTTP/1.1,[http/1.1]}{127.0.0.1:42725}
   [junit4]   2> 212168 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.Server Started @212209ms
   [junit4]   2> 212168 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42531}
   [junit4]   2> 212169 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 212169 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 212169 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 212169 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 212169 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 212169 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T16:00:45.461Z
   [junit4]   2> 212174 INFO  (zkConnectionManagerCallback-799-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 212175 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 212273 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44373/solr
   [junit4]   2> 212276 INFO  (zkConnectionManagerCallback-803-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 212285 INFO  (zkConnectionManagerCallback-805-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 212289 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 212292 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 212292 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42531_solr
   [junit4]   2> 212293 INFO  (zkCallback-748-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 212293 INFO  (zkCallback-788-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 212293 INFO  (zkCallback-795-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 212293 INFO  (zkCallback-772-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 212293 INFO  (zkCallback-764-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 212293 INFO  (zkCallback-756-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 212295 INFO  (zkCallback-779-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 212296 INFO  (zkCallback-804-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 212315 INFO  (zkConnectionManagerCallback-812-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 212317 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 212317 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44373/solr ready
   [junit4]   2> 212317 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 212332 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42531.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 212340 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42531.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 212340 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42531.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ae1d2a
   [junit4]   2> 212341 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestPullReplicaErrorHandling_4DC797584DF48098-001/tempDir-001/node2/.
   [junit4]   2> 212376 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.TestPullReplicaErrorHandling Adding proxy for URL: http://127.0.0.1:42725/solr. Proxy: http://127.0.0.1:42531/solr
   [junit4]   2> 212376 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@1c687b39{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 212377 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1626546183
   [junit4]   2> 212377 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 212378 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@486d0beb: rootName = solr_43405, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@2627e4a1
   [junit4]   2> 212385 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 212385 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@516fe651: rootName = solr_43405, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2ad4affc
   [junit4]   2> 212389 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 212389 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1a9a97bf: rootName = solr_43405, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2b994eac
   [junit4]   2> 212390 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:43405_solr
   [junit4]   2> 212391 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 212391 INFO  (zkCallback-756-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 212391 INFO  (zkCallback-804-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 212391 INFO  (zkCallback-788-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 212391 INFO  (zkCallback-772-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 212391 INFO  (zkCallback-795-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 212391 INFO  (zkCallback-779-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 212391 INFO  (zkCallback-764-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 212391 INFO  (zkCallback-748-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 212392 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.Overseer Overseer (id=72218165249638415-127.0.0.1:43405_solr-n_0000000003) closing
   [junit4]   2> 212392 INFO  (OverseerStateUpdate-72218165249638415-127.0.0.1:43405_solr-n_0000000003) [n:127.0.0.1:43405_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:43405_solr
   [junit4]   2> 212392 WARN  (OverseerAutoScalingTriggerThread-72218165249638415-127.0.0.1:43405_solr-n_0000000003) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 212393 INFO  (zkCallback-811-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 212397 INFO  (zkCallback-772-thread-1) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41869_solr
   [junit4]   2> 212398 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3f4b6219{/solr,null,UNAVAILABLE}
   [junit4]   2> 212398 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 212398 INFO  (zkCallback-772-thread-1) [n:127.0.0.1:41869_solr    ] o.a.s.c.Overseer Overseer (id=72218165249638421-127.0.0.1:41869_solr-n_0000000004) starting
   [junit4]   2> 212399 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 212407 INFO  (OverseerStateUpdate-72218165249638421-127.0.0.1:41869_solr-n_0000000004) [n:127.0.0.1:41869_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:41869_solr
   [junit4]   2> 212419 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 212419 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 212419 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 212424 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@63d6ae80{/solr,null,AVAILABLE}
   [junit4]   2> 212425 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@47b4436f{HTTP/1.1,[http/1.1]}{127.0.0.1:44625}
   [junit4]   2> 212425 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.e.j.s.Server Started @212466ms
   [junit4]   2> 212425 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42073}
   [junit4]   2> 212425 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 212425 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 212425 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 212425 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 212425 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 212426 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-13T16:00:45.718Z
   [junit4]   2> 212435 INFO  (zkConnectionManagerCallback-815-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 212436 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 212572 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44373/solr
   [junit4]   2> 212576 INFO  (zkConnectionManagerCallback-819-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 212587 INFO  (zkConnectionManagerCallback-821-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 212590 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 212601 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 212601 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.ZkCont

[...truncated too long message...]

  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
   [junit4]   2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
   [junit4]   2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
   [junit4]   2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
   [junit4]   2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
   [junit4]   2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:674)
   [junit4]   2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
   [junit4]   2> at org.eclipse.jetty.server.Server.handle(Server.java:531)
   [junit4]   2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
   [junit4]   2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
   [junit4]   2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
   [junit4]   2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
   [junit4]   2> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
   [junit4]   2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
   [junit4]   2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
   [junit4]   2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue/qn-
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   [junit4]   2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:786)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:398)
   [junit4]   2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:398)
   [junit4]   2> at org.apache.solr.cloud.ZkDistributedQueue.offer(ZkDistributedQueue.java:321)
   [junit4]   2> at org.apache.solr.cloud.ZkController.unregister(ZkController.java:1605)
   [junit4]   2> at org.apache.solr.cloud.ZkController.unregister(ZkController.java:1573)
   [junit4]   2> at org.apache.solr.core.CoreContainer.unload(CoreContainer.java:1524)
   [junit4]   2> ... 40 more
   [junit4]   2>
   [junit4]   2> 1276234 ERROR (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1276235 INFO  (SUITE-TestPullReplicaErrorHandling-seed#[4DC797584DF48098]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36375 36375
   [junit4]   2> 1277096 INFO  (Thread-101) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36375 36375
   [junit4]   2> 1277097 WARN  (Thread-101) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 18 /solr/aliases.json
   [junit4]   2> 17 /solr/clusterprops.json
   [junit4]   2> 9 /solr/security.json
   [junit4]   2> 9 /solr/configs/conf
   [junit4]   2> 8 /solr/collections/pull_replica_error_handling_test_pull_replica_disconnects_from_zoo_keeper/terms/shard1
   [junit4]   2> 8 /solr/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/terms/shard1
   [junit4]   2> 7 /solr/collections/pull_replica_error_handling_test_cant_connect_to_leader/terms/shard1
   [junit4]   2> 7 /solr/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/terms/shard2
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 18 /solr/clusterstate.json
   [junit4]   2> 10 /solr/clusterprops.json
   [junit4]   2> 7 /solr/collections/pull_replica_error_handling_test_pull_replica_disconnects_from_zoo_keeper/state.json
   [junit4]   2> 5 /solr/autoscaling.json
   [junit4]   2> 4 /solr/collections/pull_replica_error_handling_test_cant_connect_to_pull_replica/state.json
   [junit4]   2> 2 /solr/overseer_elect/election/72218480227385369-127.0.0.1:36481_solr-n_0000000005
   [junit4]   2> 2 /solr/collections/pull_replica_error_handling_test_cant_connect_to_leader/state.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 18 /solr/collections
   [junit4]   2> 17 /solr/live_nodes
   [junit4]   2> 5 /solr/overseer/queue
   [junit4]   2> 5 /solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 5 /solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 5 /solr/overseer/collection-queue-work
   [junit4]   2> 5 /solr/overseer/queue-work
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplicaErrorHandling_4DC797584DF48098-002
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene80): {fieldName_s=PostingsFormat(name=MockRandom), id=FST50}, docValues:{}, maxPointsInLeafNode=659, maxMBSortInHeap=5.939204798488952, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@3bc5c137), locale=sr, timezone=Europe/Minsk
   [junit4]   2> NOTE: Linux 4.15.0-32-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=8,threads=1,free=415269024,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [TestPullReplicaErrorHandling, TestPullReplicaErrorHandling]
   [junit4] Completed [5/5 (5!)] on J0 in 631.97s, 3 tests, 1 error <<< FAILURES!
   [junit4]
   [junit4]
   [junit4] Tests with failures [seed: 4DC797584DF48098]:
   [junit4]   - org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader
   [junit4]   - org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader
   [junit4]   - org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader
   [junit4]   - org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader
   [junit4]   - org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToLeader
   [junit4]
   [junit4]
   [junit4] JVM J0:     0.67 ..  1277.90 =  1277.22s
   [junit4] JVM J1:     0.63 ..  1273.17 =  1272.54s
   [junit4] JVM J2:     0.70 ..   647.64 =   646.93s
   [junit4] Execution time total: 21 minutes 17 seconds
   [junit4] Tests summary: 5 suites, 15 tests, 5 errors

BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1567: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1092: There were test failures: 5 suites, 15 tests, 5 errors [seed: 4DC797584DF48098]

Total time: 21 minutes 19 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   5/5 failed: org.apache.solr.cloud.TestPullReplicaErrorHandling
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=dea3d69f95b5521aea5a04a82f9d8536284ed741, workspace=/var/lib/jenkins/workspace/Lucene-Solr-master-Linux
[WARNINGS] Computing warning deltas based on reference build #22851
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-11-ea+28) - Build # 22853 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/22853/
Java: 64bit/jdk-11-ea+28 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

18 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.ltr.TestLTROnSolrCloud

Error Message:
69 threads leaked from SUITE scope at org.apache.solr.ltr.TestLTROnSolrCloud:     1) Thread[id=192, name=qtp1274737139-192, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]         at java.base@11/sun.nio.ch.EPoll.wait(Native Method)         at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)         at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)         at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)         at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.base@11/java.lang.Thread.run(Thread.java:834)    2) Thread[id=185, name=qtp1826556823-185, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]         at java.base@11/sun.nio.ch.EPoll.wait(Native Method)         at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)         at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)         at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$156/0x00007f61d8d79858.run(Unknown Source)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.base@11/java.lang.Thread.run(Thread.java:834)    3) Thread[id=238, name=OverseerCollectionConfigSetProcessor-72219603194675206-127.0.0.1:34689_solr-n_0000000000, state=TIMED_WAITING, group=Overseer collection creation process.]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)         at app//org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:463)         at app//org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:243)         at app//org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:233)         at java.base@11/java.lang.Thread.run(Thread.java:834)    4) Thread[id=188, name=qtp1274737139-188, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]         at java.base@11/sun.nio.ch.EPoll.wait(Native Method)         at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)         at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)         at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)         at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.base@11/java.lang.Thread.run(Thread.java:834)    5) Thread[id=237, name=MetricsHistoryHandler-76-thread-1, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)    6) Thread[id=242, name=ScheduledTrigger-77-thread-1, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)    7) Thread[id=249, name=ScheduledTrigger-77-thread-4, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)    8) Thread[id=211, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/java.lang.Thread.sleep(Native Method)         at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.base@11/java.lang.Thread.run(Thread.java:834)    9) Thread[id=228, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/java.lang.Thread.sleep(Native Method)         at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.base@11/java.lang.Thread.run(Thread.java:834)   10) Thread[id=288, name=SolrRrdBackendFactory-81-thread-2, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)   11) Thread[id=248, name=ScheduledTrigger-77-thread-3, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)   12) Thread[id=173, name=Thread-26, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/java.lang.Object.wait(Native Method)         at java.base@11/java.lang.Thread.join(Thread.java:1305)         at java.base@11/java.lang.Thread.join(Thread.java:1379)         at app//org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:313)         at app//org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:313)         at app//org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:496)   13) Thread[id=222, name=jetty-launcher-46-thread-2-EventThread, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)         at java.base@11/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)         at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)   14) Thread[id=233, name=jetty-launcher-46-thread-2-EventThread, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)         at java.base@11/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)         at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)   15) Thread[id=207, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/java.lang.Thread.sleep(Native Method)         at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.base@11/java.lang.Thread.run(Thread.java:834)   16) Thread[id=224, name=zkConnectionManagerCallback-64-thread-1, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)         at java.base@11/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)   17) Thread[id=195, name=qtp1826556823-195, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]         at java.base@11/sun.nio.ch.EPoll.wait(Native Method)         at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)         at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)         at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)         at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.base@11/java.lang.Thread.run(Thread.java:834)   18) Thread[id=261, name=zkConnectionManagerCallback-90-thread-1, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)         at java.base@11/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)   19) Thread[id=208, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/java.lang.Thread.sleep(Native Method)         at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.base@11/java.lang.Thread.run(Thread.java:834)   20) Thread[id=183, name=qtp1274737139-183, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)         at java.base@11/java.lang.Thread.run(Thread.java:834)   21) Thread[id=231, name=jetty-launcher-46-thread-2-SendThread(127.0.0.1:44017), state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]         at java.base@11/sun.nio.ch.EPoll.wait(Native Method)         at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)         at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)         at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)         at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)         at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1144)   22) Thread[id=184, name=qtp1274737139-184, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)         at java.base@11/java.lang.Thread.run(Thread.java:834)   23) Thread[id=187, name=qtp1826556823-187, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]         at java.base@11/sun.nio.ch.EPoll.wait(Native Method)         at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)         at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)         at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)         at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.base@11/java.lang.Thread.run(Thread.java:834)   24) Thread[id=209, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/java.lang.Thread.sleep(Native Method)         at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.base@11/java.lang.Thread.run(Thread.java:834)   25) Thread[id=262, name=Scheduler-1090689667, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)   26) Thread[id=232, name=jetty-launcher-46-thread-1-EventThread, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)         at java.base@11/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)         at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)   27) Thread[id=186, name=qtp1274737139-186, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)         at java.base@11/java.lang.Thread.run(Thread.java:834)   28) Thread[id=210, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/java.lang.Thread.sleep(Native Method)         at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.base@11/java.lang.Thread.run(Thread.java:834)   29) Thread[id=245, name=Thread-43, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/java.lang.Object.wait(Native Method)         at java.base@11/java.lang.Object.wait(Object.java:328)         at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:1857)   30) Thread[id=289, name=qtp1826556823-289, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]         at java.base@11/sun.nio.ch.EPoll.wait(Native Method)         at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)         at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)         at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)         at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.base@11/java.lang.Thread.run(Thread.java:834)   31) Thread[id=230, name=jetty-launcher-46-thread-1-SendThread(127.0.0.1:44017), state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]         at java.base@11/sun.nio.ch.EPoll.wait(Native Method)         at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)         at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)         at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)         at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)         at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1144)   32) Thread[id=239, name=OverseerAutoScalingTriggerThread-72219603194675206-127.0.0.1:34689_solr-n_0000000000, state=WAITING, group=Overseer autoscaling triggers]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)         at app//org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189)         at java.base@11/java.lang.Thread.run(Thread.java:834)   33) Thread[id=194, name=qtp1274737139-194, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]         at java.base@11/sun.nio.ch.EPoll.wait(Native Method)         at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)         at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)         at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)         at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.base@11/java.lang.Thread.run(Thread.java:834)   34) Thread[id=200, name=Scheduler-326051045, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)   35) Thread[id=235, name=zkConnectionManagerCallback-79-thread-1, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)         at java.base@11/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)   36) Thread[id=273, name=qtp1274737139-273, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2211)         at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)         at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.base@11/java.lang.Thread.run(Thread.java:834)   37) Thread[id=247, name=ScheduledTrigger-77-thread-2, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)   38) Thread[id=219, name=jetty-launcher-46-thread-1-SendThread(127.0.0.1:44017), state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]         at java.base@11/sun.nio.ch.EPoll.wait(Native Method)         at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)         at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)         at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)         at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)         at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1144)   39) Thread[id=271, name=qtp1826556823-271, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)         at java.base@11/java.lang.Thread.run(Thread.java:834)   40) Thread[id=263, name=OverseerThreadFactory-79-thread-1, state=WAITING, group=Overseer collection creation process.]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@11/java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.base@11/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:361)         at java.base@11/java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:920)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)   41) Thread[id=260, name=TEST-TestLTROnSolrCloud.testSimpleQuery-seed#[60E04ACC72F4BA0F]-EventThread, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)         at java.base@11/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)         at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)   42) Thread[id=236, name=SolrRrdBackendFactory-75-thread-1, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)   43) Thread[id=220, name=jetty-launcher-46-thread-2-SendThread(127.0.0.1:44017), state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]         at java.base@11/sun.nio.ch.EPoll.wait(Native Method)         at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)         at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)         at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)         at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)         at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1144)   44) Thread[id=223, name=zkConnectionManagerCallback-65-thread-1, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)         at java.base@11/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)   45) Thread[id=234, name=zkConnectionManagerCallback-77-thread-1, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)         at java.base@11/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)   46) Thread[id=269, name=qtp1274737139-269, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]         at java.base@11/sun.nio.ch.EPoll.wait(Native Method)         at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)         at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)         at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)         at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)         at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.base@11/java.lang.Thread.run(Thread.java:834)   47) Thread[id=255, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/java.lang.Thread.sleep(Native Method)         at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.base@11/java.lang.Thread.run(Thread.java:834)   48) Thread[id=270, name=Scheduler-1713856743, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)   49) Thread[id=227, name=OverseerStateUpdate-72219603194675206-127.0.0.1:34689_solr-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)         at app//org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:463)         at app//org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:207)         at java.base@11/java.lang.Thread.run(Thread.java:834)   50) Thread[id=287, name=SolrRrdBackendFactory-75-thread-2, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)   51) Thread[id=191, name=qtp1826556823-191, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)         at java.base@11/java.lang.Thread.run(Thread.java:834)   52) Thread[id=221, name=jetty-launcher-46-thread-1-EventThread, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)         at java.base@11/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)         at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)   53) Thread[id=272, name=qtp1274737139-272, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)         at java.base@11/java.lang.Thread.run(Thread.java:834)   54) Thread[id=214, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/java.lang.Thread.sleep(Native Method)         at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.base@11/java.lang.Thread.run(Thread.java:834)   55) Thread[id=243, name=SolrRrdBackendFactory-81-thread-1, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)   56) Thread[id=259, name=TEST-TestLTROnSolrCloud.testSimpleQuery-seed#[60E04ACC72F4BA0F]-SendThread(127.0.0.1:44017), state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]         at java.base@11/sun.nio.ch.EPoll.wait(Native Method)         at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)         at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)         at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)         at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)         at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1144)   57) Thread[id=244, name=MetricsHistoryHandler-82-thread-1, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)   58) Thread[id=174, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]         at java.base@11/sun.nio.ch.EPoll.wait(Native Method)         at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)         at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)         at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)         at app//org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:196)         at java.base@11/java.lang.Thread.run(Thread.java:834)   59) Thread[id=175, name=SessionTracker, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/java.lang.Object.wait(Native Method)         at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:147)   60) Thread[id=190, name=qtp1274737139-190-acceptor-0@1597b916-ServerConnector@1b5045d4{SSL,[ssl, http/1.1]}{127.0.0.1:34689}, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]         at java.base@11/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at java.base@11/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)         at java.base@11/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)         at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:369)         at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.base@11/java.lang.Thread.run(Thread.java:834)   61) Thread[id=193, name=qtp1826556823-193-acceptor-0@19e6b6ad-ServerConnector@17214be1{SSL,[ssl, http/1.1]}{127.0.0.1:40687}, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]         at java.base@11/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at java.base@11/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)         at java.base@11/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)         at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:369)         at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.base@11/java.lang.Thread.run(Thread.java:834)   62) Thread[id=199, name=Scheduler-574344910, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)         at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)         at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)         at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)         at java.base@11/java.lang.Thread.run(Thread.java:834)   63) Thread[id=189, name=qtp1826556823-189, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2211)         at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)         at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.base@11/java.lang.Thread.run(Thread.java:834)   64) Thread[id=177, name=ProcessThread(sid:0 cport:44017):, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)         at java.base@11/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)         at app//org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:123)   65) Thread[id=198, name=qtp1826556823-198, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2211)         at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)         at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.base@11/java.lang.Thread.run(Thread.java:834)   66) Thread[id=250, name=Thread-44, state=WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/java.lang.Object.wait(Native Method)         at java.base@11/java.lang.Object.wait(Object.java:328)         at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:1857)   67) Thread[id=229, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/java.lang.Thread.sleep(Native Method)         at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.base@11/java.lang.Thread.run(Thread.java:834)   68) Thread[id=196, name=qtp1274737139-196, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)         at java.base@11/java.lang.Thread.run(Thread.java:834)   69) Thread[id=197, name=qtp1826556823-197, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]         at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)         at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)         at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2211)         at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)         at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)         at java.base@11/java.lang.Thread.run(Thread.java:834)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 69 threads leaked from SUITE scope at org.apache.solr.ltr.TestLTROnSolrCloud:
   1) Thread[id=192, name=qtp1274737139-192, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/sun.nio.ch.EPoll.wait(Native Method)
        at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
        at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
        at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
   2) Thread[id=185, name=qtp1826556823-185, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/sun.nio.ch.EPoll.wait(Native Method)
        at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
        at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
        at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$156/0x00007f61d8d79858.run(Unknown Source)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
   3) Thread[id=238, name=OverseerCollectionConfigSetProcessor-72219603194675206-127.0.0.1:34689_solr-n_0000000000, state=TIMED_WAITING, group=Overseer collection creation process.]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at app//org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:463)
        at app//org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:243)
        at app//org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:233)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
   4) Thread[id=188, name=qtp1274737139-188, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/sun.nio.ch.EPoll.wait(Native Method)
        at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
        at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
        at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
   5) Thread[id=237, name=MetricsHistoryHandler-76-thread-1, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
        at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
   6) Thread[id=242, name=ScheduledTrigger-77-thread-1, state=WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
        at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
   7) Thread[id=249, name=ScheduledTrigger-77-thread-4, state=WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
        at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
   8) Thread[id=211, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/java.lang.Thread.sleep(Native Method)
        at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
   9) Thread[id=228, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/java.lang.Thread.sleep(Native Method)
        at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  10) Thread[id=288, name=SolrRrdBackendFactory-81-thread-2, state=WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
        at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  11) Thread[id=248, name=ScheduledTrigger-77-thread-3, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
        at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  12) Thread[id=173, name=Thread-26, state=WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/java.lang.Object.wait(Native Method)
        at java.base@11/java.lang.Thread.join(Thread.java:1305)
        at java.base@11/java.lang.Thread.join(Thread.java:1379)
        at app//org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:313)
        at app//org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:313)
        at app//org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:496)
  13) Thread[id=222, name=jetty-launcher-46-thread-2-EventThread, state=WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
  14) Thread[id=233, name=jetty-launcher-46-thread-2-EventThread, state=WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
  15) Thread[id=207, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/java.lang.Thread.sleep(Native Method)
        at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  16) Thread[id=224, name=zkConnectionManagerCallback-64-thread-1, state=WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  17) Thread[id=195, name=qtp1826556823-195, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/sun.nio.ch.EPoll.wait(Native Method)
        at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
        at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
        at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  18) Thread[id=261, name=zkConnectionManagerCallback-90-thread-1, state=WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  19) Thread[id=208, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/java.lang.Thread.sleep(Native Method)
        at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  20) Thread[id=183, name=qtp1274737139-183, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  21) Thread[id=231, name=jetty-launcher-46-thread-2-SendThread(127.0.0.1:44017), state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/sun.nio.ch.EPoll.wait(Native Method)
        at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
        at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
        at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1144)
  22) Thread[id=184, name=qtp1274737139-184, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  23) Thread[id=187, name=qtp1826556823-187, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/sun.nio.ch.EPoll.wait(Native Method)
        at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
        at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
        at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  24) Thread[id=209, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/java.lang.Thread.sleep(Native Method)
        at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  25) Thread[id=262, name=Scheduler-1090689667, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
        at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  26) Thread[id=232, name=jetty-launcher-46-thread-1-EventThread, state=WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at app//org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
  27) Thread[id=186, name=qtp1274737139-186, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:717)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  28) Thread[id=210, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/java.lang.Thread.sleep(Native Method)
        at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  29) Thread[id=245, name=Thread-43, state=WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/java.lang.Object.wait(Native Method)
        at java.base@11/java.lang.Object.wait(Object.java:328)
        at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:1857)
  30) Thread[id=289, name=qtp1826556823-289, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/sun.nio.ch.EPoll.wait(Native Method)
        at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
        at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
        at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  31) Thread[id=230, name=jetty-launcher-46-thread-1-SendThread(127.0.0.1:44017), state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/sun.nio.ch.EPoll.wait(Native Method)
        at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
        at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
        at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1144)
  32) Thread[id=239, name=OverseerAutoScalingTriggerThread-72219603194675206-127.0.0.1:34689_solr-n_0000000000, state=WAITING, group=Overseer autoscaling triggers]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at app//org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  33) Thread[id=194, name=qtp1274737139-194, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/sun.nio.ch.EPoll.wait(Native Method)
        at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
        at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
        at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
        at app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
        at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  34) Thread[id=200, name=Scheduler-326051045, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
        at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  35) Thread[id=235, name=zkConnectionManagerCallback-79-thread-1, state=WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  36) Thread[id=273, name=qtp1274737139-273, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2211)
        at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)
        at app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  37) Thread[id=247, name=ScheduledTrigger-77-thread-2, state=WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1177)
        at java.base@11/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
  38) Thread[id=219, name=jetty-launcher-46-thread-1-SendThread(127.0.0.1:44017), state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/sun.nio.ch.EPoll.wait(Native Method)
        at java.base@11/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
        at java.base@11/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
        at java.base@11/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:136)
        at app//org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
        at app//org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1144)
  39) Thread[id=271, name=qtp1826556823-271, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
        at java.base@11/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
        at java.base@11/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
        at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:653)
        at app//org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThre

[...truncated too long message...]

EAT J2 PID([hidden email]): 2018-09-13T22:16:31, stalled for 69.2s at: ShardSplitTest.test
   [junit4] HEARTBEAT J0 PID([hidden email]): 2018-09-13T22:16:43, stalled for  181s at: ShardSplitTest.test
   [junit4] HEARTBEAT J1 PID([hidden email]): 2018-09-13T22:16:55, stalled for  181s at: ShardSplitTest.test
   [junit4] HEARTBEAT J2 PID([hidden email]): 2018-09-13T22:17:31, stalled for  129s at: ShardSplitTest.test
   [junit4] Suite: org.apache.solr.cloud.api.collections.ShardSplitTest
   [junit4] OK      40.5s J0 | ShardSplitTest.testSplitStaticIndexReplicationLink
   [junit4] OK      25.9s J0 | ShardSplitTest.testSplitShardWithRuleLink
   [junit4] OK      36.8s J0 | ShardSplitTest.testSplitMixedReplicaTypes
   [junit4] OK      34.1s J0 | ShardSplitTest.testSplitMixedReplicaTypesLink
   [junit4] OK      27.7s J0 | ShardSplitTest.testSplitShardWithRule
   [junit4] OK      55.7s J0 | ShardSplitTest.testSplitAfterFailedSplit2
   [junit4] OK      33.6s J0 | ShardSplitTest.testSplitStaticIndexReplication
   [junit4] OK      29.9s J0 | ShardSplitTest.testSplitAfterFailedSplit
   [junit4] OK       184s J0 | ShardSplitTest.test
   [junit4] OK      51.1s J0 | ShardSplitTest.testSplitWithChaosMonkey
   [junit4] Completed [4/10 (1!)] on J0 in 522.80s, 10 tests
   [junit4]
   [junit4] Suite: org.apache.solr.cloud.api.collections.ShardSplitTest
   [junit4] OK      40.4s J1 | ShardSplitTest.testSplitStaticIndexReplicationLink
   [junit4] OK      34.3s J1 | ShardSplitTest.testSplitShardWithRuleLink
   [junit4] OK      37.0s J1 | ShardSplitTest.testSplitMixedReplicaTypes
   [junit4] OK      39.7s J1 | ShardSplitTest.testSplitMixedReplicaTypesLink
   [junit4] OK      25.3s J1 | ShardSplitTest.testSplitShardWithRule
   [junit4] OK      54.4s J1 | ShardSplitTest.testSplitAfterFailedSplit2
   [junit4] OK      32.0s J1 | ShardSplitTest.testSplitStaticIndexReplication
   [junit4] OK      32.5s J1 | ShardSplitTest.testSplitAfterFailedSplit
   [junit4] OK       190s J1 | ShardSplitTest.test
   [junit4] OK      48.8s J1 | ShardSplitTest.testSplitWithChaosMonkey
   [junit4] Completed [5/10 (1!)] on J1 in 536.95s, 10 tests
   [junit4]
   [junit4] Duplicate suite name used with XML reports: org.apache.solr.cloud.api.collections.ShardSplitTest. This may confuse tools that process XML reports. Set 'ignoreDuplicateSuites' to true to skip this message.
   [junit4] Suite: org.apache.solr.cloud.api.collections.ShardSplitTest
   [junit4] OK      33.7s J2 | ShardSplitTest.testSplitStaticIndexReplicationLink
   [junit4] OK      31.9s J2 | ShardSplitTest.testSplitShardWithRuleLink
   [junit4] OK      36.5s J2 | ShardSplitTest.testSplitMixedReplicaTypes
   [junit4] OK      37.0s J2 | ShardSplitTest.testSplitMixedReplicaTypesLink
   [junit4] OK      27.5s J2 | ShardSplitTest.testSplitShardWithRule
   [junit4] OK      54.2s J2 | ShardSplitTest.testSplitAfterFailedSplit2
   [junit4] OK      32.0s J2 | ShardSplitTest.testSplitStaticIndexReplication
   [junit4] OK      37.6s J2 | ShardSplitTest.testSplitAfterFailedSplit
   [junit4] OK       185s J2 | ShardSplitTest.test
   [junit4] OK      48.6s J2 | ShardSplitTest.testSplitWithChaosMonkey
   [junit4] Completed [6/10 (1!)] on J2 in 524.64s, 10 tests
   [junit4]
   [junit4] Suite: org.apache.solr.cloud.ReplaceNodeTest
   [junit4] OK      17.4s J2 | ReplaceNodeTest.test
   [junit4] Completed [7/10 (1!)] on J2 in 19.61s, 1 test
   [junit4]
   [junit4] Suite: org.apache.solr.cloud.ReplaceNodeTest
   [junit4] OK      17.2s J2 | ReplaceNodeTest.test
   [junit4] Completed [8/10 (1!)] on J2 in 19.83s, 1 test
   [junit4]
   [junit4] JVM J2: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/temp/junit4-J2-20180913_220855_6277005062158998140566.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J2: EOF ----
   [junit4] HEARTBEAT J0 PID([hidden email]): 2018-09-13T22:23:31, stalled for 67.1s at: ShardSplitTest.test
   [junit4] HEARTBEAT J1 PID([hidden email]): 2018-09-13T22:23:43, stalled for 62.8s at: ShardSplitTest.test
   [junit4] HEARTBEAT J0 PID([hidden email]): 2018-09-13T22:24:31, stalled for  127s at: ShardSplitTest.test
   [junit4] HEARTBEAT J1 PID([hidden email]): 2018-09-13T22:24:43, stalled for  123s at: ShardSplitTest.test
   [junit4] HEARTBEAT J0 PID([hidden email]): 2018-09-13T22:25:31, stalled for  187s at: ShardSplitTest.test
   [junit4] HEARTBEAT J1 PID([hidden email]): 2018-09-13T22:25:43, stalled for  183s at: ShardSplitTest.test
   [junit4] Suite: org.apache.solr.cloud.api.collections.ShardSplitTest
   [junit4] OK      32.2s J0 | ShardSplitTest.testSplitStaticIndexReplicationLink
   [junit4] OK      26.7s J0 | ShardSplitTest.testSplitShardWithRuleLink
   [junit4] OK      35.9s J0 | ShardSplitTest.testSplitMixedReplicaTypes
   [junit4] OK      35.4s J0 | ShardSplitTest.testSplitMixedReplicaTypesLink
   [junit4] OK      33.6s J0 | ShardSplitTest.testSplitShardWithRule
   [junit4] OK      53.2s J0 | ShardSplitTest.testSplitAfterFailedSplit2
   [junit4] OK      32.6s J0 | ShardSplitTest.testSplitStaticIndexReplication
   [junit4] OK      35.2s J0 | ShardSplitTest.testSplitAfterFailedSplit
   [junit4] OK       187s J0 | ShardSplitTest.test
   [junit4] OK      48.5s J0 | ShardSplitTest.testSplitWithChaosMonkey
   [junit4] Completed [9/10 (1!)] on J0 in 521.33s, 10 tests
   [junit4]
   [junit4] JVM J0: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/temp/junit4-J0-20180913_220855_62715054922483788960731.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----
   [junit4] Suite: org.apache.solr.cloud.api.collections.ShardSplitTest
   [junit4] OK      32.6s J1 | ShardSplitTest.testSplitStaticIndexReplicationLink
   [junit4] OK      34.3s J1 | ShardSplitTest.testSplitShardWithRuleLink
   [junit4] OK      36.4s J1 | ShardSplitTest.testSplitMixedReplicaTypes
   [junit4] OK      37.7s J1 | ShardSplitTest.testSplitMixedReplicaTypesLink
   [junit4] OK      25.5s J1 | ShardSplitTest.testSplitShardWithRule
   [junit4] OK      53.2s J1 | ShardSplitTest.testSplitAfterFailedSplit2
   [junit4] OK      31.9s J1 | ShardSplitTest.testSplitStaticIndexReplication
   [junit4] OK      35.1s J1 | ShardSplitTest.testSplitAfterFailedSplit
   [junit4] OK       190s J1 | ShardSplitTest.test
   [junit4] OK      31.2s J1 | ShardSplitTest.testSplitWithChaosMonkey
   [junit4] Completed [10/10 (1!)] on J1 in 509.01s, 10 tests
   [junit4]
   [junit4] JVM J1: stderr was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/temp/junit4-J1-20180913_220855_6273920035117075894438.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----
   [junit4]
   [junit4] Tests with failures [seed: CB4F6CDCCC527784]:
   [junit4]   - org.apache.solr.cloud.ReplaceNodeTest.test
   [junit4]
   [junit4]
   [junit4] JVM J0:     0.76 ..  1046.17 =  1045.41s
   [junit4] JVM J1:     0.74 ..  1048.05 =  1047.31s
   [junit4] JVM J2:     0.76 ..   661.28 =   660.51s
   [junit4] Execution time total: 17 minutes 28 seconds
   [junit4] Tests summary: 10 suites, 55 tests, 1 error

BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1567: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1092: There were test failures: 10 suites, 55 tests, 1 error [seed: CB4F6CDCCC527784]

Total time: 17 minutes 29 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.api.collections.ShardSplitTest
[repro]   1/5 failed: org.apache.solr.cloud.ReplaceNodeTest
[repro]   2/5 failed: org.apache.solr.client.solrj.embedded.LargeVolumeJettyTest
[repro]   2/5 failed: org.apache.solr.ltr.TestLTROnSolrCloud
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=d6641ffb1d5fd5c9d3743561e2c5158951317713, workspace=/var/lib/jenkins/workspace/Lucene-Solr-master-Linux
[WARNINGS] Computing warning deltas based on reference build #22852
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_172) - Build # 22854 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/22854/
Java: 64bit/jdk1.8.0_172 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.NodeLostTriggerTest.testTrigger

Error Message:
[127.0.0.1:41477_solr] doesn't contain 127.0.0.1:41345_solr

Stack Trace:
java.lang.AssertionError: [127.0.0.1:41477_solr] doesn't contain 127.0.0.1:41345_solr
        at __randomizedtesting.SeedInfo.seed([39F6F71B76BC849D:5A3DC199EF73F7B0]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.NodeLostTriggerTest.testTrigger(NodeLostTriggerTest.java:117)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13493 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.NodeLostTriggerTest
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.NodeLostTriggerTest_39F6F71B76BC849D-001/init-core-data-001
   [junit4]   2> 974376 WARN  (SUITE-NodeLostTriggerTest-seed#[39F6F71B76BC849D]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 974376 INFO  (SUITE-NodeLostTriggerTest-seed#[39F6F71B76BC849D]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 974377 INFO  (SUITE-NodeLostTriggerTest-seed#[39F6F71B76BC849D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 974377 INFO  (SUITE-NodeLostTriggerTest-seed#[39F6F71B76BC849D]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 974378 INFO  (SUITE-NodeLostTriggerTest-seed#[39F6F71B76BC849D]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 5 servers in /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.NodeLostTriggerTest_39F6F71B76BC849D-001/tempDir-001
   [junit4]   2> 974378 INFO  (SUITE-NodeLostTriggerTest-seed#[39F6F71B76BC849D]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 974378 INFO  (Thread-3768) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 974378 INFO  (Thread-3768) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 974381 ERROR (Thread-3768) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 974478 INFO  (SUITE-NodeLostTriggerTest-seed#[39F6F71B76BC849D]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:34981
   [junit4]   2> 974480 INFO  (zkConnectionManagerCallback-3236-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974482 INFO  (jetty-launcher-3233-thread-4) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 974482 INFO  (jetty-launcher-3233-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 974482 INFO  (jetty-launcher-3233-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 974482 INFO  (jetty-launcher-3233-thread-3) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 974483 INFO  (jetty-launcher-3233-thread-5) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 974483 INFO  (jetty-launcher-3233-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 974483 INFO  (jetty-launcher-3233-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 974483 INFO  (jetty-launcher-3233-thread-3) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 974483 INFO  (jetty-launcher-3233-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 974483 INFO  (jetty-launcher-3233-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 974483 INFO  (jetty-launcher-3233-thread-2) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 974483 INFO  (jetty-launcher-3233-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 974483 INFO  (jetty-launcher-3233-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 974483 INFO  (jetty-launcher-3233-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 974483 INFO  (jetty-launcher-3233-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@bcdebda{/solr,null,AVAILABLE}
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@9f5252c{/solr,null,AVAILABLE}
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@69f86a1{/solr,null,AVAILABLE}
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-4) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-5) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-5) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-5) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5787c0d9{/solr,null,AVAILABLE}
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4d159df7{SSL,[ssl, http/1.1]}{127.0.0.1:41345}
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@33ffbd56{SSL,[ssl, http/1.1]}{127.0.0.1:34131}
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-3) [    ] o.e.j.s.Server Started @974510ms
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-2) [    ] o.e.j.s.Server Started @974510ms
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41345}
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34131}
   [junit4]   2> 974484 ERROR (jetty-launcher-3233-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 974484 ERROR (jetty-launcher-3233-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-3) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-14T01:12:13.566Z
   [junit4]   2> 974484 INFO  (jetty-launcher-3233-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-14T01:12:13.566Z
   [junit4]   2> 974485 INFO  (jetty-launcher-3233-thread-5) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7aa81e71{/solr,null,AVAILABLE}
   [junit4]   2> 974485 INFO  (jetty-launcher-3233-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4a67fbe9{SSL,[ssl, http/1.1]}{127.0.0.1:41087}
   [junit4]   2> 974485 INFO  (jetty-launcher-3233-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1bf84a54{SSL,[ssl, http/1.1]}{127.0.0.1:36467}
   [junit4]   2> 974485 INFO  (jetty-launcher-3233-thread-5) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4056811f{SSL,[ssl, http/1.1]}{127.0.0.1:37257}
   [junit4]   2> 974485 INFO  (jetty-launcher-3233-thread-1) [    ] o.e.j.s.Server Started @974512ms
   [junit4]   2> 974485 INFO  (jetty-launcher-3233-thread-4) [    ] o.e.j.s.Server Started @974512ms
   [junit4]   2> 974485 INFO  (jetty-launcher-3233-thread-5) [    ] o.e.j.s.Server Started @974512ms
   [junit4]   2> 974485 INFO  (jetty-launcher-3233-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41087}
   [junit4]   2> 974485 INFO  (jetty-launcher-3233-thread-5) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37257}
   [junit4]   2> 974485 INFO  (jetty-launcher-3233-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36467}
   [junit4]   2> 974485 ERROR (jetty-launcher-3233-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 974485 INFO  (jetty-launcher-3233-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 974485 INFO  (jetty-launcher-3233-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 974485 ERROR (jetty-launcher-3233-thread-5) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 974485 ERROR (jetty-launcher-3233-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 974485 INFO  (jetty-launcher-3233-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 974485 INFO  (jetty-launcher-3233-thread-4) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 974485 INFO  (jetty-launcher-3233-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 974485 INFO  (jetty-launcher-3233-thread-5) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 974486 INFO  (jetty-launcher-3233-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-14T01:12:13.567Z
   [junit4]   2> 974486 INFO  (jetty-launcher-3233-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 974486 INFO  (jetty-launcher-3233-thread-5) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 974486 INFO  (jetty-launcher-3233-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 974486 INFO  (jetty-launcher-3233-thread-5) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 974486 INFO  (jetty-launcher-3233-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 974486 INFO  (jetty-launcher-3233-thread-5) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 974486 INFO  (jetty-launcher-3233-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-14T01:12:13.568Z
   [junit4]   2> 974486 INFO  (jetty-launcher-3233-thread-5) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-14T01:12:13.568Z
   [junit4]   2> 974486 INFO  (zkConnectionManagerCallback-3240-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974487 INFO  (zkConnectionManagerCallback-3238-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974487 INFO  (jetty-launcher-3233-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 974487 INFO  (zkConnectionManagerCallback-3244-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974487 INFO  (jetty-launcher-3233-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 974487 INFO  (zkConnectionManagerCallback-3246-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974487 INFO  (jetty-launcher-3233-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 974488 INFO  (zkConnectionManagerCallback-3242-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974488 INFO  (jetty-launcher-3233-thread-5) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 974488 INFO  (jetty-launcher-3233-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 974506 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1009402cb510004, likely client has closed socket
   [junit4]   2> 974507 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1009402cb510001, likely client has closed socket
   [junit4]   2> 974507 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1009402cb510002, likely client has closed socket
   [junit4]   2> 974507 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1009402cb510003, likely client has closed socket
   [junit4]   2> 974550 INFO  (jetty-launcher-3233-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34981/solr
   [junit4]   2> 974577 INFO  (zkConnectionManagerCallback-3250-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974587 INFO  (zkConnectionManagerCallback-3252-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974589 INFO  (jetty-launcher-3233-thread-5) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34981/solr
   [junit4]   2> 974601 INFO  (zkConnectionManagerCallback-3256-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974617 INFO  (zkConnectionManagerCallback-3260-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974622 INFO  (jetty-launcher-3233-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34981/solr
   [junit4]   2> 974623 INFO  (jetty-launcher-3233-thread-5) [n:127.0.0.1:37257_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37257_solr
   [junit4]   2> 974624 INFO  (jetty-launcher-3233-thread-5) [n:127.0.0.1:37257_solr    ] o.a.s.c.Overseer Overseer (id=72220333759856649-127.0.0.1:37257_solr-n_0000000000) starting
   [junit4]   2> 974629 INFO  (zkConnectionManagerCallback-3266-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974631 INFO  (zkConnectionManagerCallback-3269-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974631 INFO  (jetty-launcher-3233-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34981/solr
   [junit4]   2> 974633 INFO  (jetty-launcher-3233-thread-2) [n:127.0.0.1:34131_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34131_solr
   [junit4]   2> 974634 INFO  (jetty-launcher-3233-thread-3) [n:127.0.0.1:41345_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 974637 INFO  (jetty-launcher-3233-thread-3) [n:127.0.0.1:41345_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 974637 INFO  (jetty-launcher-3233-thread-3) [n:127.0.0.1:41345_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41345_solr
   [junit4]   2> 974641 INFO  (zkCallback-3268-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 974645 INFO  (zkConnectionManagerCallback-3275-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974645 INFO  (zkCallback-3259-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 974646 INFO  (jetty-launcher-3233-thread-5) [n:127.0.0.1:37257_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 974647 INFO  (jetty-launcher-3233-thread-5) [n:127.0.0.1:37257_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34981/solr ready
   [junit4]   2> 974649 INFO  (zkConnectionManagerCallback-3277-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974650 INFO  (zkCallback-3251-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 974650 INFO  (OverseerStateUpdate-72220333759856649-127.0.0.1:37257_solr-n_0000000000) [n:127.0.0.1:37257_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:37257_solr
   [junit4]   2> 974650 DEBUG (OverseerAutoScalingTriggerThread-72220333759856649-127.0.0.1:37257_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 974652 INFO  (jetty-launcher-3233-thread-5) [n:127.0.0.1:37257_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 974652 DEBUG (OverseerAutoScalingTriggerThread-72220333759856649-127.0.0.1:37257_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 974652 INFO  (jetty-launcher-3233-thread-5) [n:127.0.0.1:37257_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37257_solr
   [junit4]   2> 974652 DEBUG (OverseerAutoScalingTriggerThread-72220333759856649-127.0.0.1:37257_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 974652 DEBUG (OverseerAutoScalingTriggerThread-72220333759856649-127.0.0.1:37257_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 974653 INFO  (zkConnectionManagerCallback-3285-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974658 DEBUG (OverseerAutoScalingTriggerThread-72220333759856649-127.0.0.1:37257_solr-n_0000000000) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:34131_solr, 127.0.0.1:41345_solr]
   [junit4]   2> 974662 INFO  (jetty-launcher-3233-thread-2) [n:127.0.0.1:34131_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 974662 INFO  (jetty-launcher-3233-thread-2) [n:127.0.0.1:34131_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34981/solr ready
   [junit4]   2> 974666 INFO  (zkCallback-3259-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 974666 INFO  (zkCallback-3274-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 974666 INFO  (zkCallback-3268-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 974666 INFO  (zkCallback-3251-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 974669 INFO  (zkConnectionManagerCallback-3291-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974669 DEBUG (OverseerAutoScalingTriggerThread-72220333759856649-127.0.0.1:37257_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 974670 INFO  (jetty-launcher-3233-thread-2) [n:127.0.0.1:34131_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 974670 INFO  (zkConnectionManagerCallback-3289-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974670 INFO  (jetty-launcher-3233-thread-3) [n:127.0.0.1:41345_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 974670 DEBUG (ScheduledTrigger-5020-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 974671 INFO  (jetty-launcher-3233-thread-3) [n:127.0.0.1:41345_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34981/solr ready
   [junit4]   2> 974673 DEBUG (OverseerAutoScalingTriggerThread-72220333759856649-127.0.0.1:37257_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 974673 INFO  (jetty-launcher-3233-thread-4) [n:127.0.0.1:36467_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 974675 INFO  (jetty-launcher-3233-thread-4) [n:127.0.0.1:36467_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 974675 INFO  (jetty-launcher-3233-thread-4) [n:127.0.0.1:36467_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36467_solr
   [junit4]   2> 974676 INFO  (zkCallback-3268-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 974677 INFO  (zkCallback-3251-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 974681 INFO  (jetty-launcher-3233-thread-5) [n:127.0.0.1:37257_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 974686 INFO  (zkCallback-3274-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 974687 INFO  (zkCallback-3288-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 974690 INFO  (jetty-launcher-3233-thread-2) [n:127.0.0.1:34131_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34131.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 974693 INFO  (zkCallback-3259-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 974694 INFO  (jetty-launcher-3233-thread-2) [n:127.0.0.1:34131_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34131.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 974694 INFO  (jetty-launcher-3233-thread-2) [n:127.0.0.1:34131_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34131.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 974695 INFO  (zkCallback-3284-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 974695 INFO  (jetty-launcher-3233-thread-2) [n:127.0.0.1:34131_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.NodeLostTriggerTest_39F6F71B76BC849D-001/tempDir-001/node2/.
   [junit4]   2> 974695 INFO  (jetty-launcher-3233-thread-3) [n:127.0.0.1:41345_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 974699 INFO  (zkConnectionManagerCallback-3302-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974699 INFO  (jetty-launcher-3233-thread-4) [n:127.0.0.1:36467_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 974699 INFO  (jetty-launcher-3233-thread-4) [n:127.0.0.1:36467_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34981/solr ready
   [junit4]   2> 974700 INFO  (jetty-launcher-3233-thread-4) [n:127.0.0.1:36467_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 974702 INFO  (zkCallback-3290-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 974702 INFO  (jetty-launcher-3233-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34981/solr
   [junit4]   2> 974704 INFO  (zkConnectionManagerCallback-3306-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974705 INFO  (zkConnectionManagerCallback-3308-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974707 INFO  (jetty-launcher-3233-thread-1) [n:127.0.0.1:41087_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 974709 INFO  (jetty-launcher-3233-thread-1) [n:127.0.0.1:41087_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 974709 INFO  (jetty-launcher-3233-thread-1) [n:127.0.0.1:41087_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41087_solr
   [junit4]   2> 974713 INFO  (zkCallback-3259-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 974713 INFO  (zkCallback-3268-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 974730 INFO  (zkCallback-3290-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 974730 INFO  (zkCallback-3284-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 974730 INFO  (zkCallback-3307-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 974730 INFO  (zkCallback-3274-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 974730 INFO  (zkCallback-3251-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 974730 INFO  (zkCallback-3288-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 974730 INFO  (zkCallback-3301-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 974736 INFO  (jetty-launcher-3233-thread-5) [n:127.0.0.1:37257_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37257.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 974736 INFO  (jetty-launcher-3233-thread-4) [n:127.0.0.1:36467_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36467.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 974737 INFO  (jetty-launcher-3233-thread-3) [n:127.0.0.1:41345_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41345.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 974744 INFO  (jetty-launcher-3233-thread-4) [n:127.0.0.1:36467_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36467.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 974744 INFO  (jetty-launcher-3233-thread-5) [n:127.0.0.1:37257_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37257.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 974745 INFO  (jetty-launcher-3233-thread-4) [n:127.0.0.1:36467_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36467.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 974745 INFO  (jetty-launcher-3233-thread-5) [n:127.0.0.1:37257_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37257.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 974745 INFO  (jetty-launcher-3233-thread-4) [n:127.0.0.1:36467_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.NodeLostTriggerTest_39F6F71B76BC849D-001/tempDir-001/node4/.
   [junit4]   2> 974746 INFO  (jetty-launcher-3233-thread-5) [n:127.0.0.1:37257_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.NodeLostTriggerTest_39F6F71B76BC849D-001/tempDir-001/node5/.
   [junit4]   2> 974746 INFO  (zkConnectionManagerCallback-3315-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974746 INFO  (jetty-launcher-3233-thread-3) [n:127.0.0.1:41345_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41345.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 974746 INFO  (jetty-launcher-3233-thread-3) [n:127.0.0.1:41345_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41345.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 974747 INFO  (jetty-launcher-3233-thread-3) [n:127.0.0.1:41345_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.NodeLostTriggerTest_39F6F71B76BC849D-001/tempDir-001/node3/.
   [junit4]   2> 974749 INFO  (jetty-launcher-3233-thread-1) [n:127.0.0.1:41087_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
   [junit4]   2> 974750 INFO  (jetty-launcher-3233-thread-1) [n:127.0.0.1:41087_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34981/solr ready
   [junit4]   2> 974750 INFO  (jetty-launcher-3233-thread-1) [n:127.0.0.1:41087_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 974759 INFO  (jetty-launcher-3233-thread-1) [n:127.0.0.1:41087_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41087.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 974764 INFO  (jetty-launcher-3233-thread-1) [n:127.0.0.1:41087_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41087.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 974764 INFO  (jetty-launcher-3233-thread-1) [n:127.0.0.1:41087_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41087.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 974765 INFO  (jetty-launcher-3233-thread-1) [n:127.0.0.1:41087_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.NodeLostTriggerTest_39F6F71B76BC849D-001/tempDir-001/node1/.
   [junit4]   2> 974847 INFO  (zkConnectionManagerCallback-3318-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974849 INFO  (zkConnectionManagerCallback-3323-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974849 INFO  (SUITE-NodeLostTriggerTest-seed#[39F6F71B76BC849D]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
   [junit4]   2> 974849 INFO  (SUITE-NodeLostTriggerTest-seed#[39F6F71B76BC849D]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34981/solr ready
   [junit4]   2> 974881 INFO  (TEST-NodeLostTriggerTest.testActionLifecycle-seed#[39F6F71B76BC849D]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testActionLifecycle
   [junit4]   2> 974882 DEBUG (TEST-NodeLostTriggerTest.testActionLifecycle-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_added_trigger - Initial livenodes: [127.0.0.1:41087_solr, 127.0.0.1:37257_solr, 127.0.0.1:34131_solr, 127.0.0.1:41345_solr, 127.0.0.1:36467_solr]
   [junit4]   2> 974883 INFO  (TEST-NodeLostTriggerTest.testActionLifecycle-seed#[39F6F71B76BC849D]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testActionLifecycle
   [junit4]   2> 974914 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testListenerAcceptance
   [junit4]   2> 974914 DEBUG (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_added_trigger - Initial livenodes: [127.0.0.1:41087_solr, 127.0.0.1:37257_solr, 127.0.0.1:34131_solr, 127.0.0.1:41345_solr, 127.0.0.1:36467_solr]
   [junit4]   2> 974915 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 974916 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 974916 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 974916 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 974916 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@bf68bf6{/solr,null,AVAILABLE}
   [junit4]   2> 974917 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3bcafc80{SSL,[ssl, http/1.1]}{127.0.0.1:41477}
   [junit4]   2> 974917 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.Server Started @974943ms
   [junit4]   2> 974917 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41477}
   [junit4]   2> 974917 ERROR (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 974917 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 974917 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 974917 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 974917 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 974917 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-14T01:12:13.999Z
   [junit4]   2> 974918 INFO  (zkConnectionManagerCallback-3325-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 974918 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 975131 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34981/solr
   [junit4]   2> 975132 INFO  (zkConnectionManagerCallback-3329-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 975133 INFO  (zkConnectionManagerCallback-3331-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 975136 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [n:127.0.0.1:41477_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
   [junit4]   2> 975137 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [n:127.0.0.1:41477_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 975137 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [n:127.0.0.1:41477_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41477_solr
   [junit4]   2> 975138 INFO  (zkCallback-3259-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
   [junit4]   2> 975138 INFO  (zkCallback-3251-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
   [junit4]   2> 975138 INFO  (zkCallback-3274-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
   [junit4]   2> 975138 INFO  (zkCallback-3288-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
   [junit4]   2> 975138 INFO  (zkCallback-3268-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
   [junit4]   2> 975138 INFO  (zkCallback-3284-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
   [junit4]   2> 975138 INFO  (zkCallback-3307-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
   [junit4]   2> 975138 INFO  (zkCallback-3301-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
   [junit4]   2> 975138 INFO  (zkCallback-3322-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
   [junit4]   2> 975138 INFO  (zkCallback-3330-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
   [junit4]   2> 975138 INFO  (zkCallback-3314-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
   [junit4]   2> 975140 INFO  (zkCallback-3290-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
   [junit4]   2> 975154 INFO  (zkConnectionManagerCallback-3338-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 975154 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [n:127.0.0.1:41477_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (6)
   [junit4]   2> 975155 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [n:127.0.0.1:41477_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34981/solr ready
   [junit4]   2> 975155 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [n:127.0.0.1:41477_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 975185 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [n:127.0.0.1:41477_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41477.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 975192 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [n:127.0.0.1:41477_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41477.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 975192 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [n:127.0.0.1:41477_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41477.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 975193 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [n:127.0.0.1:41477_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.NodeLostTriggerTest_39F6F71B76BC849D-001/tempDir-001/node6/.
   [junit4]   2> 975253 INFO  (zkConnectionManagerCallback-3341-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 975254 DEBUG (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_added_trigger with currently live nodes: 6
   [junit4]   2> 975254 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@3bcafc80{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 975255 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2011917458
   [junit4]   2> 975255 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 975255 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3fbd411d: rootName = solr_41477, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@742d0e7c
   [junit4]   2> 975263 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 975263 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5d48b6e7: rootName = solr_41477, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@4cacfe20
   [junit4]   2> 975266 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 975266 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7f1dc057: rootName = solr_41477, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@7add2619
   [junit4]   2> 975267 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:41477_solr
   [junit4]   2> 975268 INFO  (zkCallback-3288-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
   [junit4]   2> 975268 INFO  (zkCallback-3259-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
   [junit4]   2> 975268 INFO  (zkCallback-3274-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
   [junit4]   2> 975268 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 975268 INFO  (zkCallback-3284-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
   [junit4]   2> 975268 INFO  (zkCallback-3301-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
   [junit4]   2> 975268 INFO  (zkCallback-3251-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
   [junit4]   2> 975268 INFO  (zkCallback-3314-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
   [junit4]   2> 975268 INFO  (zkCallback-3268-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
   [junit4]   2> 975268 INFO  (zkCallback-3322-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
   [junit4]   2> 975268 INFO  (zkCallback-3330-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
   [junit4]   2> 975268 INFO  (zkCallback-3337-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
   [junit4]   2> 975268 INFO  (zkCallback-3290-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
   [junit4]   2> 975269 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@bf68bf6{/solr,null,UNAVAILABLE}
   [junit4]   2> 975269 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 975270 INFO  (zkConnectionManagerCallback-3343-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 975270 DEBUG (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_added_trigger with currently live nodes: 5
   [junit4]   2> 975270 DEBUG (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:41477_solr
   [junit4]   2> 975270 DEBUG (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:41477_solr]
   [junit4]   2> 975271 DEBUG (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger processor for lost nodes: [127.0.0.1:41477_solr] is not ready, will try later
   [junit4]   2> 975271 DEBUG (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_added_trigger with currently live nodes: 5
   [junit4]   2> 975271 DEBUG (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:41477_solr]
   [junit4]   2> 975271 DEBUG (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_added_trigger with currently live nodes: 5
   [junit4]   2> 975271 INFO  (TEST-NodeLostTriggerTest.testListenerAcceptance-seed#[39F6F71B76BC849D]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testListenerAcceptance
   [junit4]   2> 975276 INFO  (zkCallback-3307-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
   [junit4]   2> 975304 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testTrigger
   [junit4]   2> 975304 DEBUG (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:41087_solr, 127.0.0.1:37257_solr, 127.0.0.1:34131_solr, 127.0.0.1:41345_solr, 127.0.0.1:36467_solr]
   [junit4]   2> 975305 DEBUG (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:41477_solr
   [junit4]   2> 975305 DEBUG (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 5
   [junit4]   2> 975305 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@4d159df7{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 975306 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=326498044
   [junit4]   2> 975306 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 975306 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@45cb8f4f: rootName = solr_41345, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@55433a18
   [junit4]   2> 975311 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 975312 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7950d43d: rootName = solr_41345, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@4cacfe20
   [junit4]   2> 975314 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 975314 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5a9e2039: rootName = solr_41345, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@7add2619
   [junit4]   2> 975315 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:41345_solr
   [junit4]   2> 975316 INFO  (zkCallback-3259-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 975316 INFO  (zkCallback-3307-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 975316 INFO  (zkCallback-3274-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 975316 INFO  (zkCallback-3288-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 975316 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 975316 INFO  (zkCallback-3322-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 975316 INFO  (zkCallback-3290-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 975316 INFO  (zkCallback-3301-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 975316 INFO  (zkCallback-3314-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 975316 INFO  (zkCallback-3251-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 975316 INFO  (zkCallback-3284-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 975316 INFO  (zkCallback-3268-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
   [junit4]   2> 975318 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@bcdebda{/solr,null,UNAVAILABLE}
   [junit4]   2> 975319 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 975319 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@4a67fbe9{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 975320 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1722133855
   [junit4]   2> 975320 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 975320 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@40f024ab: rootName = solr_41087, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@a99d1f6
   [junit4]   2> 975325 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 975325 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@74f90e4f: rootName = solr_41087, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@4cacfe20
   [junit4]   2> 975328 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 975328 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5a8d221b: rootName = solr_41087, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@7add2619
   [junit4]   2> 975329 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:41087_solr
   [junit4]   2> 975329 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 975329 INFO  (zkCallback-3274-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 975329 INFO  (zkCallback-3251-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 975329 INFO  (zkCallback-3301-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 975329 INFO  (zkCallback-3307-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 975329 INFO  (zkCallback-3314-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 975330 INFO  (zkCallback-3259-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 975330 INFO  (zkCallback-3284-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 975330 INFO  (zkCallback-3290-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 975330 INFO  (zkCallback-3322-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 975332 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@69f86a1{/solr,null,UNAVAILABLE}
   [junit4]   2> 975332 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 975671 DEBUG (ScheduledTrigger-5020-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 975671 DEBUG (ScheduledTrigger-5020-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:41345_solr
   [junit4]   2> 976334 DEBUG (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
   [junit4]   2> 976334 DEBUG (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:41087_solr
   [junit4]   2> 976334 DEBUG (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:41345_solr
   [junit4]   2> 976672 DEBUG (ScheduledTrigger-5020-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 977334 DEBUG (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
   [junit4]   2> 977672 DEBUG (ScheduledTrigger-5020-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 978334 DEBUG (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
   [junit4]   2> 978334 DEBUG (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:41477_solr]
   [junit4]   2> 978672 DEBUG (ScheduledTrigger-5020-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 979335 INFO  (TEST-NodeLostTriggerTest.testTrigger-seed#[39F6F71B76BC849D]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testTrigger
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=NodeLostTriggerTest -Dtests.method=testTrigger -Dtests.seed=39F6F71B76BC849D -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en-US -Dtests.timezone=US/Eastern -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 4.07s J0 | NodeLostTriggerTest.testTrigger <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: [127.0.0.1:41477_solr] doesn't contain 127.0.0.1:41345_solr
   [junit4]    > at __randomizedtesting.SeedInfo.seed([39F6F71B76BC849D:5A3DC199EF73F7B0]:0)
   [junit4]    > at org.apache.solr.cloud.autoscaling.NodeLostTriggerTest.testTrigger(NodeLostTriggerTest.java:117)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 979353 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testRestoreState
   [junit4]   2> 979354 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 979358 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 979358 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 979358 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 979358 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@33d71d26{/solr,null,AVAILABLE}
   [junit4]   2> 979359 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@171957d8{SSL,[ssl, http/1.1]}{127.0.0.1:36183}
   [junit4]   2> 979359 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.Server Started @979385ms
   [junit4]   2> 979359 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36183}
   [junit4]   2> 979359 ERROR (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 979359 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 979359 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 979359 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 979359 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 979359 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-09-14T01:12:18.441Z
   [junit4]   2> 979360 INFO  (zkConnectionManagerCallback-3345-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 979360 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 979418 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34981/solr
   [junit4]   2> 979418 INFO  (zkConnectionManagerCallback-3349-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 979419 INFO  (zkConnectionManagerCallback-3351-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 979422 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 979424 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 979424 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36183_solr
   [junit4]   2> 979424 INFO  (zkCallback-3301-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 979424 INFO  (zkCallback-3290-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 979424 INFO  (zkCallback-3322-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 979424 INFO  (zkCallback-3284-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 979424 INFO  (zkCallback-3251-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 979424 INFO  (zkCallback-3259-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 979424 INFO  (zkCallback-3350-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 979425 INFO  (zkCallback-3274-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 979442 INFO  (zkConnectionManagerCallback-3358-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 979442 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 979443 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34981/solr ready
   [junit4]   2> 979443 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 979460 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36183.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 979468 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36183.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 979468 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36183.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@474a80f9
   [junit4]   2> 979470 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.NodeLostTriggerTest_39F6F71B76BC849D-001/tempDir-001/node7/.
   [junit4]   2> 979511 DEBUG (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:36183_solr, 127.0.0.1:37257_solr, 127.0.0.1:34131_solr, 127.0.0.1:36467_solr]
   [junit4]   2> 979511 DEBUG (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:41087_solr
   [junit4]   2> 979511 DEBUG (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:41345_solr
   [junit4]   2> 979512 DEBUG (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
   [junit4]   2> 979512 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@171957d8{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 979513 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=534992206
   [junit4]   2> 979513 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 979513 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5e4b0801: rootName = solr_36183, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@3071a358
   [junit4]   2> 979522 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 979522 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@419ac204: rootName = solr_36183, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@4cacfe20
   [junit4]   2> 979525 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 979525 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@33ad148a: rootName = solr_36183, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@7add2619
   [junit4]   2> 979525 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:36183_solr
   [junit4]   2> 979526 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 979526 INFO  (zkCallback-3274-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 979526 INFO  (zkCallback-3322-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 979526 INFO  (zkCallback-3251-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 979526 INFO  (zkCallback-3284-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 979526 INFO  (zkCallback-3290-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 979526 INFO  (zkCallback-3259-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 979526 INFO  (zkCallback-3357-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 979526 INFO  (zkCallback-3301-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 979526 INFO  (zkCallback-3350-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
   [junit4]   2> 979526 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@33d71d26{/solr,null,UNAVAILABLE}
   [junit4]   2> 979527 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 979527 DEBUG (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
   [junit4]   2> 979527 DEBUG (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:36183_solr
   [junit4]   2> 979527 DEBUG (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger some_different_name - Initial livenodes: [127.0.0.1:37257_solr, 127.0.0.1:34131_solr, 127.0.0.1:36467_solr]
   [junit4]   2> 979527 DEBUG (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:36183_solr
   [junit4]   2> 979527 DEBUG (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:37257_solr, 127.0.0.1:34131_solr, 127.0.0.1:36467_solr]
   [junit4]   2> 979528 DEBUG (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
   [junit4]   2> 979673 DEBUG (ScheduledTrigger-5020-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 980528 DEBUG (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
   [junit4]   2> 980673 DEBUG (ScheduledTrigger-5020-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 981528 DEBUG (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
   [junit4]   2> 981673 DEBUG (ScheduledTrigger-5020-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 982528 DEBUG (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
   [junit4]   2> 982673 DEBUG (ScheduledTrigger-5020-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 983528 DEBUG (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
   [junit4]   2> 983528 DEBUG (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:41087_solr, 127.0.0.1:41345_solr, 127.0.0.1:36183_solr]
   [junit4]   2> 983673 DEBUG (ScheduledTrigger-5020-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 984529 INFO  (TEST-NodeLostTriggerTest.testRestoreState-seed#[39F6F71B76BC849D]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testRestoreState
   [junit4]   2> 984531 INFO  (jetty-closer-3234-thread-3) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@1bf84a54{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 984531 INFO  (jetty-closer-3234-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@33ffbd56{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 984531 INFO  (jetty-closer-3234-thread-2) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@4056811f{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 984531 INFO  (jetty-closer-3234-thread-3) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=186822345
   [junit4]   2> 984532 INFO  (jetty-closer-3234-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1076740704
   [junit4]   2> 984532 INFO  (jetty-closer-3234-thread-2) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=222948190
   [junit4]   2> 984532 INFO  (jetty-closer-3234-thread-3) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 984532 INFO  (jetty-closer-3234-thread-3) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3152ccf6: rootName = solr_36467, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@31e69f36
   [junit4]   2> 984532 INFO  (jetty-closer-3234-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 984532 INFO  (jetty-closer-3234-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5848cf04: rootName = solr_37257, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@160cc169
   [junit4]   2> 984533 INFO  (jetty-closer-3234-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 984533 INFO  (jetty-closer-3234-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@41e5d385: rootName = solr_34131, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@7ed3df1
   [junit4]   2> 984537 INFO  (jetty-closer-3234-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 984537 INFO  (jetty-closer-3234-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@330a252b: rootName = solr_34131, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@4cacfe20
   [junit4]   2> 984538 INFO  (jetty-closer-3234-thread-3) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 984538 INFO  (jetty-closer-3234-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 984538 INFO  (jetty-closer-3234-thread-3) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3441f910: rootName = solr_36467, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@4cacfe20
   [junit4]   2> 984538 INFO  (jetty-closer-3234-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@683a9f4c: rootName = solr_37257, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@4cacfe20
   [junit4]   2> 984540 INFO  (jetty-closer-3234-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 984540 INFO  (jetty-closer-3234-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7d44689c: rootName = solr_34131, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@7add2619
   [junit4]   2> 984541 INFO  (jetty-closer-3234-thread-1) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:34131_solr
   [junit4]   2> 984541 INFO  (zkCallback-3290-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 984541 INFO  (zkCallback-3301-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 984541 INFO  (zkCallback-3284-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 984541 INFO  (zkCallback-3274-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 984541 INFO  (zkCallback-3259-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 984541 INFO  (jetty-closer-3234-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 984542 INFO  (jetty-closer-3234-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 984542 INFO  (zkCallback-3251-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 984542 INFO  (jetty-closer-3234-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 984542 INFO  (jetty-closer-3234-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6af5228f: rootName = solr_37257, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@7add2619
   [junit4]   2> 984543 INFO  (jetty-closer-3234-thread-3) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 984543 INFO  (jetty-closer-3234-thread-3) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@515d8acf: rootName = solr_36467, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@7add2619
   [junit4]   2> 984543 INFO  (jetty-closer-3234-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@9f5252c{/solr,null,UNAVAILABLE}
   [junit4]   2> 984543 INFO  (jetty-closer-3234-thread-1) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 984544 INFO  (jetty-closer-3234-thread-2) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:37257_solr
   [junit4]   2> 984544 INFO  (jetty-closer-3234-thread-3) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:36467_solr
   [junit4]   2> 984544 INFO  (jetty-closer-3234-thread-3) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 984544 INFO  (zkCallback-3290-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
   [junit4]   2> 984544 INFO  (zkCallback-3274-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
   [junit4]   2> 984544 INFO  (jetty-closer-3234-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 984544 INFO  (zkCallback-3259-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
   [junit4]   2> 984545 INFO  (zkCallback-3301-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
   [junit4]   2> 984545 INFO  (jetty-closer-3234-thread-2) [    ] o.a.s.c.Overseer Overseer (id=72220333759856649-127.0.0.1:37257_solr-n_0000000000) closing
   [junit4]   2> 984545 WARN  (OverseerAutoScalingTriggerThread-72220333759856649-127.0.0.1:37257_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 984545 DEBUG (jetty-closer-3234-thread-2) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 984545 DEBUG (jetty-closer-3234-thread-2) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 984545 INFO  (OverseerStateUpdate-72220333759856649-127.0.0.1:37257_solr-n_0000000000) [n:127.0.0.1:37257_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37257_solr
   [junit4]   2> 984545 DEBUG (jetty-closer-3234-thread-2) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 984545 DEBUG (jetty-closer-3234-thread-2) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 984545 DEBUG (jetty-closer-3234-thread-2) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 984545 DEBUG (jetty-closer-3234-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 984547 INFO  (jetty-closer-3234-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7aa81e71{/solr,null,UNAVAILABLE}
   [junit4]   2> 984547 INFO  (jetty-closer-3234-thread-2) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 986046 WARN  (zkCallback-3290-thread-1) [    ] o.a.s.c.ZkController Unable to register nodeLost path for 127.0.0.1:37257_solr
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling/nodeLost/127.0.0.1:37257_solr
   [junit4]   2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) ~[zookeeper-3.4.11.jar:3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0]
   [junit4]   2

[...truncated too long message...]

ompile-lucene-core:

init:

-clover.disable:

-clover.load:

-clover.classpath:

-clover.setup:

clover:

compile-core:

-clover.disable:

-clover.load:

-clover.classpath:

-clover.setup:

clover:

common.compile-core:

compile-core:

common.compile-test:
    [mkdir] Created dir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/classes/test
    [javac] Compiling 932 source files to /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/classes/test
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Note: Some input files use unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.
    [javac] Creating empty /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/classes/test/org/apache/solr/cloud/autoscaling/sim/package-info.class
     [copy] Copying 1 file to /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/classes/test

common-solr.compile-test:

compile-test:

BUILD SUCCESSFUL
Total time: 47 seconds
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.NodeLostTriggerTest" -Dtests.showOutput=onerror "-Dargs=-XX:+UseCompressedOops -XX:+UseSerialGC" -Dtests.seed=39F6F71B76BC849D -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en-US -Dtests.timezone=US/Eastern -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
Buildfile: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/build.xml

-clover.disable:

ivy-configure:
[ivy:configure] :: Apache Ivy 2.4.0 - 20141213170938 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = /home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/top-level-ivy-settings.xml

install-junit4-taskdef:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

resolve-groovy:
[ivy:cachepath] :: resolving dependencies :: org.codehaus.groovy#groovy-all-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.codehaus.groovy#groovy-all;2.4.15 in public
[ivy:cachepath] :: resolution report :: resolve 17ms :: artifacts dl 0ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   1   |   0   |   0   |   0   ||   1   |   0   |
        ---------------------------------------------------------------------

-init-totals:

-test:
    [mkdir] Created dir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test
[junit4:pickseed] Seed property 'tests.seed' already defined: 39F6F71B76BC849D
    [mkdir] Created dir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/temp
   [junit4] <JUnit4> says g'day! Master seed: 39F6F71B76BC849D
   [junit4] Executing 5 suites with 3 JVMs.
   [junit4]
   [junit4] Started J2 PID([hidden email]).
   [junit4] Started J1 PID([hidden email]).
   [junit4] Started J0 PID([hidden email]).
   [junit4] Suite: org.apache.solr.cloud.autoscaling.NodeLostTriggerTest
   [junit4] OK      0.19s J0 | NodeLostTriggerTest.testActionLifecycle
   [junit4] OK      0.62s J0 | NodeLostTriggerTest.testListenerAcceptance
   [junit4] OK      10.3s J0 | NodeLostTriggerTest.testTrigger
   [junit4] OK      5.23s J0 | NodeLostTriggerTest.testRestoreState
   [junit4] Completed [1/5] on J0 in 23.37s, 4 tests
   [junit4]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.NodeLostTriggerTest
   [junit4] OK      0.08s J1 | NodeLostTriggerTest.testActionLifecycle
   [junit4] OK      0.64s J1 | NodeLostTriggerTest.testListenerAcceptance
   [junit4] OK      10.5s J1 | NodeLostTriggerTest.testTrigger
   [junit4] OK      5.29s J1 | NodeLostTriggerTest.testRestoreState
   [junit4] Completed [2/5] on J1 in 24.87s, 4 tests
   [junit4]
   [junit4] Duplicate suite name used with XML reports: org.apache.solr.cloud.autoscaling.NodeLostTriggerTest. This may confuse tools that process XML reports. Set 'ignoreDuplicateSuites' to true to skip this message.
   [junit4] Suite: org.apache.solr.cloud.autoscaling.NodeLostTriggerTest
   [junit4] OK      0.16s J2 | NodeLostTriggerTest.testActionLifecycle
   [junit4] OK      1.06s J2 | NodeLostTriggerTest.testListenerAcceptance
   [junit4] OK      10.3s J2 | NodeLostTriggerTest.testTrigger
   [junit4] OK      5.24s J2 | NodeLostTriggerTest.testRestoreState
   [junit4] Completed [3/5] on J2 in 25.11s, 4 tests
   [junit4]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.NodeLostTriggerTest
   [junit4] OK      0.03s J1 | NodeLostTriggerTest.testActionLifecycle
   [junit4] OK      0.37s J1 | NodeLostTriggerTest.testListenerAcceptance
   [junit4] OK      10.2s J1 | NodeLostTriggerTest.testTrigger
   [junit4] OK      5.26s J1 | NodeLostTriggerTest.testRestoreState
   [junit4] Completed [4/5] on J1 in 17.01s, 4 tests
   [junit4]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.NodeLostTriggerTest
   [junit4] OK      0.03s J0 | NodeLostTriggerTest.testActionLifecycle
   [junit4] OK      0.25s J0 | NodeLostTriggerTest.testListenerAcceptance
   [junit4] OK      10.3s J0 | NodeLostTriggerTest.testTrigger
   [junit4] OK      5.19s J0 | NodeLostTriggerTest.testRestoreState
   [junit4] Completed [5/5] on J0 in 18.66s, 4 tests
   [junit4]
   [junit4] JVM J0:     0.41 ..    43.31 =    42.90s
   [junit4] JVM J1:     0.40 ..    42.99 =    42.60s
   [junit4] JVM J2:     0.36 ..    26.17 =    25.81s
   [junit4] Execution time total: 43 seconds
   [junit4] Tests summary: 5 suites, 20 tests
   [junit4] Could not remove temporary path: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0 (java.nio.file.DirectoryNotEmptyException: Remaining files: [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp])
   [junit4] Could not remove temporary path: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1 (java.nio.file.DirectoryNotEmptyException: Remaining files: [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp])
   [junit4] Could not remove temporary path: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2 (java.nio.file.DirectoryNotEmptyException: Remaining files: [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp])
     [echo] 5 slowest tests:
[junit4:tophints] 1230.89s | org.apache.solr.cloud.BasicDistributedZkTest
[junit4:tophints] 1060.59s | org.apache.solr.cloud.cdcr.CdcrOpsAndBoundariesTest
[junit4:tophints] 992.13s | org.apache.solr.cloud.cdcr.CdcrWithNodesRestartsTest
[junit4:tophints] 499.19s | org.apache.solr.cloud.api.collections.ShardSplitTest
[junit4:tophints] 212.52s | org.apache.solr.update.TestInPlaceUpdatesDistrib

-check-totals:

test-nocompile:

BUILD SUCCESSFUL
Total time: 45 seconds
[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.autoscaling.NodeLostTriggerTest
[repro] Exiting with code 0
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
java.lang.InterruptedException: no matches found within 10000
        at hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:2845)
        at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2724)
        at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2705)
        at hudson.FilePath.act(FilePath.java:1076)
        at hudson.FilePath.act(FilePath.java:1059)
        at hudson.FilePath.validateAntFileMask(FilePath.java:2703)
        at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
        at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
        at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
        at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
        at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
        at hudson.model.Build$BuildExecution.post2(Build.java:186)
        at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
        at hudson.model.Run.execute(Run.java:1840)
        at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
        at hudson.model.ResourceController.execute(ResourceController.java:97)
        at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/*_pid*.log". Configuration error?
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=d6641ffb1d5fd5c9d3743561e2c5158951317713, workspace=/var/lib/jenkins/workspace/Lucene-Solr-master-Linux
[WARNINGS] Computing warning deltas based on reference build #22853
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-11-ea+28) - Build # 22855 - Still Unstable!

Policeman Jenkins Server-2
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/22855/
Java: 64bit/jdk-11-ea+28 -XX:+UseCompressedOops -XX:+UseSerialGC

29 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DocValuesNotIndexedTest

Error Message:
Collection not found: dv_coll

Stack Trace:
org.apache.solr.common.SolrException: Collection not found: dv_coll
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7]:0)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:851)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.cloud.DocValuesNotIndexedTest.createCluster(DocValuesNotIndexedTest.java:155)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DocValuesNotIndexedTest

Error Message:
Collection not found: dv_coll

Stack Trace:
org.apache.solr.common.SolrException: Collection not found: dv_coll
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7]:0)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:851)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.cloud.DocValuesNotIndexedTest.createCluster(DocValuesNotIndexedTest.java:155)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DocValuesNotIndexedTest

Error Message:
Collection not found: dv_coll

Stack Trace:
org.apache.solr.common.SolrException: Collection not found: dv_coll
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7]:0)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:851)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.cloud.DocValuesNotIndexedTest.createCluster(DocValuesNotIndexedTest.java:155)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DocValuesNotIndexedTest

Error Message:
Collection not found: dv_coll

Stack Trace:
org.apache.solr.common.SolrException: Collection not found: dv_coll
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7]:0)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:851)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.cloud.DocValuesNotIndexedTest.createCluster(DocValuesNotIndexedTest.java:155)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestSQLHandler

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.handler.TestSQLHandler:     1) Thread[id=3318, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestSQLHandler]         at java.base@11/java.lang.Thread.sleep(Native Method)         at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.base@11/java.lang.Thread.run(Thread.java:834)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestSQLHandler:
   1) Thread[id=3318, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestSQLHandler]
        at java.base@11/java.lang.Thread.sleep(Native Method)
        at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestSQLHandler

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.handler.TestSQLHandler:     1) Thread[id=3330, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestSQLHandler]         at java.base@11/java.lang.Thread.sleep(Native Method)         at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.base@11/java.lang.Thread.run(Thread.java:834)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestSQLHandler:
   1) Thread[id=3330, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestSQLHandler]
        at java.base@11/java.lang.Thread.sleep(Native Method)
        at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestSQLHandler

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.handler.TestSQLHandler:     1) Thread[id=5586, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestSQLHandler]         at java.base@11/java.lang.Thread.sleep(Native Method)         at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.base@11/java.lang.Thread.run(Thread.java:834)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestSQLHandler:
   1) Thread[id=5586, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestSQLHandler]
        at java.base@11/java.lang.Thread.sleep(Native Method)
        at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestSQLHandler

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.handler.TestSQLHandler:     1) Thread[id=16535, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestSQLHandler]         at java.base@11/java.lang.Thread.sleep(Native Method)         at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.base@11/java.lang.Thread.run(Thread.java:834)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestSQLHandler:
   1) Thread[id=16535, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestSQLHandler]
        at java.base@11/java.lang.Thread.sleep(Native Method)
        at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.base@11/java.lang.Thread.run(Thread.java:834)
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7]:0)


FAILED:  org.apache.solr.cloud.TestPullReplica.testCreateDelete {seed=[BD01A5BE9CE544F7:A61154F26A15BCC7]}

Error Message:


Stack Trace:
java.lang.NullPointerException
        at org.apache.solr.cloud.TestPullReplica.testCreateDelete(TestPullReplica.java:174)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.TestPullReplica.testCreateDelete {seed=[BD01A5BE9CE544F7:1247E80E5ED777EB]}

Error Message:
Error from server at https://127.0.0.1:45781/solr: delete the collection time out:180s

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:45781/solr: delete the collection time out:180s
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.TestPullReplica.tearDown(TestPullReplica.java:116)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:993)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.TestPullReplica.testRealTimeGet

Error Message:
Unexpected replica count null Live Nodes: [127.0.0.1:34649_solr, 127.0.0.1:45781_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Unexpected replica count
null
Live Nodes: [127.0.0.1:34649_solr, 127.0.0.1:45781_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7:E56C50BD3E5FEA3E]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.TestPullReplica.testRealTimeGet(TestPullReplica.java:348)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.TestPullReplica.testRealTimeGet

Error Message:
Unexpected replica count null Live Nodes: [127.0.0.1:40817_solr, 127.0.0.1:40939_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Unexpected replica count
null
Live Nodes: [127.0.0.1:40817_solr, 127.0.0.1:40939_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7:E56C50BD3E5FEA3E]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.TestPullReplica.testRealTimeGet(TestPullReplica.java:348)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.TestSkipOverseerOperations.testSkipDownOperations

Error Message:
Expected 2x2 for collection: collection2 null Live Nodes: [127.0.0.1:37289_solr, 127.0.0.1:40451_solr, 127.0.0.1:46175_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Expected 2x2 for collection: collection2
null
Live Nodes: [127.0.0.1:37289_solr, 127.0.0.1:40451_solr, 127.0.0.1:46175_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7:7A93D7F861A9FF21]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.TestSkipOverseerOperations.testSkipDownOperations(TestSkipOverseerOperations.java:104)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.TestSkipOverseerOperations.testSkipDownOperations

Error Message:
Expected 2x2 for collection: collection2 null Live Nodes: [127.0.0.1:34717_solr, 127.0.0.1:39137_solr, 127.0.0.1:44151_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Expected 2x2 for collection: collection2
null
Live Nodes: [127.0.0.1:34717_solr, 127.0.0.1:39137_solr, 127.0.0.1:44151_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7:7A93D7F861A9FF21]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.TestSkipOverseerOperations.testSkipDownOperations(TestSkipOverseerOperations.java:104)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.TestSkipOverseerOperations.testSkipDownOperations

Error Message:
Expected 2x2 for collection: collection2 null Live Nodes: [127.0.0.1:34661_solr, 127.0.0.1:35273_solr, 127.0.0.1:42527_solr] Last available state: null

Stack Trace:
java.lang.AssertionError: Expected 2x2 for collection: collection2
null
Live Nodes: [127.0.0.1:34661_solr, 127.0.0.1:35273_solr, 127.0.0.1:42527_solr]
Last available state: null
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7:7A93D7F861A9FF21]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:280)
        at org.apache.solr.cloud.TestSkipOverseerOperations.testSkipDownOperations(TestSkipOverseerOperations.java:104)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest.testAsyncIdRaceCondition

Error Message:
expected:<9> but was:<8>

Stack Trace:
java.lang.AssertionError: expected:<9> but was:<8>
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7:E88CC853F08BBD5C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at org.apache.solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest.testAsyncIdRaceCondition(CollectionsAPIAsyncDistributedZkTest.java:243)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest.testAsyncIdRaceCondition

Error Message:
expected:<9> but was:<8>

Stack Trace:
java.lang.AssertionError: expected:<9> but was:<8>
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7:E88CC853F08BBD5C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at org.apache.solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest.testAsyncIdRaceCondition(CollectionsAPIAsyncDistributedZkTest.java:243)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.cdcr.CdcrBidirectionalTest.testBiDir

Error Message:
Collection not found: cdcr-cluster2

Stack Trace:
org.apache.solr.common.SolrException: Collection not found: cdcr-cluster2
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7:F8DA555C84CB08B5]:0)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:851)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:484)
        at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:463)
        at org.apache.solr.cloud.cdcr.CdcrTestsUtil.waitForClusterToSync(CdcrTestsUtil.java:125)
        at org.apache.solr.cloud.cdcr.CdcrTestsUtil.waitForClusterToSync(CdcrTestsUtil.java:118)
        at org.apache.solr.cloud.cdcr.CdcrBidirectionalTest.testBiDir(CdcrBidirectionalTest.java:100)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.cdcr.CdcrBidirectionalTest.testBiDir

Error Message:
Collection not found: cdcr-cluster1

Stack Trace:
org.apache.solr.common.SolrException: Collection not found: cdcr-cluster1
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7:F8DA555C84CB08B5]:0)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:851)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:957)
        at org.apache.solr.cloud.cdcr.CdcrTestsUtil.invokeCdcrAction(CdcrTestsUtil.java:83)
        at org.apache.solr.cloud.cdcr.CdcrTestsUtil.cdcrStart(CdcrTestsUtil.java:60)
        at org.apache.solr.cloud.cdcr.CdcrBidirectionalTest.testBiDir(CdcrBidirectionalTest.java:78)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.cdcr.CdcrBidirectionalTest.testBiDir

Error Message:
Captured an uncaught exception in thread: Thread[id=557, name=cdcr-replicator-165-thread-1, state=RUNNABLE, group=TGRP-CdcrBidirectionalTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=557, name=cdcr-replicator-165-thread-1, state=RUNNABLE, group=TGRP-CdcrBidirectionalTest]
Caused by: java.lang.AssertionError: 1611576312818827264 != 1611576312815681536
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7]:0)
        at org.apache.solr.update.CdcrUpdateLog$CdcrLogReader.forwardSeek(CdcrUpdateLog.java:611)
        at org.apache.solr.handler.CdcrReplicator.run(CdcrReplicator.java:125)
        at org.apache.solr.handler.CdcrReplicatorScheduler.lambda$start$0(CdcrReplicatorScheduler.java:81)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.cdcr.CdcrBidirectionalTest.testBiDir

Error Message:
Collection not found: cdcr-cluster1

Stack Trace:
org.apache.solr.common.SolrException: Collection not found: cdcr-cluster1
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7:F8DA555C84CB08B5]:0)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:851)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
        at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:957)
        at org.apache.solr.cloud.cdcr.CdcrTestsUtil.invokeCdcrAction(CdcrTestsUtil.java:83)
        at org.apache.solr.cloud.cdcr.CdcrTestsUtil.cdcrStart(CdcrTestsUtil.java:60)
        at org.apache.solr.cloud.cdcr.CdcrBidirectionalTest.testBiDir(CdcrBidirectionalTest.java:78)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.cloud.cdcr.CdcrBidirectionalTest.testBiDir

Error Message:
Collection not found: cdcr-cluster2

Stack Trace:
org.apache.solr.common.SolrException: Collection not found: cdcr-cluster2
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7:F8DA555C84CB08B5]:0)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:851)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:484)
        at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:463)
        at org.apache.solr.cloud.cdcr.CdcrTestsUtil.waitForClusterToSync(CdcrTestsUtil.java:125)
        at org.apache.solr.cloud.cdcr.CdcrTestsUtil.waitForClusterToSync(CdcrTestsUtil.java:118)
        at org.apache.solr.cloud.cdcr.CdcrBidirectionalTest.testBiDir(CdcrBidirectionalTest.java:100)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.handler.TestSQLHandler.doTest

Error Message:
--> https://127.0.0.1:45671/collection1_shard2_replica_n41:Failed to execute sqlQuery 'select id, field_i, str_s, field_i_p, field_f_p, field_d_p, field_l_p from collection1 where (text='(XXXX)' OR text='XXXX') AND text='XXXX' order by field_i desc' against JDBC connection 'jdbc:calcitesolr:'. Error while executing SQL "select id, field_i, str_s, field_i_p, field_f_p, field_d_p, field_l_p from collection1 where (text='(XXXX)' OR text='XXXX') AND text='XXXX' order by field_i desc": java.io.IOException: java.util.concurrent.ExecutionException: java.io.IOException: --> https://127.0.0.1:45999/collection1_shard2_replica_n45/:id must have DocValues to use this feature.

Stack Trace:
java.io.IOException: --> https://127.0.0.1:45671/collection1_shard2_replica_n41:Failed to execute sqlQuery 'select id, field_i, str_s, field_i_p, field_f_p, field_d_p, field_l_p from collection1 where (text='(XXXX)' OR text='XXXX') AND text='XXXX' order by field_i desc' against JDBC connection 'jdbc:calcitesolr:'.
Error while executing SQL "select id, field_i, str_s, field_i_p, field_f_p, field_d_p, field_l_p from collection1 where (text='(XXXX)' OR text='XXXX') AND text='XXXX' order by field_i desc": java.io.IOException: java.util.concurrent.ExecutionException: java.io.IOException: --> https://127.0.0.1:45999/collection1_shard2_replica_n45/:id must have DocValues to use this feature.
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7:1A451D1AF15E574E]:0)
        at org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:215)
        at org.apache.solr.handler.TestSQLHandler.getTuples(TestSQLHandler.java:2712)
        at org.apache.solr.handler.TestSQLHandler.testBasicSelect(TestSQLHandler.java:137)
        at org.apache.solr.handler.TestSQLHandler.doTest(TestSQLHandler.java:85)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.handler.TestSQLHandler.doTest

Error Message:
--> https://127.0.0.1:36671/collection1_shard2_replica_n41:Failed to execute sqlQuery 'select id, field_i, str_s, field_i_p, field_f_p, field_d_p, field_l_p from collection1 where (text='(XXXX)' OR text='XXXX') AND text='XXXX' order by field_i desc' against JDBC connection 'jdbc:calcitesolr:'. Error while executing SQL "select id, field_i, str_s, field_i_p, field_f_p, field_d_p, field_l_p from collection1 where (text='(XXXX)' OR text='XXXX') AND text='XXXX' order by field_i desc": java.io.IOException: java.util.concurrent.ExecutionException: java.io.IOException: --> https://127.0.0.1:36671/collection1_shard2_replica_n41/:id must have DocValues to use this feature.

Stack Trace:
java.io.IOException: --> https://127.0.0.1:36671/collection1_shard2_replica_n41:Failed to execute sqlQuery 'select id, field_i, str_s, field_i_p, field_f_p, field_d_p, field_l_p from collection1 where (text='(XXXX)' OR text='XXXX') AND text='XXXX' order by field_i desc' against JDBC connection 'jdbc:calcitesolr:'.
Error while executing SQL "select id, field_i, str_s, field_i_p, field_f_p, field_d_p, field_l_p from collection1 where (text='(XXXX)' OR text='XXXX') AND text='XXXX' order by field_i desc": java.io.IOException: java.util.concurrent.ExecutionException: java.io.IOException: --> https://127.0.0.1:36671/collection1_shard2_replica_n41/:id must have DocValues to use this feature.
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7:1A451D1AF15E574E]:0)
        at org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:215)
        at org.apache.solr.handler.TestSQLHandler.getTuples(TestSQLHandler.java:2712)
        at org.apache.solr.handler.TestSQLHandler.testBasicSelect(TestSQLHandler.java:137)
        at org.apache.solr.handler.TestSQLHandler.doTest(TestSQLHandler.java:85)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)


FAILED:  org.apache.solr.handler.TestSQLHandler.doTest

Error Message:
--> https://127.0.0.1:44521/collection1_shard2_replica_n41:Failed to execute sqlQuery 'select id, field_i, str_s, field_i_p, field_f_p, field_d_p, field_l_p from collection1 where (text='(XXXX)' OR text='XXXX') AND text='XXXX' order by field_i desc' against JDBC connection 'jdbc:calcitesolr:'. Error while executing SQL "select id, field_i, str_s, field_i_p, field_f_p, field_d_p, field_l_p from collection1 where (text='(XXXX)' OR text='XXXX') AND text='XXXX' order by field_i desc": java.io.IOException: java.util.concurrent.ExecutionException: java.io.IOException: --> https://127.0.0.1:43425/collection1_shard2_replica_n45/:id must have DocValues to use this feature.

Stack Trace:
java.io.IOException: --> https://127.0.0.1:44521/collection1_shard2_replica_n41:Failed to execute sqlQuery 'select id, field_i, str_s, field_i_p, field_f_p, field_d_p, field_l_p from collection1 where (text='(XXXX)' OR text='XXXX') AND text='XXXX' order by field_i desc' against JDBC connection 'jdbc:calcitesolr:'.
Error while executing SQL "select id, field_i, str_s, field_i_p, field_f_p, field_d_p, field_l_p from collection1 where (text='(XXXX)' OR text='XXXX') AND text='XXXX' order by field_i desc": java.io.IOException: java.util.concurrent.ExecutionException: java.io.IOException: --> https://127.0.0.1:43425/collection1_shard2_replica_n45/:id must have DocValues to use this feature.
        at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7:1A451D1AF15E574E]:0)
        at org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:215)
        at org.apache.solr.handler.TestSQLHandler.getTuples(TestSQLHandler.java:2712)
        at org.apache.solr.handler.TestSQLHandler.testBasicSelect(TestSQLHandler.java:137)
        at org.apache.solr.handler.TestSQLHandler.doTest(TestSQLHandler.java:85)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
        at com.carrotsearch.randomizedtesting.Randomi

[...truncated too long message...]

Core org.apache.solr.core.SolrCore@3a204fbc
   [junit4]   2> 1014601 INFO  (coreCloseExecutor-674-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=3a204fbc
   [junit4]   2> 1014601 INFO  (coreCloseExecutor-674-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@d2e4725: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@7beb0083
   [junit4]   2> 1014609 INFO  (TEST-DistributedMLTComponentTest.test-seed#[BD01A5BE9CE544F7]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@383d7e57{/,null,UNAVAILABLE}
   [junit4]   2> 1014609 INFO  (TEST-DistributedMLTComponentTest.test-seed#[BD01A5BE9CE544F7]) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1014610 INFO  (TEST-DistributedMLTComponentTest.test-seed#[BD01A5BE9CE544F7]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@48ccfe6b{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1014610 INFO  (TEST-DistributedMLTComponentTest.test-seed#[BD01A5BE9CE544F7]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=384313754
   [junit4]   2> 1014611 INFO  (TEST-DistributedMLTComponentTest.test-seed#[BD01A5BE9CE544F7]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1014611 INFO  (TEST-DistributedMLTComponentTest.test-seed#[BD01A5BE9CE544F7]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6550ff0d: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@647c2e86
   [junit4]   2> 1014615 INFO  (TEST-DistributedMLTComponentTest.test-seed#[BD01A5BE9CE544F7]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1014615 INFO  (TEST-DistributedMLTComponentTest.test-seed#[BD01A5BE9CE544F7]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@18396c9e: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@4750d3ee
   [junit4]   2> 1014618 INFO  (TEST-DistributedMLTComponentTest.test-seed#[BD01A5BE9CE544F7]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1014618 INFO  (TEST-DistributedMLTComponentTest.test-seed#[BD01A5BE9CE544F7]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7c8d2433: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@5f1d818a
   [junit4]   2> 1014619 INFO  (coreCloseExecutor-676-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@58a51a08
   [junit4]   2> 1014619 INFO  (coreCloseExecutor-676-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=58a51a08
   [junit4]   2> 1014619 INFO  (coreCloseExecutor-676-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@514b6f1f: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@6952c248
   [junit4]   2> 1014630 INFO  (TEST-DistributedMLTComponentTest.test-seed#[BD01A5BE9CE544F7]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2b41fde1{/,null,UNAVAILABLE}
   [junit4]   2> 1014630 INFO  (TEST-DistributedMLTComponentTest.test-seed#[BD01A5BE9CE544F7]) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=DistributedMLTComponentTest -Dtests.method=test -Dtests.seed=BD01A5BE9CE544F7 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=asa -Dtests.timezone=Asia/Thimphu -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   14.4s J0 | DistributedMLTComponentTest.test <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:37035//collection1: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://[ff01::114]:33332, https://[ff01::083]:33332, https://[ff01::213]:33332, https://127.0.0.1:37035//collection1]
   [junit4]    > at __randomizedtesting.SeedInfo.seed([BD01A5BE9CE544F7:35559A643219290F]:0)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
   [junit4]    > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
   [junit4]    > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
   [junit4]    > at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
   [junit4]    > at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:957)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase.queryServer(BaseDistributedSearchTestCase.java:559)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:611)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:589)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:568)
   [junit4]    > at org.apache.solr.handler.component.DistributedMLTComponentTest.test(DistributedMLTComponentTest.java:148)
   [junit4]    > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    > at java.base/java.lang.reflect.Method.invoke(Method.java:566)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
   [junit4]    > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
   [junit4]    > at java.base/java.lang.Thread.run(Thread.java:834)
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.handler.component.DistributedMLTComponentTest_BD01A5BE9CE544F7-001
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@3ee7d4d5), locale=asa, timezone=Asia/Thimphu
   [junit4]   2> NOTE: Linux 4.15.0-32-generic amd64/Oracle Corporation 11 (64-bit)/cpus=8,threads=1,free=277263632,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [CdcrBidirectionalTest, TestSQLHandler, TestSkipOverseerOperations, TestSQLHandler, TestPullReplica, DistributedMLTComponentTest]
   [junit4] Completed [35/35 (17!)] on J0 in 14.41s, 1 test, 1 error <<< FAILURES!

[...truncated 20 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1567: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1092: There were test failures: 35 suites, 103 tests, 6 suite-level errors, 12 errors, 4 failures, 5 ignored [seed: BD01A5BE9CE544F7]

Total time: 16 minutes 56 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   1/5 failed: org.apache.solr.cloud.TestPullReplica
[repro]   1/5 failed: org.apache.solr.cloud.api.collections.CollectionsAPIAsyncDistributedZkTest
[repro]   2/5 failed: org.apache.solr.cloud.TestSkipOverseerOperations
[repro]   2/5 failed: org.apache.solr.handler.component.DistributedMLTComponentTest
[repro]   3/5 failed: org.apache.solr.cloud.DocValuesNotIndexedTest
[repro]   3/5 failed: org.apache.solr.handler.TestSQLHandler
[repro]   5/5 failed: org.apache.solr.cloud.cdcr.CdcrBidirectionalTest
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=a7a4f68da5e3aa53206eba6b52512f73b6d41394, workspace=/var/lib/jenkins/workspace/Lucene-Solr-master-Linux
[WARNINGS] Computing warning deltas based on reference build #22854
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]