[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 78 - Unstable

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 78 - Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/78/

8 tests failed.
FAILED:  org.apache.solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete

Error Message:
Error from server at http://127.0.0.1:54712/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title> </head> <body><h2>HTTP ERROR 404</h2> <p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason: <pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.10.v20180503</a><hr/>  </body> </html>

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:54712/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.10.v20180503</a><hr/>

</body>
</html>

        at __randomizedtesting.SeedInfo.seed([B4380E3F99975242:17C2A09A1E7FB8E7]:0)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1015)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
        at org.apache.solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete(TestCollectionsAPIViaSolrCloudCluster.java:170)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:54712/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre>    Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.10.v20180503</a><hr/>

</body>
</html>

        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:528)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        ... 1 more


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger

Error Message:
number of ops expected:<2> but was:<1>

Stack Trace:
java.lang.AssertionError: number of ops expected:<2> but was:<1>
        at __randomizedtesting.SeedInfo.seed([B4380E3F99975242:D7F338BD0058216F]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:187)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration

Error Message:
did not finish processing in time

Stack Trace:
java.lang.AssertionError: did not finish processing in time
        at __randomizedtesting.SeedInfo.seed([B4380E3F99975242:E7814C8F7B86C7B8]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration(IndexSizeTriggerTest.java:425)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeAddedTrigger

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
        at __randomizedtesting.SeedInfo.seed([B4380E3F99975242:9252B0068298FD2E]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeAddedTrigger(TestTriggerIntegration.java:410)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
        at __randomizedtesting.SeedInfo.seed([B4380E3F99975242:E97010B65651F40D]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1201)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.update.AddBlockUpdateTest.testSolrNestedFieldsSingleVal

Error Message:


Stack Trace:
java.lang.NullPointerException
        at __randomizedtesting.SeedInfo.seed([B4380E3F99975242:7C7765A7DD43D7B0]:0)
        at org.apache.solr.update.AddBlockUpdateTest.getNewClock(AddBlockUpdateTest.java:671)
        at org.apache.solr.update.AddBlockUpdateTest.indexSolrInputDocumentsDirectly(AddBlockUpdateTest.java:661)
        at org.apache.solr.update.AddBlockUpdateTest.testSolrNestedFieldsSingleVal(AddBlockUpdateTest.java:338)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.update.AddBlockUpdateTest.testExceptionThrownChildDocWAnonymousChildren

Error Message:
 Expected: (exception with message a string containing "Anonymous child docs can only hang from others or the root" and an instance of org.apache.solr.common.SolrException)      got: <java.lang.NullPointerException>

Stack Trace:
java.lang.AssertionError:
Expected: (exception with message a string containing "Anonymous child docs can only hang from others or the root" and an instance of org.apache.solr.common.SolrException)
     got: <java.lang.NullPointerException>

        at __randomizedtesting.SeedInfo.seed([B4380E3F99975242:9C887C38F4793E01]:0)
        at org.junit.Assert.assertThat(Assert.java:780)
        at org.junit.Assert.assertThat(Assert.java:738)
        at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:114)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.update.AddBlockUpdateTest.testSolrNestedFieldsList

Error Message:


Stack Trace:
java.lang.NullPointerException
        at __randomizedtesting.SeedInfo.seed([B4380E3F99975242:6CC623E739086C1D]:0)
        at org.apache.solr.update.AddBlockUpdateTest.getNewClock(AddBlockUpdateTest.java:671)
        at org.apache.solr.update.AddBlockUpdateTest.indexSolrInputDocumentsDirectly(AddBlockUpdateTest.java:661)
        at org.apache.solr.update.AddBlockUpdateTest.testSolrNestedFieldsList(AddBlockUpdateTest.java:305)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13067 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
   [junit4]   2> 1066050 INFO  (SUITE-TestTriggerIntegration-seed#[B4380E3F99975242]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_B4380E3F99975242-001/init-core-data-001
   [junit4]   2> 1066052 WARN  (SUITE-TestTriggerIntegration-seed#[B4380E3F99975242]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=3 numCloses=3
   [junit4]   2> 1066052 INFO  (SUITE-TestTriggerIntegration-seed#[B4380E3F99975242]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1066054 INFO  (SUITE-TestTriggerIntegration-seed#[B4380E3F99975242]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1066337 DEBUG (SUITE-TestTriggerIntegration-seed#[B4380E3F99975242]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
   [junit4]   2> 1066371 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1066371 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 1066372 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 1066372 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 1066389 INFO  (SUITE-TestTriggerIntegration-seed#[B4380E3F99975242]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1066401 INFO  (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B4380E3F99975242]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTrigger
   [junit4]   2> 1066402 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 1066404 INFO  (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1066425 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 1066438 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1066438 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1066438 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1066438 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1066447 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1066447 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
   [junit4]   2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 1066447 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1066448 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1066452 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1066452 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 1066465 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 1066465 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 1066471 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 1066481 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1066481 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 1066481 DEBUG (ScheduledTrigger-3352-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1066501 DEBUG (ScheduledTrigger-3352-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1066522 DEBUG (ScheduledTrigger-3352-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1066542 DEBUG (ScheduledTrigger-3352-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1066562 DEBUG (ScheduledTrigger-3352-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1066582 DEBUG (ScheduledTrigger-3352-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1066602 DEBUG (simCloudManagerPool-3351-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1066604 DEBUG (ScheduledTrigger-3352-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1066606 DEBUG (simCloudManagerPool-3351-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 1066615 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 1066615 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1066615 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 1066624 DEBUG (ScheduledTrigger-3352-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1066636 DEBUG (simCloudManagerPool-3351-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1066636 DEBUG (simCloudManagerPool-3351-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 1066636 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 1066637 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 1066637 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 1066637 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=2, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
   [junit4]   2> 1066640 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 1066645 DEBUG (ScheduledTrigger-3352-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1066640 DEBUG (ScheduledTrigger-3352-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1066817 DEBUG (ScheduledTrigger-3352-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1066818 DEBUG (ScheduledTrigger-3352-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10002_solr at time 29329014529112378
   [junit4]   2> 1066834 DEBUG (ScheduledTrigger-3352-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1066845 DEBUG (ScheduledTrigger-3352-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1066845 DEBUG (ScheduledTrigger-3352-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1066855 DEBUG (ScheduledTrigger-3352-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1066865 DEBUG (ScheduledTrigger-3352-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1066865 DEBUG (ScheduledTrigger-3352-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1066865 DEBUG (ScheduledTrigger-3352-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10002_solr] added at times [29329014529112378], now=29329016911147128
   [junit4]   2> 1066866 DEBUG (ScheduledTrigger-3352-thread-3) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"683295549eb93aT3ghy69gvvol4c1654ytqdronc",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29329014529112378,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29329014529112378],
   [junit4]   2>     "nodeNames":["127.0.0.1:10002_solr"]}}
   [junit4]   2> 1066866 DEBUG (ScheduledTrigger-3352-thread-3) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
   [junit4]   2> 1067095 INFO  (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B4380E3F99975242]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTrigger
   [junit4]   2> 1067107 INFO  (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes: 3
   [junit4]   2> ## Empty nodes: 3
   [junit4]   2> ## Dead nodes: 0
   [junit4]   2> ## Collections: [.system]
   [junit4]   2> ## Max replicas per node: 0
   [junit4]   2> ## Min replicas per node: 0
   [junit4]   2> ## Total replicas: 0
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ## - autoscaling        2
   [junit4]   2> ## - update             1
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2>
   [junit4]   2> 1067108 DEBUG (simCloudManagerPool-3351-thread-3) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1067111 DEBUG (ScheduledTrigger-3352-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10002_solr
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestTriggerIntegration -Dtests.method=testNodeAddedTrigger -Dtests.seed=B4380E3F99975242 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=tr-TR -Dtests.timezone=Africa/Freetown -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.72s J1 | TestTriggerIntegration.testNodeAddedTrigger <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: The trigger did not fire at all
   [junit4]    > at __randomizedtesting.SeedInfo.seed([B4380E3F99975242:9252B0068298FD2E]:0)
   [junit4]    > at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeAddedTrigger(TestTriggerIntegration.java:410)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1067129 INFO  (TEST-TestTriggerIntegration.testSearchRate-seed#[B4380E3F99975242]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSearchRate
   [junit4]   2> 1067129 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 1067129 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10002_solr
   [junit4]   2> 1067129 INFO  (TEST-TestTriggerIntegration.testSearchRate-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1067129 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1067129 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1067130 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1067130 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1067130 DEBUG (AutoscalingActionExecutor-3353-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"683295549eb93aT3ghy69gvvol4c1654ytqdronc",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29329014529112378,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29329014529112378],
   [junit4]   2>     "_enqueue_time_":29329016953659628,
   [junit4]   2>     "nodeNames":["127.0.0.1:10002_solr"]}}
   [junit4]   2> 1067131 WARN  (AutoscalingActionExecutor-3353-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Exception executing actions
   [junit4]   2> org.apache.lucene.store.AlreadyClosedException: ScheduledTrigger node_added_trigger has been closed.
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.dequeue(ScheduledTriggers.java:545) ~[java/:?]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:318) ~[java/:?]
   [junit4]   2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 1067131 DEBUG (AutoscalingActionExecutor-3353-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 52 ms for event id=683295549eb93aT3ghy69gvvol4c1654ytqdronc
   [junit4]   2> 1067131 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1067131 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1067131 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1067137 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1067137 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 1067138 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion -1
   [junit4]   2> 1067138 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 1067139 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 1067158 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1067158 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
   [junit4]   2> 1067159 DEBUG (ScheduledTrigger-3355-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067192 DEBUG (ScheduledTrigger-3355-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067213 DEBUG (ScheduledTrigger-3355-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067233 DEBUG (ScheduledTrigger-3355-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067263 DEBUG (ScheduledTrigger-3355-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067272 DEBUG (simCloudManagerPool-3354-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1067272 DEBUG (simCloudManagerPool-3354-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
   [junit4]   2> 1067273 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
   [junit4]   2> 1067273 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1067273 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
   [junit4]   2> 1067283 DEBUG (ScheduledTrigger-3355-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067303 DEBUG (ScheduledTrigger-3355-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067323 DEBUG (ScheduledTrigger-3355-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067343 DEBUG (ScheduledTrigger-3355-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067364 DEBUG (ScheduledTrigger-3355-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067377 DEBUG (simCloudManagerPool-3354-thread-5) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for collection1 / shard1: {"core_node1":{
   [junit4]   2>     "core":"collection1_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"collection1"}}
   [junit4]   2> 1067384 DEBUG (ScheduledTrigger-3355-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067404 DEBUG (ScheduledTrigger-3355-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067424 DEBUG (ScheduledTrigger-3355-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067429 DEBUG (simCloudManagerPool-3354-thread-6) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1067430 DEBUG (simCloudManagerPool-3354-thread-6) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
   [junit4]   2> 1067499 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
   [junit4]   2> 1067499 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1067499 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
   [junit4]   2> 1067499 DEBUG (ScheduledTrigger-3355-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067506 DEBUG (simCloudManagerPool-3354-thread-7) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1067506 DEBUG (simCloudManagerPool-3354-thread-7) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
   [junit4]   2> 1067507 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
   [junit4]   2> 1067507 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1067507 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
   [junit4]   2> 1067520 DEBUG (ScheduledTrigger-3355-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067540 DEBUG (ScheduledTrigger-3355-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067560 DEBUG (ScheduledTrigger-3355-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067581 DEBUG (ScheduledTrigger-3355-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067601 DEBUG (ScheduledTrigger-3355-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1067618 DEBUG (ScheduledTrigger-3355-thread-3) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"68329d8fa94fbaT3ghy69gvvol4c1654ytqdrone",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime":29329049879400378,
   [junit4]   2>   "eventType":"SEARCHRATE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "hotReplicas":[
   [junit4]   2>       "{\"core_node1\":{\n    \"core\":\"collection1_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}",
   [junit4]   2>       "{\"core_node2\":{\n    \"core\":\"collection1_shard1_replica_n2\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}"],
   [junit4]   2>     "coldNodes":{},
   [junit4]   2>     "coldShards":{},
   [junit4]   2>     "coldReplicas":[],
   [junit4]   2>     "coldCollections":{},
   [junit4]   2>     "hotShards":{"collection1":{"shard1":500.0}},
   [junit4]   2>     "hotCollections":{"collection1":500.0},
   [junit4]   2>     "hotNodes":{
   [junit4]   2>       "127.0.0.1:10001_solr":250.0,
   [junit4]   2>       "127.0.0.1:10000_solr":250.0},
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}"]}}
   [junit4]   2> 1067641 DEBUG (ScheduledTrigger-3355-thread-3) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, search_rate_trigger]
   [junit4]   2> 1067851 DEBUG (simCloudManagerPool-3354-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1067909 INFO  (TEST-TestTriggerIntegration.testSearchRate-seed#[B4380E3F99975242]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testSearchRate
   [junit4]   2> 1067912 INFO  (TEST-TestTriggerIntegration.testSearchRate-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes: 2
   [junit4]   2> ## Empty nodes: 0
   [junit4]   2> ## Dead nodes: 0
   [junit4]   2> ## Collections: [collection1, .system]
   [junit4]   2> ## Max replicas per node: 2
   [junit4]   2> ## Min replicas per node: 1
   [junit4]   2> ## Total replicas: 3
   [junit4]   2> ## * .system 1
   [junit4]   2> ## - active           1
   [junit4]   2> ## * collection1 2
   [junit4]   2> ## - active           2
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ## - CREATE             1
   [junit4]   2> ## - autoscaling        3
   [junit4]   2> ## - update             1
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> ## * Trigger: search_rate_trigger
   [junit4]   2> ## - STARTED         1
   [junit4]   2>
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestTriggerIntegration -Dtests.method=testSearchRate -Dtests.seed=B4380E3F99975242 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=tr-TR -Dtests.timezone=Africa/Freetown -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.80s J1 | TestTriggerIntegration.testSearchRate <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: The trigger did not fire at all
   [junit4]    > at __randomizedtesting.SeedInfo.seed([B4380E3F99975242:E97010B65651F40D]:0)
   [junit4]    > at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1201)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1067918 INFO  (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[B4380E3F99975242]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerThrottling
   [junit4]   2> 1067918 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
   [junit4]   2> 1067918 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
   [junit4]   2> 1067918 INFO  (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1067919 DEBUG (AutoscalingActionExecutor-3356-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"68329d8fa94fbaT3ghy69gvvol4c1654ytqdrone",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime":29329049879400378,
   [junit4]   2>   "eventType":"SEARCHRATE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "hotReplicas":[
   [junit4]   2>       "{\"core_node1\":{\n    \"core\":\"collection1_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}",
   [junit4]   2>       "{\"core_node2\":{\n    \"core\":\"collection1_shard1_replica_n2\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}"],
   [junit4]   2>     "coldNodes":{},
   [junit4]   2>     "coldShards":{},
   [junit4]   2>     "coldReplicas":[],
   [junit4]   2>     "coldCollections":{},
   [junit4]   2>     "hotShards":{"collection1":{"shard1":500.0}},
   [junit4]   2>     "hotCollections":{"collection1":500.0},
   [junit4]   2>     "hotNodes":{
   [junit4]   2>       "127.0.0.1:10001_solr":250.0,
   [junit4]   2>       "127.0.0.1:10000_solr":250.0},
   [junit4]   2>     "_enqueue_time_":29329055706988928,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}"]}}
   [junit4]   2> 1067919 DEBUG (AutoscalingActionExecutor-3356-thread-1) [    ] o.a.s.c.a.ComputePlanAction -- processing event: {
   [junit4]   2>   "id":"68329d8fa94fbaT3ghy69gvvol4c1654ytqdrone",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime":29329049879400378,
   [junit4]   2>   "eventType":"SEARCHRATE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "hotReplicas":[
   [junit4]   2>       "{\"core_node1\":{\n    \"core\":\"collection1_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}",
   [junit4]   2>       "{\"core_node2\":{\n    \"core\":\"collection1_shard1_replica_n2\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}"],
   [junit4]   2>     "coldNodes":{},
   [junit4]   2>     "coldShards":{},
   [junit4]   2>     "coldReplicas":[],
   [junit4]   2>     "coldCollections":{},
   [junit4]   2>     "hotShards":{"collection1":{"shard1":500.0}},
   [junit4]   2>     "hotCollections":{"collection1":500.0},
   [junit4]   2>     "hotNodes":{
   [junit4]   2>       "127.0.0.1:10001_solr":250.0,
   [junit4]   2>       "127.0.0.1:10000_solr":250.0},
   [junit4]   2>     "_enqueue_time_":29329055706988928,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}"]}} with context properties: {BEFORE_ACTION=[compute]}
   [junit4]   2> 1067922 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 1067922 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1067922 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
   [junit4]   2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 1067922 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1067939 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1067939 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1067939 WARN  (AutoscalingActionExecutor-3356-thread-1) [    ] o.a.s.c.a.SystemLogListener Exception sending event to collection .system
   [junit4]   2> java.io.IOException: java.lang.InterruptedException
   [junit4]   2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:632) ~[test/:?]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.SystemLogListener.onEvent(SystemLogListener.java:118) ~[java/:?]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:789) ~[java/:?]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:310) ~[java/:?]
   [junit4]   2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> Caused by: java.lang.InterruptedException
   [junit4]   2> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404) ~[?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:1.8.0_172]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:630) ~[test/:?]
   [junit4]   2> ... 9 more
   [junit4]   2> 1067940 WARN  (AutoscalingActionExecutor-3356-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Exception executing actions
   [junit4]   2> java.lang.Exception: Error executing action: compute for trigger event: {
   [junit4]   2>   "id":"68329d8fa94fbaT3ghy69gvvol4c1654ytqdrone",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime":29329049879400378,
   [junit4]   2>   "eventType":"SEARCHRATE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "hotReplicas":[
   [junit4]   2>       "{\"core_node1\":{\n    \"core\":\"collection1_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}",
   [junit4]   2>       "{\"core_node2\":{\n    \"core\":\"collection1_shard1_replica_n2\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}"],
   [junit4]   2>     "coldNodes":{},
   [junit4]   2>     "coldShards":{},
   [junit4]   2>     "coldReplicas":[],
   [junit4]   2>     "coldCollections":{},
   [junit4]   2>     "hotShards":{"collection1":{"shard1":500.0}},
   [junit4]   2>     "hotCollections":{"collection1":500.0},
   [junit4]   2>     "hotNodes":{
   [junit4]   2>       "127.0.0.1:10001_solr":250.0,
   [junit4]   2>       "127.0.0.1:10000_solr":250.0},
   [junit4]   2>     "_enqueue_time_":29329055706988928,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}"]}}
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:311) ~[java/:?]
   [junit4]   2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> Caused by: org.apache.solr.common.SolrException: Unexpected exception while processing event: {
   [junit4]   2>   "id":"68329d8fa94fbaT3ghy69gvvol4c1654ytqdrone",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime":29329049879400378,
   [junit4]   2>   "eventType":"SEARCHRATE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "hotReplicas":[
   [junit4]   2>       "{\"core_node1\":{\n    \"core\":\"collection1_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10001_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}",
   [junit4]   2>       "{\"core_node2\":{\n    \"core\":\"collection1_shard1_replica_n2\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":250.0,\n    \"node_name\":\"127.0.0.1:10000_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":250,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"collection1\"}}"],
   [junit4]   2>     "coldNodes":{},
   [junit4]   2>     "coldShards":{},
   [junit4]   2>     "coldReplicas":[],
   [junit4]   2>     "coldCollections":{},
   [junit4]   2>     "hotShards":{"collection1":{"shard1":500.0}},
   [junit4]   2>     "hotCollections":{"collection1":500.0},
   [junit4]   2>     "hotNodes":{
   [junit4]   2>       "127.0.0.1:10001_solr":250.0,
   [junit4]   2>       "127.0.0.1:10000_solr":250.0},
   [junit4]   2>     "_enqueue_time_":29329055706988928,
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"collection1\",\n  \"second\":\"shard1\"}]}}"]}}
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ComputePlanAction.process(ComputePlanAction.java:156) ~[java/:?]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:308) ~[java/:?]
   [junit4]   2> ... 6 more
   [junit4]   2> Caused by: java.lang.Exception: Action: compute executed but no policy is configured
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ComputePlanAction.process(ComputePlanAction.java:82) ~[java/:?]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:308) ~[java/:?]
   [junit4]   2> ... 6 more
   [junit4]   2> 1067941 DEBUG (AutoscalingActionExecutor-3356-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 1132 ms for event id=68329d8fa94fbaT3ghy69gvvol4c1654ytqdrone
   [junit4]   2> 1067942 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1067942 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1067942 DEBUG (TEST-TestTriggerIntegration.testTriggerThrottling-seed#[B4380E3F99975242]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1067971 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1067971 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
   [junit4]   2> 1067972 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion -1
   [junit4]   2> 1067972 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
   [junit4]   2> 1067983 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 1067984 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1067984 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
   [junit4]   2> 1067984 DEBUG (ScheduledTrigger-3358-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1068004 DEBUG (ScheduledTrigger-3358-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1068168 DEBUG (ScheduledTrigger-3358-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1068175 DEBUG (simCloudManagerPool-3357-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1068176 DEBUG (simCloudManagerPool-3357-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
   [junit4]   2> 1068177 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
   [junit4]   2> 1068178 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1068178 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
   [junit4]   2> 1068189 DEBUG (ScheduledTrigger-3358-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1068203 DEBUG (simCloudManagerPool-3357-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1068203 DEBUG (simCloudManagerPool-3357-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 13
   [junit4]   2> 1068205 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 13
   [junit4]   2> 1068206 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 1068206 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 1068206 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$ThrottlingTesterAction}], enabled=true}
   [junit4]   2> 1068206 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion 13
   [junit4]   2> 1068206 DEBUG (ScheduledTrigger-3358-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 1068206 DEBUG (ScheduledTrigger-3358-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 1068209 DEBUG (ScheduledTrigger-3358-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1068215 DEBUG (simCloudManagerPool-3357-thread-3) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1068222 DEBUG (simCloudManagerPool-3357-thread-3) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 14
   [junit4]   2> 1068226 DEBUG (ScheduledTrigger-3358-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 1068226 DEBUG (ScheduledTrigger-3358-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [juni

[...truncated too long message...]

e = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1589764109
     [copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1589764109
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1589764109

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1020915611
     [copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1020915611
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1020915611

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 134ms :: artifacts dl 1ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
        ---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 187 minutes 2 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 79 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/79/

1 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup

Error Message:
should be at least one inactive event

Stack Trace:
java.lang.AssertionError: should be at least one inactive event
        at __randomizedtesting.SeedInfo.seed([2927A44D25DAF653:340B643F4499D158]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup(ScheduledMaintenanceTriggerTest.java:224)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12355 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest
   [junit4]   2> 86444 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_2927A44D25DAF653-001/init-core-data-001
   [junit4]   2> 86445 WARN  (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=21 numCloses=21
   [junit4]   2> 86447 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 86473 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 86474 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_2927A44D25DAF653-001/tempDir-001
   [junit4]   2> 86474 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 86494 INFO  (Thread-109) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 86494 INFO  (Thread-109) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 86539 ERROR (Thread-109) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 86600 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:42301
   [junit4]   2> 86730 INFO  (zkConnectionManagerCallback-217-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 86745 INFO  (jetty-launcher-214-thread-1) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 86759 INFO  (jetty-launcher-214-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 86759 INFO  (jetty-launcher-214-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 86759 INFO  (jetty-launcher-214-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 86759 INFO  (jetty-launcher-214-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@18027251{/solr,null,AVAILABLE}
   [junit4]   2> 86763 INFO  (jetty-launcher-214-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4069910b{HTTP/1.1,[http/1.1]}{127.0.0.1:53447}
   [junit4]   2> 86763 INFO  (jetty-launcher-214-thread-1) [    ] o.e.j.s.Server Started @86843ms
   [junit4]   2> 86763 INFO  (jetty-launcher-214-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=53447}
   [junit4]   2> 86763 ERROR (jetty-launcher-214-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 86763 INFO  (jetty-launcher-214-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 86763 INFO  (jetty-launcher-214-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 86764 INFO  (jetty-launcher-214-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 86771 INFO  (jetty-launcher-214-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 86771 INFO  (jetty-launcher-214-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-18T08:53:08.566Z
   [junit4]   2> 86780 INFO  (zkConnectionManagerCallback-219-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 86781 INFO  (jetty-launcher-214-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 87657 INFO  (jetty-launcher-214-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42301/solr
   [junit4]   2> 87692 INFO  (zkConnectionManagerCallback-223-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 87733 INFO  (zkConnectionManagerCallback-225-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 88258 INFO  (jetty-launcher-214-thread-1) [n:127.0.0.1:53447_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 88303 INFO  (jetty-launcher-214-thread-1) [n:127.0.0.1:53447_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:53447_solr
   [junit4]   2> 88305 INFO  (jetty-launcher-214-thread-1) [n:127.0.0.1:53447_solr    ] o.a.s.c.Overseer Overseer (id=73995092827308035-127.0.0.1:53447_solr-n_0000000000) starting
   [junit4]   2> 88556 INFO  (zkConnectionManagerCallback-232-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 88563 INFO  (jetty-launcher-214-thread-1) [n:127.0.0.1:53447_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42301/solr ready
   [junit4]   2> 88592 INFO  (jetty-launcher-214-thread-1) [n:127.0.0.1:53447_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53447_solr
   [junit4]   2> 88597 INFO  (OverseerStateUpdate-73995092827308035-127.0.0.1:53447_solr-n_0000000000) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 88617 DEBUG (OverseerAutoScalingTriggerThread-73995092827308035-127.0.0.1:53447_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 88621 DEBUG (OverseerAutoScalingTriggerThread-73995092827308035-127.0.0.1:53447_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 88622 DEBUG (OverseerAutoScalingTriggerThread-73995092827308035-127.0.0.1:53447_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 88622 DEBUG (OverseerAutoScalingTriggerThread-73995092827308035-127.0.0.1:53447_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 88645 INFO  (zkCallback-231-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 88663 DEBUG (OverseerAutoScalingTriggerThread-73995092827308035-127.0.0.1:53447_solr-n_0000000000) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:53447_solr]
   [junit4]   2> 88676 DEBUG (OverseerAutoScalingTriggerThread-73995092827308035-127.0.0.1:53447_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 88676 DEBUG (OverseerAutoScalingTriggerThread-73995092827308035-127.0.0.1:53447_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 88679 DEBUG (ScheduledTrigger-145-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 88712 INFO  (jetty-launcher-214-thread-1) [n:127.0.0.1:53447_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 88814 INFO  (jetty-launcher-214-thread-1) [n:127.0.0.1:53447_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53447.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d9ebbc0
   [junit4]   2> 88835 INFO  (jetty-launcher-214-thread-1) [n:127.0.0.1:53447_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53447.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d9ebbc0
   [junit4]   2> 88835 INFO  (jetty-launcher-214-thread-1) [n:127.0.0.1:53447_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53447.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3d9ebbc0
   [junit4]   2> 88845 INFO  (jetty-launcher-214-thread-1) [n:127.0.0.1:53447_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_2927A44D25DAF653-001/tempDir-001/node1/.
   [junit4]   2> 88932 INFO  (zkConnectionManagerCallback-235-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 88964 INFO  (zkConnectionManagerCallback-240-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 88973 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 88975 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42301/solr ready
   [junit4]   2> 89221 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
   [junit4]   2> 89222 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 89224 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 89249 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 89249 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 89276 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 89292 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10000_solr]
   [junit4]   2> 89293 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 89293 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 89294 DEBUG (ScheduledTrigger-154-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89315 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89335 DEBUG (ScheduledTrigger-154-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89360 DEBUG (ScheduledTrigger-154-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89380 DEBUG (ScheduledTrigger-154-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89525 DEBUG (ScheduledTrigger-154-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89543 INFO  (TEST-ScheduledMaintenanceTriggerTest.testTriggerDefaults-seed#[2927A44D25DAF653]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerDefaults
   [junit4]   2> 89544 INFO  (TEST-ScheduledMaintenanceTriggerTest.testTriggerDefaults-seed#[2927A44D25DAF653]) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest {
   [junit4]   2>   "cluster-preferences":[
   [junit4]   2>     {
   [junit4]   2>       "minimize":"cores",
   [junit4]   2>       "precision":1},
   [junit4]   2>     {"maximize":"freedisk"}],
   [junit4]   2>   "triggers":{
   [junit4]   2>     ".scheduled_maintenance":{
   [junit4]   2>       "name":".scheduled_maintenance",
   [junit4]   2>       "startTime":"NOW",
   [junit4]   2>       "event":"scheduled",
   [junit4]   2>       "every":"+1DAY",
   [junit4]   2>       "actions":[
   [junit4]   2>         {
   [junit4]   2>           "name":"inactive_shard_plan",
   [junit4]   2>           "class":"solr.InactiveShardPlanAction"},
   [junit4]   2>         {
   [junit4]   2>           "name":"execute_plan",
   [junit4]   2>           "class":"solr.ExecutePlanAction"}],
   [junit4]   2>       "enabled":true},
   [junit4]   2>     ".auto_add_replicas":{
   [junit4]   2>       "name":".auto_add_replicas",
   [junit4]   2>       "event":"nodeLost",
   [junit4]   2>       "waitFor":120,
   [junit4]   2>       "actions":[
   [junit4]   2>         {
   [junit4]   2>           "name":"auto_add_replicas_plan",
   [junit4]   2>           "class":"solr.AutoAddReplicasPlanAction"},
   [junit4]   2>         {
   [junit4]   2>           "name":"execute_plan",
   [junit4]   2>           "class":"solr.ExecutePlanAction"}],
   [junit4]   2>       "enabled":true}},
   [junit4]   2>   "listeners":{
   [junit4]   2>     ".scheduled_maintenance.system":{
   [junit4]   2>       "trigger":".scheduled_maintenance",
   [junit4]   2>       "afterAction":[],
   [junit4]   2>       "stage":[
   [junit4]   2>         "STARTED",
   [junit4]   2>         "ABORTED",
   [junit4]   2>         "SUCCEEDED",
   [junit4]   2>         "FAILED",
   [junit4]   2>         "BEFORE_ACTION",
   [junit4]   2>         "AFTER_ACTION",
   [junit4]   2>         "IGNORED"],
   [junit4]   2>       "class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
   [junit4]   2>       "beforeAction":[]},
   [junit4]   2>     ".auto_add_replicas.system":{
   [junit4]   2>       "trigger":".auto_add_replicas",
   [junit4]   2>       "afterAction":[],
   [junit4]   2>       "stage":[
   [junit4]   2>         "STARTED",
   [junit4]   2>         "ABORTED",
   [junit4]   2>         "SUCCEEDED",
   [junit4]   2>         "FAILED",
   [junit4]   2>         "BEFORE_ACTION",
   [junit4]   2>         "AFTER_ACTION",
   [junit4]   2>         "IGNORED"],
   [junit4]   2>       "class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
   [junit4]   2>       "beforeAction":[]}},
   [junit4]   2>   "properties":{}}
   [junit4]   2> 89546 DEBUG (ScheduledTrigger-154-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89566 DEBUG (ScheduledTrigger-154-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89587 DEBUG (ScheduledTrigger-154-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89608 DEBUG (ScheduledTrigger-154-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89622 DEBUG (simCloudManagerPool-153-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 89622 DEBUG (simCloudManagerPool-153-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 89628 DEBUG (ScheduledTrigger-154-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89637 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 89637 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 89637 INFO  (TEST-ScheduledMaintenanceTriggerTest.testTriggerDefaults-seed#[2927A44D25DAF653]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testTriggerDefaults
   [junit4]   2> 89637 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 89649 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89655 INFO  (TEST-ScheduledMaintenanceTriggerTest.testInactiveShardCleanup-seed#[2927A44D25DAF653]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testInactiveShardCleanup
   [junit4]   2> 89669 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89681 DEBUG (ScheduledTrigger-145-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89689 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89710 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89730 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89752 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89773 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89793 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89813 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89834 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89856 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89877 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89897 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89918 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89932 DEBUG (simCloudManagerPool-153-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for ScheduledMaintenanceTriggerTest_collection1 / shard1: {"core_node1":{
   [junit4]   2>     "core":"ScheduledMaintenanceTriggerTest_collection1_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"ScheduledMaintenanceTriggerTest_collection1"}}
   [junit4]   2> 89938 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89953 DEBUG (simCloudManagerPool-153-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for ScheduledMaintenanceTriggerTest_collection1 / shard1_1: {"core_node5":{
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>     "base_url":"http://127.0.0.1:10000/solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "core":"ScheduledMaintenanceTriggerTest_collection1_shard1_1_replica0",
   [junit4]   2>     "shard":"shard1_1",
   [junit4]   2>     "collection":"ScheduledMaintenanceTriggerTest_collection1"}}
   [junit4]   2> 89956 DEBUG (simCloudManagerPool-153-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for ScheduledMaintenanceTriggerTest_collection1 / shard1_0: {"core_node4":{
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":20480,
   [junit4]   2>     "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>     "base_url":"http://127.0.0.1:10000/solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":1.9073486328125E-5,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "core":"ScheduledMaintenanceTriggerTest_collection1_shard1_0_replica0",
   [junit4]   2>     "shard":"shard1_0",
   [junit4]   2>     "collection":"ScheduledMaintenanceTriggerTest_collection1"}}
   [junit4]   2> 89958 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89979 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 89992 DEBUG (simCloudManagerPool-153-thread-9) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 89993 DEBUG (simCloudManagerPool-153-thread-9) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 89999 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 90008 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 90009 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 90009 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 90020 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 90040 DEBUG (ScheduledTrigger-154-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 90045 DEBUG (simCloudManagerPool-153-thread-10) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 90045 DEBUG (simCloudManagerPool-153-thread-10) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 90046 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 90046 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 90046 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 90046 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.ScheduledTrigger ScheduledTrigger .scheduled_maintenance firing registered processor for scheduled time 2018-06-18T08:53:24.894Z, now=2018-06-18T08:54:04.017Z
   [junit4]   2> 90049 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"69083f6dfb10a1T36a7btcjn7m4fqgmy3saus20a",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":29563941075751073,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{"actualEventTime":1529312044017}}
   [junit4]   2> 90050 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, .scheduled_maintenance]
   [junit4]   2> 90163 DEBUG (simCloudManagerPool-153-thread-11) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 90169 INFO  (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=29563947164680773, stage=STARTED, actionName='null', event={
   [junit4]   2>   "id":"69083f6dfb10a1T36a7btcjn7m4fqgmy3saus20a",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":29563941075751073,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1529312044017,
   [junit4]   2>     "_enqueue_time_":29563941256207573}}, context={}, config={
   [junit4]   2>   "trigger":".scheduled_maintenance",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":"inactive_shard_plan",
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":"inactive_shard_plan"}, message='null'}
   [junit4]   2> 90184 DEBUG (AutoscalingActionExecutor-155-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"69083f6dfb10a1T36a7btcjn7m4fqgmy3saus20a",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":29563941075751073,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1529312044017,
   [junit4]   2>     "_enqueue_time_":29563941256207573}}
   [junit4]   2> 90188 INFO  (AutoscalingActionExecutor-155-thread-1) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=29563948137888573, stage=BEFORE_ACTION, actionName='inactive_shard_plan', event={
   [junit4]   2>   "id":"69083f6dfb10a1T36a7btcjn7m4fqgmy3saus20a",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":29563941075751073,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1529312044017,
   [junit4]   2>     "_enqueue_time_":29563941256207573}}, context={properties.BEFORE_ACTION=[inactive_shard_plan], source=.scheduled_maintenance}, config={
   [junit4]   2>   "trigger":".scheduled_maintenance",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":"inactive_shard_plan",
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":"inactive_shard_plan"}, message='null'}
   [junit4]   2> 90191 DEBUG (AutoscalingActionExecutor-155-thread-1) [    ] o.a.s.c.a.InactiveShardPlanAction ScheduledMaintenanceTriggerTest_collection1/shard1: tstamp=1529312037455721800, time=1529312051230047650, delta=13
   [junit4]   2> 90191 DEBUG (AutoscalingActionExecutor-155-thread-1) [    ] o.a.s.c.a.InactiveShardPlanAction -- delete inactive ScheduledMaintenanceTriggerTest_collection1 / shard1
   [junit4]   2> 90212 INFO  (AutoscalingActionExecutor-155-thread-1) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=29563948425539773, stage=AFTER_ACTION, actionName='inactive_shard_plan', event={
   [junit4]   2>   "id":"69083f6dfb10a1T36a7btcjn7m4fqgmy3saus20a",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":29563941075751073,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1529312044017,
   [junit4]   2>     "_enqueue_time_":29563941256207573}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$DeleteShard, method=GET, params.action=DELETESHARD, params.collection=ScheduledMaintenanceTriggerTest_collection1, params.shard=shard1}], properties.BEFORE_ACTION=[inactive_shard_plan], source=.scheduled_maintenance, properties.inactive_shard_plan={inactive={ScheduledMaintenanceTriggerTest_collection1=[shard1]}, cleanup={ScheduledMaintenanceTriggerTest_collection1=[shard1]}}, properties.AFTER_ACTION=[inactive_shard_plan]}, config={
   [junit4]   2>   "trigger":".scheduled_maintenance",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":"inactive_shard_plan",
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":"inactive_shard_plan"}, message='null'}
   [junit4]   2> 90216 DEBUG (AutoscalingActionExecutor-155-thread-1) [    ] o.a.s.c.a.ExecutePlanAction -- processing event: {
   [junit4]   2>   "id":"69083f6dfb10a1T36a7btcjn7m4fqgmy3saus20a",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":29563941075751073,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1529312044017,
   [junit4]   2>     "_enqueue_time_":29563941256207573}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$DeleteShard@53eea9c1], inactive_shard_plan={inactive={ScheduledMaintenanceTriggerTest_collection1=[shard1]}, cleanup={ScheduledMaintenanceTriggerTest_collection1=[shard1]}}, AFTER_ACTION=[inactive_shard_plan], BEFORE_ACTION=[inactive_shard_plan, execute_plan]}
   [junit4]   2> 90216 DEBUG (AutoscalingActionExecutor-155-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=DELETESHARD&collection=ScheduledMaintenanceTriggerTest_collection1&shard=shard1
   [junit4]   2> 90360 DEBUG (simCloudManagerPool-153-thread-17) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 90361 INFO  (AutoscalingActionExecutor-155-thread-1) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=29563956803092573, stage=SUCCEEDED, actionName='null', event={
   [junit4]   2>   "id":"69083f6dfb10a1T36a7btcjn7m4fqgmy3saus20a",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":29563941075751073,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1529312044017,
   [junit4]   2>     "_enqueue_time_":29563941256207573}}, context={}, config={
   [junit4]   2>   "trigger":".scheduled_maintenance",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":"inactive_shard_plan",
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":"inactive_shard_plan"}, message='null'}
   [junit4]   2> 90364 DEBUG (AutoscalingActionExecutor-155-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 90364 DEBUG (AutoscalingActionExecutor-155-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .scheduled_maintenance after 100ms
   [junit4]   2> 90364 DEBUG (AutoscalingActionExecutor-155-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 8994 ms for event id=69083f6dfb10a1T36a7btcjn7m4fqgmy3saus20a
   [junit4]   2> 90538 DEBUG (ScheduledTrigger-154-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 90559 DEBUG (ScheduledTrigger-154-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 90596 DEBUG (simCloudManagerPool-153-thread-18) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 90596 DEBUG (simCloudManagerPool-153-thread-18) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 90597 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 90597 DEBUG (ScheduledTrigger-154-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 90598 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 90598 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 90598 WARN  (ScheduledTrigger-154-thread-2) [    ] o.a.s.c.a.ScheduledTrigger ScheduledTrigger ran but was already closed
   [junit4]   2> 90599 ERROR (ScheduledTrigger-154-thread-2) [    ] o.a.s.c.a.ScheduledTriggers Unexpected exception from trigger: .scheduled_maintenance
   [junit4]   2> java.lang.RuntimeException: Trigger has been closed
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTrigger.run(ScheduledTrigger.java:155) ~[java/:?]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.run(ScheduledTriggers.java:588) [java/:?]
   [junit4]   2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 90618 DEBUG (ScheduledTrigger-154-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 90622 DEBUG (simCloudManagerPool-153-thread-19) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 90622 DEBUG (simCloudManagerPool-153-thread-19) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 90628 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 90628 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 90628 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 90629 INFO  (TEST-ScheduledMaintenanceTriggerTest.testInactiveShardCleanup-seed#[2927A44D25DAF653]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testInactiveShardCleanup
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ScheduledMaintenanceTriggerTest -Dtests.method=testInactiveShardCleanup -Dtests.seed=2927A44D25DAF653 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=fr-CA -Dtests.timezone=Europe/Istanbul -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 0.99s J2 | ScheduledMaintenanceTriggerTest.testInactiveShardCleanup <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: should be at least one inactive event
   [junit4]    > at __randomizedtesting.SeedInfo.seed([2927A44D25DAF653:340B643F4499D158]:0)
   [junit4]    > at org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup(ScheduledMaintenanceTriggerTest.java:224)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 90638 DEBUG (ScheduledTrigger-154-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 90658 DEBUG (ScheduledTrigger-154-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 90678 DEBUG (ScheduledTrigger-154-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 90681 DEBUG (ScheduledTrigger-145-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 90699 DEBUG (ScheduledTrigger-154-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 90720 DEBUG (ScheduledTrigger-154-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 90730 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 90730 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 90730 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 90732 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 90733 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 90733 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 90733 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 90738 INFO  (jetty-closer-215-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@4069910b{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 90740 INFO  (jetty-closer-215-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=924704828
   [junit4]   2> 90745 INFO  (jetty-closer-215-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 90745 INFO  (jetty-closer-215-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3774005e: rootName = solr_53447, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@5642c822
   [junit4]   2> 90777 INFO  (jetty-closer-215-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 90777 INFO  (jetty-closer-215-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@550e25f3: rootName = solr_53447, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@6f88a6fe
   [junit4]   2> 90794 INFO  (jetty-closer-215-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 90794 INFO  (jetty-closer-215-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5c67f22d: rootName = solr_53447, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@377c038e
   [junit4]   2> 90804 INFO  (jetty-closer-215-thread-1) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:53447_solr
   [junit4]   2> 90806 INFO  (zkCallback-231-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 90807 INFO  (zkCallback-224-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 90808 INFO  (jetty-closer-215-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 90809 INFO  (jetty-closer-215-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 90812 INFO  (jetty-closer-215-thread-1) [    ] o.a.s.c.Overseer Overseer (id=73995092827308035-127.0.0.1:53447_solr-n_0000000000) closing
   [junit4]   2> 90812 INFO  (OverseerStateUpdate-73995092827308035-127.0.0.1:53447_solr-n_0000000000) [    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:53447_solr
   [junit4]   2> 90813 WARN  (OverseerAutoScalingTriggerThread-73995092827308035-127.0.0.1:53447_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 90814 DEBUG (jetty-closer-215-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 90815 DEBUG (jetty-closer-215-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 90815 DEBUG (jetty-closer-215-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 90815 DEBUG (jetty-closer-215-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 90815 DEBUG (jetty-closer-215-thread-1) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 90815 DEBUG (jetty-closer-215-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 90836 INFO  (jetty-closer-215-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@18027251{/solr,null,UNAVAILABLE}
   [junit4]   2> 90838 INFO  (jetty-closer-215-thread-1) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 90864 ERROR (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 90865 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[2927A44D25DAF653]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42301 42301
   [junit4]   2> 91053 INFO  (Thread-109) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42301 42301
   [junit4]   2> 91054 WARN  (Thread-109) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/aliases.json
   [junit4]   2> 3 /solr/clusterprops.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/clusterstate.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/live_nodes
   [junit4]   2> 3 /solr/collections
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_2927A44D25DAF653-001
   [junit4]   2> Jun 18, 2018 8:53:12 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@7d8647c), locale=fr-CA, timezone=Europe/Istanbul
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=235931968,total=310378496
   [junit4]   2> NOTE: All tests run in this JVM: [NodeLostTriggerTest, TestManagedSynonymGraphFilterFactory, ReplaceNodeNoTargetTest, OverriddenZkACLAndCredentialsProvidersTest, TestExpandComponent, ScheduledMaintenanceTriggerTest]
   [junit4] Completed [11/818 (1!)] on J2 in 5.08s, 2 tests, 1 failure <<< FAILURES!

[...truncated 53050 lines...]
[asciidoctor:convert] asciidoctor: ERROR: about-this-guide.adoc: line 1: invalid part, must have at least one section (e.g., chapter, appendix, etc.)
[asciidoctor:convert] asciidoctor: ERROR: solr-glossary.adoc: line 1: invalid part, must have at least one section (e.g., chapter, appendix, etc.)
     [java] Processed 2241 links (1790 relative) to 3127 anchors in 247 files
     [echo] Validated Links & Anchors via: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-ref-guide/bare-bones-html/

-documentation-lint:
    [jtidy] Checking for broken html (such as invalid tags)...
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/build/jtidy_tmp
     [echo] Checking for broken links...
     [exec]
     [exec] Crawl/parse...
     [exec]
     [exec] Verify...
     [echo] Checking for malformed docs...

jar-checksums:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

common.resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1535146743
     [copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1535146743
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1535146743

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null654473688
     [copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null654473688
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null654473688

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 49ms :: artifacts dl 2ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
        ---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 174 minutes 23 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 80 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/80/

1 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
        at __randomizedtesting.SeedInfo.seed([A35E453ADC4267E8:FE165BB31384C1A7]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1201)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13409 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_A35E453ADC4267E8-001/init-core-data-001
   [junit4]   2> 1482207 DEBUG (SUITE-TestTriggerIntegration-seed#[A35E453ADC4267E8]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10016_solr
   [junit4]   2> 1482207 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1482207 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 1482208 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 1482208 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 1482209 INFO  (SUITE-TestTriggerIntegration-seed#[A35E453ADC4267E8]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1482216 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
   [junit4]   2> 1482217 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1482217 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 1482223 DEBUG (ScheduledTrigger-3680-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482231 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 1482231 INFO  (TEST-TestTriggerIntegration.testCooldown-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1482231 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1482232 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1482232 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1482232 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1482232 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1482232 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1482232 DEBUG (TEST-TestTriggerIntegration.testCooldown-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1482233 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1482233 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 1482237 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 1482237 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 1482237 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
   [junit4]   2> 1482238 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1482238 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 1482238 DEBUG (ScheduledTrigger-3685-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482258 DEBUG (ScheduledTrigger-3685-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482279 DEBUG (ScheduledTrigger-3685-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482342 DEBUG (ScheduledTrigger-3685-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482347 DEBUG (simCloudManagerPool-3684-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1482347 DEBUG (simCloudManagerPool-3684-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 1482348 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 1482348 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1482348 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 1482350 DEBUG (simCloudManagerPool-3684-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1482350 DEBUG (simCloudManagerPool-3684-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 1482351 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 1482351 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 1482351 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger - Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10017_solr]
   [junit4]   2> 1482351 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
   [junit4]   2> 1482351 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 1482351 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 1482351 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 1482352 DEBUG (simCloudManagerPool-3684-thread-3) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1482353 DEBUG (simCloudManagerPool-3684-thread-3) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 1482353 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 1482353 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 1482363 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1482371 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 1482371 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1482371 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10018_solr at time 29703443679716759
   [junit4]   2> 1482383 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1482392 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 1482392 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1482392 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing registered processor for nodes: [127.0.0.1:10018_solr] added at times [29703443679716759], now=29703444693308009
   [junit4]   2> 1482392 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"69871fe922a597Te116fkt8tfq9pffeimnm8zqzq",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":29703443679716759,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703443679716759],
   [junit4]   2>     "nodeNames":["127.0.0.1:10018_solr"]}}
   [junit4]   2> 1482396 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_cooldown_trigger]
   [junit4]   2> 1482419 DEBUG (simCloudManagerPool-3684-thread-4) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1482419 DEBUG (ScheduledTrigger-3685-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10018_solr
   [junit4]   2> 1482419 DEBUG (AutoscalingActionExecutor-3686-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"69871fe922a597Te116fkt8tfq9pffeimnm8zqzq",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":29703443679716759,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703443679716759],
   [junit4]   2>     "_enqueue_time_":29703444937672159,
   [junit4]   2>     "nodeNames":["127.0.0.1:10018_solr"]}}
   [junit4]   2> 1482421 DEBUG (simCloudManagerPool-3684-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1482421 DEBUG (AutoscalingActionExecutor-3686-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 1482421 DEBUG (AutoscalingActionExecutor-3686-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_cooldown_trigger after 100ms
   [junit4]   2> 1482421 DEBUG (AutoscalingActionExecutor-3686-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 109 ms for event id=69871fe922a597Te116fkt8tfq9pffeimnm8zqzq
   [junit4]   2> 1482521 DEBUG (ScheduledTrigger-3685-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1482522 DEBUG (ScheduledTrigger-3685-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 1482522 DEBUG (ScheduledTrigger-3685-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1482522 DEBUG (ScheduledTrigger-3685-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10019_solr at time 29703451196576559
   [junit4]   2> 1482541 DEBUG (ScheduledTrigger-3685-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1482542 DEBUG (ScheduledTrigger-3685-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
   [junit4]   2> 1482542 DEBUG (ScheduledTrigger-3685-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1482542 DEBUG (ScheduledTrigger-3685-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing registered processor for nodes: [127.0.0.1:10019_solr] added at times [29703451196576559], now=29703452207848959
   [junit4]   2> 1482542 DEBUG (ScheduledTrigger-3685-thread-4) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"698721a92cd32fTe116fkt8tfq9pffeimnm8zqzt",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":29703451196576559,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703451196576559],
   [junit4]   2>     "nodeNames":["127.0.0.1:10019_solr"]}}
   [junit4]   2> 1482542 DEBUG (ScheduledTrigger-3685-thread-4) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_cooldown_trigger]
   [junit4]   2> 1482543 DEBUG (simCloudManagerPool-3684-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1482543 DEBUG (ScheduledTrigger-3685-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10019_solr
   [junit4]   2> 1482543 DEBUG (AutoscalingActionExecutor-3686-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"698721a92cd32fTe116fkt8tfq9pffeimnm8zqzt",
   [junit4]   2>   "source":"node_added_cooldown_trigger",
   [junit4]   2>   "eventTime":29703451196576559,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703451196576559],
   [junit4]   2>     "_enqueue_time_":29703452219375509,
   [junit4]   2>     "nodeNames":["127.0.0.1:10019_solr"]}}
   [junit4]   2> 1482566 DEBUG (simCloudManagerPool-3684-thread-9) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1482566 DEBUG (AutoscalingActionExecutor-3686-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 1482566 DEBUG (AutoscalingActionExecutor-3686-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_cooldown_trigger after 100ms
   [junit4]   2> 1482566 DEBUG (AutoscalingActionExecutor-3686-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 1160 ms for event id=698721a92cd32fTe116fkt8tfq9pffeimnm8zqzt
   [junit4]   2> 1482585 INFO  (TEST-TestTriggerIntegration.testCooldown-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes: 4
   [junit4]   2> ## Empty nodes: 3
   [junit4]   2> ## Dead nodes: 0
   [junit4]   2> ## Collections: [.system]
   [junit4]   2> ## Max replicas per node: 1
   [junit4]   2> ## Min replicas per node: 0
   [junit4]   2> ## Total replicas: 1
   [junit4]   2> ## * .system 1
   [junit4]   2> ## - active           1
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ## - autoscaling        3
   [junit4]   2> ## - update             4
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> ## * Trigger: node_added_cooldown_trigger
   [junit4]   2> ## - STARTED         2
   [junit4]   2> ## - SUCCEEDED       2
   [junit4]   2>
   [junit4]   2> 1482589 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 1482589 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 1482589 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 1482589 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1482589 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
   [junit4]   2> 1482590 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10017_solr
   [junit4]   2> 1482590 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10019_solr
   [junit4]   2> 1482590 INFO  (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1482590 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_172]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
   [junit4]   2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 1482590 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1482591 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1482591 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1482591 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1482591 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1482591 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1482609 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1482609 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
   [junit4]   2> 1482609 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
   [junit4]   2> 1482609 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
   [junit4]   2> 1482613 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10018_solr, 127.0.0.1:10016_solr]
   [junit4]   2> 1482618 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1482618 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
   [junit4]   2> 1482620 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482641 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482661 DEBUG (ScheduledTrigger-3688-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482682 DEBUG (ScheduledTrigger-3688-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482702 DEBUG (ScheduledTrigger-3688-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482722 DEBUG (ScheduledTrigger-3688-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482723 DEBUG (simCloudManagerPool-3687-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1482723 DEBUG (simCloudManagerPool-3687-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
   [junit4]   2> 1482724 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
   [junit4]   2> 1482724 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1482724 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
   [junit4]   2> 1482742 DEBUG (simCloudManagerPool-3687-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1482742 DEBUG (ScheduledTrigger-3688-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482742 DEBUG (simCloudManagerPool-3687-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
   [junit4]   2> 1482743 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
   [junit4]   2> 1482743 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 1482743 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10018_solr, 127.0.0.1:10016_solr]
   [junit4]   2> 1482743 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestDummyAction}], enabled=true}
   [junit4]   2> 1482743 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
   [junit4]   2> 1482743 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1482743 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 1482750 DEBUG (simCloudManagerPool-3687-thread-3) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1482750 DEBUG (simCloudManagerPool-3687-thread-3) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
   [junit4]   2> 1482751 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
   [junit4]   2> 1482751 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
   [junit4]   2> 1482758 DEBUG (simCloudManagerPool-3687-thread-4) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1482758 DEBUG (simCloudManagerPool-3687-thread-4) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
   [junit4]   2> 1482762 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1482763 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
   [junit4]   2> 1482763 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
   [junit4]   2> 1482764 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1482764 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 1482783 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1482784 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1482784 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1482784 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10020_solr at time 29703464298133159
   [junit4]   2> 1482803 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1482804 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1482804 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1482857 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1482857 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1482857 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1482857 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10020_solr] added at times [29703464298133159], now=29703467958408109
   [junit4]   2> 1482857 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"698724b616b6a7Te116fkt8tfq9pffeimnm8zqzw",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29703464298133159,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703464298133159],
   [junit4]   2>     "nodeNames":["127.0.0.1:10020_solr"]}}
   [junit4]   2> 1482859 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
   [junit4]   2> 1482947 DEBUG (simCloudManagerPool-3687-thread-5) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1482983 DEBUG (ScheduledTrigger-3688-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10020_solr
   [junit4]   2> 1482984 DEBUG (AutoscalingActionExecutor-3689-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"698724b616b6a7Te116fkt8tfq9pffeimnm8zqzw",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29703464298133159,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703464298133159],
   [junit4]   2>     "_enqueue_time_":29703468074017759,
   [junit4]   2>     "nodeNames":["127.0.0.1:10020_solr"]}}
   [junit4]   2> 1483022 DEBUG (simCloudManagerPool-3687-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1483023 DEBUG (AutoscalingActionExecutor-3689-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 1483023 DEBUG (AutoscalingActionExecutor-3689-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
   [junit4]   2> 1483023 DEBUG (AutoscalingActionExecutor-3689-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 1964 ms for event id=698724b616b6a7Te116fkt8tfq9pffeimnm8zqzw
   [junit4]   2> 1483123 DEBUG (ScheduledTrigger-3688-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1483123 DEBUG (ScheduledTrigger-3688-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483123 DEBUG (ScheduledTrigger-3688-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1483123 DEBUG (ScheduledTrigger-3688-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10021_solr at time 29703481285195309
   [junit4]   2> 1483143 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1483144 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483144 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1483163 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1483164 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483164 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1483184 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1483184 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483184 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1483184 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10021_solr] added at times [29703481285195309], now=29703484318094859
   [junit4]   2> 1483184 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"698728aa98b62dTe116fkt8tfq9pffeimnm8zqzz",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29703481285195309,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703481285195309],
   [junit4]   2>     "nodeNames":["127.0.0.1:10021_solr"]}}
   [junit4]   2> 1483184 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
   [junit4]   2> 1483222 DEBUG (simCloudManagerPool-3687-thread-9) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1483222 DEBUG (ScheduledTrigger-3688-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10021_solr
   [junit4]   2> 1483222 DEBUG (AutoscalingActionExecutor-3689-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"698728aa98b62dTe116fkt8tfq9pffeimnm8zqzz",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29703481285195309,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703481285195309],
   [junit4]   2>     "_enqueue_time_":29703484330833309,
   [junit4]   2>     "nodeNames":["127.0.0.1:10021_solr"]}}
   [junit4]   2> 1483244 DEBUG (simCloudManagerPool-3687-thread-10) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10016_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1483245 WARN  (AutoscalingActionExecutor-3689-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Exception executing actions
   [junit4]   2> java.lang.Exception: Error executing action: test1 for trigger event: {
   [junit4]   2>   "id":"698728aa98b62dTe116fkt8tfq9pffeimnm8zqzz",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29703481285195309,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703481285195309],
   [junit4]   2>     "_enqueue_time_":29703484330833309,
   [junit4]   2>     "nodeNames":["127.0.0.1:10021_solr"]}}
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:311) ~[java/:?]
   [junit4]   2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> Caused by: java.lang.RuntimeException: failure
   [junit4]   2> at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestDummyAction.process(TestTriggerIntegration.java:911) ~[test/:?]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:308) ~[java/:?]
   [junit4]   2> ... 6 more
   [junit4]   2> 1483245 DEBUG (AutoscalingActionExecutor-3689-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 1483245 DEBUG (AutoscalingActionExecutor-3689-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
   [junit4]   2> 1483245 DEBUG (AutoscalingActionExecutor-3689-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 1151 ms for event id=698728aa98b62dTe116fkt8tfq9pffeimnm8zqzz
   [junit4]   2> 1483264 INFO  (TEST-TestTriggerIntegration.testListeners-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes: 4
   [junit4]   2> ## Empty nodes: 3
   [junit4]   2> ## Dead nodes: 0
   [junit4]   2> ## Collections: [.system]
   [junit4]   2> ## Max replicas per node: 1
   [junit4]   2> ## Min replicas per node: 0
   [junit4]   2> ## Total replicas: 1
   [junit4]   2> ## * .system 1
   [junit4]   2> ## - active           1
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ## - autoscaling        4
   [junit4]   2> ## - update             4
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> ## * Trigger: node_added_trigger
   [junit4]   2> ## - FAILED          1
   [junit4]   2> ## - STARTED         2
   [junit4]   2> ## - SUCCEEDED       1
   [junit4]   2>
   [junit4]   2> 1483269 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
   [junit4]   2> 1483269 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
   [junit4]   2> 1483269 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 1483269 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1483269 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
   [junit4]   2> 1483269 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10020_solr
   [junit4]   2> 1483269 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10018_solr
   [junit4]   2> 1483269 INFO  (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1483269 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_172]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
   [junit4]   2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 1483270 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1483270 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1483270 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1483270 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1483270 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1483270 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1483281 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1483283 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 13
   [junit4]   2> 1483284 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion -1
   [junit4]   2> 1483284 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 13
   [junit4]   2> 1483291 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10021_solr]
   [junit4]   2> 1483302 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1483302 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion 13
   [junit4]   2> 1483303 DEBUG (ScheduledTrigger-3691-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1483323 DEBUG (ScheduledTrigger-3691-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1483457 DEBUG (simCloudManagerPool-3690-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1483457 DEBUG (simCloudManagerPool-3690-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 14
   [junit4]   2> 1483475 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 14
   [junit4]   2> 1483475 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1483475 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion 14
   [junit4]   2> 1483475 DEBUG (ScheduledTrigger-3691-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1483475 WARN  (ScheduledTrigger-3691-thread-2) [    ] o.a.s.c.a.ScheduledTrigger ScheduledTrigger ran but was already closed
   [junit4]   2> 1483476 ERROR (ScheduledTrigger-3691-thread-2) [    ] o.a.s.c.a.ScheduledTriggers Unexpected exception from trigger: .scheduled_maintenance
   [junit4]   2> java.lang.RuntimeException: Trigger has been closed
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTrigger.run(ScheduledTrigger.java:155) ~[java/:?]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.run(ScheduledTriggers.java:588) [java/:?]
   [junit4]   2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 1483491 DEBUG (simCloudManagerPool-3690-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1483491 DEBUG (simCloudManagerPool-3690-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 15
   [junit4]   2> 1483491 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 15
   [junit4]   2> 1483491 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 1483491 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10016_solr, 127.0.0.1:10021_solr]
   [junit4]   2> 1483492 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=10, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
   [junit4]   2> 1483492 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion 15
   [junit4]   2> 1483492 DEBUG (ScheduledTrigger-3691-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483492 DEBUG (ScheduledTrigger-3691-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483492 DEBUG (ScheduledTrigger-3691-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10022_solr at time 29703499699450559
   [junit4]   2> 1483496 DEBUG (ScheduledTrigger-3691-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483512 DEBUG (ScheduledTrigger-3691-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483512 DEBUG (ScheduledTrigger-3691-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483516 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483533 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483533 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483536 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483553 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483553 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483556 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483573 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483573 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483576 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483594 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483594 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483597 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483614 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483614 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483617 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483634 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483634 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483637 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483655 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483655 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483657 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483675 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483675 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483677 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1483695 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483695 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1483695 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10022_solr] added at times [29703499699450559], now=29703509880923259
   [junit4]   2> 1483696 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"69872cf42bf2bfTe116fkt8tfq9pffeimnm8zr02",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29703499699450559,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703499699450559],
   [junit4]   2>     "nodeNames":["127.0.0.1:10022_solr"]}}
   [junit4]   2> 1483696 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
   [junit4]   2> 1483739 DEBUG (simCloudManagerPool-3690-thread-3) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10022_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1483748 DEBUG (ScheduledTrigger-3691-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10022_solr
   [junit4]   2> 1483748 DEBUG (AutoscalingActionExecutor-3692-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"69872cf42bf2bfTe116fkt8tfq9pffeimnm8zr02",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29703499699450559,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703499699450559],
   [junit4]   2>     "_enqueue_time_":29703509929516359,
   [junit4]   2>     "nodeNames":["127.0.0.1:10022_solr"]}}
   [junit4]   2> 1483763 DEBUG (simCloudManagerPool-3690-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10022_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1483774 DEBUG (AutoscalingActionExecutor-3692-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 1483774 DEBUG (AutoscalingActionExecutor-3692-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
   [junit4]   2> 1483774 DEBUG (AutoscalingActionExecutor-3692-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 1315 ms for event id=69872cf42bf2bfTe116fkt8tfq9pffeimnm8zr02
   [junit4]   2> 1483875 DEBUG (ScheduledTrigger-3691-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1483875 DEBUG (ScheduledTrigger-3691-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483875 DEBUG (ScheduledTrigger-3691-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1483875 DEBUG (ScheduledTrigger-3691-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10023_solr at time 29703518847671609
   [junit4]   2> 1483895 DEBUG (ScheduledTrigger-3691-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1483895 DEBUG (ScheduledTrigger-3691-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483895 DEBUG (ScheduledTrigger-3691-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1483967 INFO  (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1483967 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1483968 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1483968 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1483968 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1483971 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1483971 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1483971 DEBUG (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1483977 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 15
   [junit4]   2> 1483977 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion -1
   [junit4]   2> 1483977 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 15
   [junit4]   2> 1483977 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10022_solr, 127.0.0.1:10016_solr, 127.0.0.1:10021_solr, 127.0.0.1:10023_solr]
   [junit4]   2> 1483984 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 1483984 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10022_solr, 127.0.0.1:10016_solr, 127.0.0.1:10021_solr, 127.0.0.1:10023_solr]
   [junit4]   2> 1483984 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=10, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
   [junit4]   2> 1483984 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:10023_solr
   [junit4]   2> 1483984 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10023_solr
   [junit4]   2> 1483985 DEBUG (ScheduledTrigger-3694-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1483990 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion 15
   [junit4]   2> 1483990 DEBUG (ScheduledTrigger-3694-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1483990 DEBUG (ScheduledTrigger-3694-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1484005 DEBUG (ScheduledTrigger-3694-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1484010 DEBUG (ScheduledTrigger-3694-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1484010 DEBUG (ScheduledTrigger-3694-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1484025 DEBUG (ScheduledTrigger-3694-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1484031 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1484031 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1484045 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1484051 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1484051 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1484066 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1484071 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1484071 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1484086 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
   [junit4]   2> 1484091 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 1484091 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
   [junit4]   2> 1484091 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10023_solr] added at times [29703518847671609], now=29703529674733009
   [junit4]   2> 1484091 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"698731697e9d39Te116fkt8tfq9pffeimnm8zr05",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29703518847671609,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703518847671609],
   [junit4]   2>     "nodeNames":["127.0.0.1:10023_solr"]}}
   [junit4]   2> 1484092 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
   [junit4]   2> 1484106 DEBUG (simCloudManagerPool-3693-thread-1) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node3":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10022_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1484115 DEBUG (ScheduledTrigger-3694-thread-4) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10023_solr
   [junit4]   2> 1484115 DEBUG (AutoscalingActionExecutor-3695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"698731697e9d39Te116fkt8tfq9pffeimnm8zr05",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":29703518847671609,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703518847671609],
   [junit4]   2>     "_enqueue_time_":29703529700632759,
   [junit4]   2>     "nodeNames":["127.0.0.1:10023_solr"]}}
   [junit4]   2> 1484116 INFO  (TEST-TestTriggerIntegration.testEventFromRestoredState-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
   [junit4]   2> ############ CLUSTER STATE ############
   [junit4]   2> #######################################
   [junit4]   2> ## Live nodes: 4
   [junit4]   2> ## Empty nodes: 3
   [junit4]   2> ## Dead nodes: 0
   [junit4]   2> ## Collections: [.system]
   [junit4]   2> ## Max replicas per node: 1
   [junit4]   2> ## Min replicas per node: 0
   [junit4]   2> ## Total replicas: 1
   [junit4]   2> ## * .system 1
   [junit4]   2> ## - active           1
   [junit4]   2> ######### Solr op counts ##########
   [junit4]   2> ## - autoscaling        2
   [junit4]   2> ## - update             3
   [junit4]   2> ######### Autoscaling event counts ###########
   [junit4]   2> ## * Trigger: node_added_trigger
   [junit4]   2> ## - STARTED         2
   [junit4]   2> ## - SUCCEEDED       1
   [junit4]   2>
   [junit4]   2> 1484168 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 16
   [junit4]   2> 1484169 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10022_solr
   [junit4]   2> 1484169 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10016_solr
   [junit4]   2> 1484169 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10021_solr
   [junit4]   2> 1484169 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10022_solr
   [junit4]   2> 1484169 INFO  (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1484169 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1484169 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1484169 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1484169 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1484170 WARN  (AutoscalingActionExecutor-3695-thread-1) [    ] o.a.s.c.a.SystemLogListener Exception sending event to collection .system
   [junit4]   2> java.io.IOException: java.lang.InterruptedException
   [junit4]   2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:632) ~[test/:?]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.SystemLogListener.onEvent(SystemLogListener.java:118) ~[java/:?]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:789) ~[java/:?]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:756) ~[java/:?]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:321) ~[java/:?]
   [junit4]   2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> Caused by: java.lang.InterruptedException
   [junit4]   2> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404) ~[?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:1.8.0_172]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:630) ~[test/:?]
   [junit4]   2> ... 10 more
   [junit4]   2> 1484171 DEBUG (AutoscalingActionExecutor-3695-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 2777 ms for event id=698731697e9d39Te116fkt8tfq9pffeimnm8zr05
   [junit4]   2> 1484171 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1484171 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1484171 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1484172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1484172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 17
   [junit4]   2> 1484172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 17, lastZnodeVersion -1
   [junit4]   2> 1484172 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 17
   [junit4]   2> 1484182 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10023_solr]
   [junit4]   2> 1484184 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1484184 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 17, lastZnodeVersion 17
   [junit4]   2> 1484185 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1484205 DEBUG (ScheduledTrigger-3697-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1484225 DEBUG (ScheduledTrigger-3697-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1484275 DEBUG (ScheduledTrigger-3697-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1484295 DEBUG (ScheduledTrigger-3697-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1484298 DEBUG (simCloudManagerPool-3696-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1484298 DEBUG (simCloudManagerPool-3696-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 18
   [junit4]   2> 1484299 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 18
   [junit4]   2> 1484299 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1484299 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 18, lastZnodeVersion 18
   [junit4]   2> 1484311 INFO  (simCloudManagerPool-3696-thread-2) [    ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
   [junit4]   2> 1484311 DEBUG (simCloudManagerPool-3696-thread-2) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 1484311 DEBUG (simCloudManagerPool-3696-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1484311 DEBUG (simCloudManagerPool-3696-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 19
   [junit4]   2> 1484311 INFO  (simCloudManagerPool-3696-thread-2) [    ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
   [junit4]   2> 1484311 DEBUG (simCloudManagerPool-3696-thread-2) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 1484312 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 19
   [junit4]   2> 1484312 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10021_solr, 127.0.0.1:10023_solr]
   [junit4]   2> 1484312 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
   [junit4]   2> 1484312 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 19, lastZnodeVersion 19
   [junit4]   2> 1484312 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 1484312 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 1484316 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1484332 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 1484332 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1484332 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10024_solr at time 29703541729499959
   [junit4]   2> 1484336 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1484353 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 1484353 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1484353 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:10024_solr] added at times [29703541729499959], now=29703542747135659
   [junit4]   2> 1484353 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"698736bd5b9b37Te116fkt8tfq9pffeimnm8zr08",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":29703541729499959,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703541729499959],
   [junit4]   2>     "nodeNames":["127.0.0.1:10024_solr"]}}
   [junit4]   2> 1484353 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger1]
   [junit4]   2> 1484400 DEBUG (simCloudManagerPool-3696-thread-3) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node4":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10024_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1484414 DEBUG (ScheduledTrigger-3697-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10024_solr
   [junit4]   2> 1484414 DEBUG (AutoscalingActionExecutor-3698-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"698736bd5b9b37Te116fkt8tfq9pffeimnm8zr08",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":29703541729499959,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703541729499959],
   [junit4]   2>     "_enqueue_time_":29703542773519459,
   [junit4]   2>     "nodeNames":["127.0.0.1:10024_solr"]}}
   [junit4]   2> 1484414 INFO  (AutoscalingActionExecutor-3698-thread-1) [    ] o.a.s.c.a.s.TestTriggerIntegration -- event: {
   [junit4]   2>   "id":"698736bd5b9b37Te116fkt8tfq9pffeimnm8zr08",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":29703541729499959,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[29703541729499959],
   [junit4]   2>     "_enqueue_time_":29703542773519459,
   [junit4]   2>     "nodeNames":["127.0.0.1:10024_solr"]}}
   [junit4]   2> 1484414 INFO  (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1484414 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1484415 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1484415 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1484415 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1484416 WARN  (AutoscalingActionExecutor-3698-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Exception executing actions
   [junit4]   2> org.apache.lucene.store.AlreadyClosedException: ScheduledTrigger node_added_trigger1 has been closed.
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.dequeue(ScheduledTriggers.java:545) ~[java/:?]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:318) ~[java/:?]
   [junit4]   2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
   [junit4]   2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 1484416 DEBUG (AutoscalingActionExecutor-3698-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 101 ms for event id=698736bd5b9b37Te116fkt8tfq9pffeimnm8zr08
   [junit4]   2> 1484416 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1484416 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1484416 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A35E453ADC4267E8]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1484417 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.js

[...truncated too long message...]

le = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null592478687
     [copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null592478687
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null592478687

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1158087536
     [copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1158087536
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1158087536

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 196ms :: artifacts dl 13ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
        ---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 174 minutes 16 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 81 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/81/

2 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup

Error Message:
should be at least one inactive event

Stack Trace:
java.lang.AssertionError: should be at least one inactive event
        at __randomizedtesting.SeedInfo.seed([F057BA3F9E752B32:ED7B7A4DFF360C39]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup(ScheduledMaintenanceTriggerTest.java:224)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest.testBasic

Error Message:
{} expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: {} expected:<1> but was:<0>
        at __randomizedtesting.SeedInfo.seed([F057BA3F9E752B32:5BADA72A41A9AD1C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest.testBasic(SolrRrdBackendFactoryTest.java:90)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12525 lines...]
   [junit4] Suite: org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.metrics.rrd.SolrRrdBackendFactoryTest_F057BA3F9E752B32-001/init-core-data-001
   [junit4]   2> 506119 WARN  (SUITE-SolrRrdBackendFactoryTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=31 numCloses=31
   [junit4]   2> 506119 INFO  (SUITE-SolrRrdBackendFactoryTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 506121 INFO  (SUITE-SolrRrdBackendFactoryTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 506121 INFO  (SUITE-SolrRrdBackendFactoryTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 506210 INFO  (TEST-SolrRrdBackendFactoryTest.testBasic-seed#[F057BA3F9E752B32]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testBasic
   [junit4]   2> 506323 INFO  (TEST-SolrRrdBackendFactoryTest.testBasic-seed#[F057BA3F9E752B32]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testBasic
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SolrRrdBackendFactoryTest -Dtests.method=testBasic -Dtests.seed=F057BA3F9E752B32 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=es-CL -Dtests.timezone=CAT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.24s J2 | SolrRrdBackendFactoryTest.testBasic <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: {} expected:<1> but was:<0>
   [junit4]    > at __randomizedtesting.SeedInfo.seed([F057BA3F9E752B32:5BADA72A41A9AD1C]:0)
   [junit4]    > at org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest.testBasic(SolrRrdBackendFactoryTest.java:90)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.metrics.rrd.SolrRrdBackendFactoryTest_F057BA3F9E752B32-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@3004bc3e), locale=es-CL, timezone=CAT
   [junit4]   2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=45397472,total=447217664
   [junit4]   2> NOTE: All tests run in this JVM: [SpatialRPTFieldTypeTest, TestManagedResourceStorage, MetricsConfigTest, TestApiFramework, TestMultiWordSynonyms, DocValuesTest, ReplicaListTransformerTest, TestFieldCacheReopen, TestPolicyCloud, AutoAddReplicasPlanActionTest, TestDistributedMissingSort, HttpPartitionTest, DistributedQueueTest, ActionThrottleTest, ClusterStateTest, TestPolicyCloud, CircularListTest, SolrRrdBackendFactoryTest]
   [junit4] Completed [63/821 (1!)] on J2 in 0.58s, 1 test, 1 failure <<< FAILURES!

[...truncated 436 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_F057BA3F9E752B32-001/init-core-data-001
   [junit4]   2> 1431131 WARN  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=10 numCloses=10
   [junit4]   2> 1431132 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1431166 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1431166 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1431167 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_F057BA3F9E752B32-001/tempDir-001
   [junit4]   2> 1431167 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1431185 INFO  (Thread-1050) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1431185 INFO  (Thread-1050) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1431250 ERROR (Thread-1050) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1431396 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:46204
   [junit4]   2> 1431469 INFO  (zkConnectionManagerCallback-1052-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1431492 INFO  (jetty-launcher-1049-thread-1) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 1431547 INFO  (jetty-launcher-1049-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1431547 INFO  (jetty-launcher-1049-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1431547 INFO  (jetty-launcher-1049-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1431564 INFO  (jetty-launcher-1049-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@47a76f97{/solr,null,AVAILABLE}
   [junit4]   2> 1431646 INFO  (jetty-launcher-1049-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5e762bfb{SSL,[ssl, http/1.1]}{127.0.0.1:35194}
   [junit4]   2> 1431646 INFO  (jetty-launcher-1049-thread-1) [    ] o.e.j.s.Server Started @1431710ms
   [junit4]   2> 1431646 INFO  (jetty-launcher-1049-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35194}
   [junit4]   2> 1431646 ERROR (jetty-launcher-1049-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1431646 INFO  (jetty-launcher-1049-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1431646 INFO  (jetty-launcher-1049-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1431646 INFO  (jetty-launcher-1049-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1431646 INFO  (jetty-launcher-1049-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1431646 INFO  (jetty-launcher-1049-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-21T11:39:08.148Z
   [junit4]   2> 1431733 INFO  (zkConnectionManagerCallback-1054-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1431746 INFO  (jetty-launcher-1049-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1432502 INFO  (jetty-launcher-1049-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46204/solr
   [junit4]   2> 1432538 INFO  (zkConnectionManagerCallback-1058-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1432630 INFO  (zkConnectionManagerCallback-1060-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1433125 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1433126 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35194_solr
   [junit4]   2> 1433126 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.c.Overseer Overseer (id=72838688041664515-127.0.0.1:35194_solr-n_0000000000) starting
   [junit4]   2> 1433212 INFO  (zkConnectionManagerCallback-1067-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1433233 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46204/solr ready
   [junit4]   2> 1433270 DEBUG (OverseerAutoScalingTriggerThread-72838688041664515-127.0.0.1:35194_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1433271 DEBUG (OverseerAutoScalingTriggerThread-72838688041664515-127.0.0.1:35194_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 1433272 DEBUG (OverseerAutoScalingTriggerThread-72838688041664515-127.0.0.1:35194_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 1433272 DEBUG (OverseerAutoScalingTriggerThread-72838688041664515-127.0.0.1:35194_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 1433281 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35194_solr
   [junit4]   2> 1433307 INFO  (zkCallback-1059-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1433307 DEBUG (OverseerAutoScalingTriggerThread-72838688041664515-127.0.0.1:35194_solr-n_0000000000) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: []
   [junit4]   2> 1433308 DEBUG (OverseerAutoScalingTriggerThread-72838688041664515-127.0.0.1:35194_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1433308 DEBUG (ScheduledTrigger-2321-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 0
   [junit4]   2> 1433308 DEBUG (OverseerAutoScalingTriggerThread-72838688041664515-127.0.0.1:35194_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 1433362 INFO  (zkCallback-1066-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1433423 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1433739 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35194.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c5f321
   [junit4]   2> 1433867 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35194.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c5f321
   [junit4]   2> 1433867 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35194.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c5f321
   [junit4]   2> 1433868 INFO  (jetty-launcher-1049-thread-1) [n:127.0.0.1:35194_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_F057BA3F9E752B32-001/tempDir-001/node1/.
   [junit4]   2> 1434005 INFO  (zkConnectionManagerCallback-1070-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1434138 INFO  (zkConnectionManagerCallback-1075-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1434139 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1434140 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46204/solr ready
   [junit4]   2> 1434277 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10005_solr
   [junit4]   2> 1434278 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1434278 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 1434278 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 1434278 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 1434279 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1434279 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10005_solr]
   [junit4]   2> 1434280 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1434280 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 1434280 DEBUG (ScheduledTrigger-2330-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434301 DEBUG (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434372 DEBUG (ScheduledTrigger-2321-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434406 DEBUG (ScheduledTrigger-2330-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434443 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434480 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434516 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434554 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434571 INFO  (TEST-ScheduledMaintenanceTriggerTest.testInactiveShardCleanup-seed#[F057BA3F9E752B32]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testInactiveShardCleanup
   [junit4]   2> 1434589 DEBUG (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434609 DEBUG (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434772 DEBUG (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434837 DEBUG (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434857 DEBUG (ScheduledTrigger-2330-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434894 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1434931 DEBUG (simCloudManagerPool-2329-thread-5) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for ScheduledMaintenanceTriggerTest_collection1 / shard1: {"core_node1":{
   [junit4]   2>     "core":"ScheduledMaintenanceTriggerTest_collection1_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"ScheduledMaintenanceTriggerTest_collection1"}}
   [junit4]   2> 1435004 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435004 DEBUG (simCloudManagerPool-2329-thread-6) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for ScheduledMaintenanceTriggerTest_collection1 / shard1_1: {"core_node5":{
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>     "base_url":"http://127.0.0.1:10005/solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "core":"ScheduledMaintenanceTriggerTest_collection1_shard1_1_replica0",
   [junit4]   2>     "shard":"shard1_1",
   [junit4]   2>     "collection":"ScheduledMaintenanceTriggerTest_collection1"}}
   [junit4]   2> 1435005 DEBUG (simCloudManagerPool-2329-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for ScheduledMaintenanceTriggerTest_collection1 / shard1_0: {"core_node4":{
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":20480,
   [junit4]   2>     "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>     "base_url":"http://127.0.0.1:10005/solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":1.9073486328125E-5,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "core":"ScheduledMaintenanceTriggerTest_collection1_shard1_0_replica0",
   [junit4]   2>     "shard":"shard1_0",
   [junit4]   2>     "collection":"ScheduledMaintenanceTriggerTest_collection1"}}
   [junit4]   2> 1435040 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435113 DEBUG (ScheduledTrigger-2330-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435139 DEBUG (simCloudManagerPool-2329-thread-8) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1435139 DEBUG (simCloudManagerPool-2329-thread-8) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 1435202 DEBUG (ScheduledTrigger-2330-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435223 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 1435223 DEBUG (ScheduledTrigger-2330-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435223 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1435223 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 1435228 DEBUG (simCloudManagerPool-2329-thread-9) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1435228 DEBUG (simCloudManagerPool-2329-thread-9) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 1435228 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 1435228 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1435228 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 1435239 DEBUG (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.ScheduledTrigger ScheduledTrigger .scheduled_maintenance firing registered processor for scheduled time 2018-06-22T05:50:50.320Z, now=2018-06-22T05:51:36.334Z
   [junit4]   2> 1435259 DEBUG (ScheduledTrigger-2330-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435272 DEBUG (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"2a937977b32099T404d9u5tlnz465c0ra41w4vyp",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":11984098930401433,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{"actualEventTime":1529646696334}}
   [junit4]   2> 1435272 DEBUG (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, .scheduled_maintenance]
   [junit4]   2> 1435333 DEBUG (simCloudManagerPool-2329-thread-10) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1435334 INFO  (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=11984102029588133, stage=STARTED, actionName='null', event={
   [junit4]   2>   "id":"2a937977b32099T404d9u5tlnz465c0ra41w4vyp",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":11984098930401433,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1529646696334,
   [junit4]   2>     "_enqueue_time_":11984098971970533}}, context={}, config={
   [junit4]   2>   "trigger":".scheduled_maintenance",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":"inactive_shard_plan",
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":"inactive_shard_plan"}, message='null'}
   [junit4]   2> 1435349 DEBUG (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"2a937977b32099T404d9u5tlnz465c0ra41w4vyp",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":11984098930401433,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1529646696334,
   [junit4]   2>     "_enqueue_time_":11984098971970533}}
   [junit4]   2> 1435361 INFO  (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=11984103400658833, stage=BEFORE_ACTION, actionName='inactive_shard_plan', event={
   [junit4]   2>   "id":"2a937977b32099T404d9u5tlnz465c0ra41w4vyp",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":11984098930401433,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1529646696334,
   [junit4]   2>     "_enqueue_time_":11984098971970533}}, context={properties.BEFORE_ACTION=[inactive_shard_plan], source=.scheduled_maintenance}, config={
   [junit4]   2>   "trigger":".scheduled_maintenance",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":"inactive_shard_plan",
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":"inactive_shard_plan"}, message='null'}
   [junit4]   2> 1435363 DEBUG (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.InactiveShardPlanAction ScheduledMaintenanceTriggerTest_collection1/shard1: tstamp=1529646680111887450, time=1529646702562729550, delta=22
   [junit4]   2> 1435363 DEBUG (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.InactiveShardPlanAction -- delete inactive ScheduledMaintenanceTriggerTest_collection1 / shard1
   [junit4]   2> 1435434 INFO  (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=11984106240219833, stage=AFTER_ACTION, actionName='inactive_shard_plan', event={
   [junit4]   2>   "id":"2a937977b32099T404d9u5tlnz465c0ra41w4vyp",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":11984098930401433,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1529646696334,
   [junit4]   2>     "_enqueue_time_":11984098971970533}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$DeleteShard, method=GET, params.action=DELETESHARD, params.collection=ScheduledMaintenanceTriggerTest_collection1, params.shard=shard1}], properties.BEFORE_ACTION=[inactive_shard_plan], source=.scheduled_maintenance, properties.inactive_shard_plan={inactive={ScheduledMaintenanceTriggerTest_collection1=[shard1]}, cleanup={ScheduledMaintenanceTriggerTest_collection1=[shard1]}}, properties.AFTER_ACTION=[inactive_shard_plan]}, config={
   [junit4]   2>   "trigger":".scheduled_maintenance",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":"inactive_shard_plan",
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":"inactive_shard_plan"}, message='null'}
   [junit4]   2> 1435434 DEBUG (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.ExecutePlanAction -- processing event: {
   [junit4]   2>   "id":"2a937977b32099T404d9u5tlnz465c0ra41w4vyp",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":11984098930401433,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1529646696334,
   [junit4]   2>     "_enqueue_time_":11984098971970533}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$DeleteShard@41ffc1cf], inactive_shard_plan={inactive={ScheduledMaintenanceTriggerTest_collection1=[shard1]}, cleanup={ScheduledMaintenanceTriggerTest_collection1=[shard1]}}, AFTER_ACTION=[inactive_shard_plan], BEFORE_ACTION=[inactive_shard_plan, execute_plan]}
   [junit4]   2> 1435434 DEBUG (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.ExecutePlanAction Executing operation: action=DELETESHARD&collection=ScheduledMaintenanceTriggerTest_collection1&shard=shard1
   [junit4]   2> 1435435 DEBUG (ScheduledTrigger-2321-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435615 DEBUG (simCloudManagerPool-2329-thread-16) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10005_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1435615 INFO  (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest =======> CapturedEvent{timestamp=11984116100425783, stage=SUCCEEDED, actionName='null', event={
   [junit4]   2>   "id":"2a937977b32099T404d9u5tlnz465c0ra41w4vyp",
   [junit4]   2>   "source":".scheduled_maintenance",
   [junit4]   2>   "eventTime":11984098930401433,
   [junit4]   2>   "eventType":"SCHEDULED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "actualEventTime":1529646696334,
   [junit4]   2>     "_enqueue_time_":11984098971970533}}, context={}, config={
   [junit4]   2>   "trigger":".scheduled_maintenance",
   [junit4]   2>   "stage":[
   [junit4]   2>     "STARTED",
   [junit4]   2>     "ABORTED",
   [junit4]   2>     "SUCCEEDED",
   [junit4]   2>     "FAILED"],
   [junit4]   2>   "afterAction":"inactive_shard_plan",
   [junit4]   2>   "class":"org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest$CapturingTriggerListener",
   [junit4]   2>   "beforeAction":"inactive_shard_plan"}, message='null'}
   [junit4]   2> 1435615 DEBUG (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
   [junit4]   2> 1435615 DEBUG (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .scheduled_maintenance after 100ms
   [junit4]   2> 1435615 DEBUG (AutoscalingActionExecutor-2331-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 13325 ms for event id=2a937977b32099T404d9u5tlnz465c0ra41w4vyp
   [junit4]   2> 1435616 DEBUG (simCloudManagerPool-2329-thread-17) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1435616 DEBUG (simCloudManagerPool-2329-thread-17) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 1435649 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 1435686 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1435686 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 1435715 DEBUG (ScheduledTrigger-2330-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435747 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435751 DEBUG (simCloudManagerPool-2329-thread-18) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1435751 DEBUG (simCloudManagerPool-2329-thread-18) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 1435759 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 1435759 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1435759 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 1435763 INFO  (TEST-ScheduledMaintenanceTriggerTest.testInactiveShardCleanup-seed#[F057BA3F9E752B32]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testInactiveShardCleanup
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ScheduledMaintenanceTriggerTest -Dtests.method=testInactiveShardCleanup -Dtests.seed=F057BA3F9E752B32 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=uk-UA -Dtests.timezone=Indian/Mahe -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 1.28s J2 | ScheduledMaintenanceTriggerTest.testInactiveShardCleanup <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: should be at least one inactive event
   [junit4]    > at __randomizedtesting.SeedInfo.seed([F057BA3F9E752B32:ED7B7A4DFF360C39]:0)
   [junit4]    > at org.apache.solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest.testInactiveShardCleanup(ScheduledMaintenanceTriggerTest.java:224)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1435783 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435807 DEBUG (ScheduledTrigger-2330-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435828 DEBUG (ScheduledTrigger-2330-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435909 DEBUG (ScheduledTrigger-2330-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1435995 INFO  (TEST-ScheduledMaintenanceTriggerTest.testTriggerDefaults-seed#[F057BA3F9E752B32]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerDefaults
   [junit4]   2> 1435995 INFO  (TEST-ScheduledMaintenanceTriggerTest.testTriggerDefaults-seed#[F057BA3F9E752B32]) [    ] o.a.s.c.a.ScheduledMaintenanceTriggerTest {
   [junit4]   2>   "cluster-preferences":[
   [junit4]   2>     {
   [junit4]   2>       "minimize":"cores",
   [junit4]   2>       "precision":1},
   [junit4]   2>     {"maximize":"freedisk"}],
   [junit4]   2>   "triggers":{
   [junit4]   2>     ".scheduled_maintenance":{
   [junit4]   2>       "startTime":"NOW",
   [junit4]   2>       "event":"scheduled",
   [junit4]   2>       "every":"+1DAY",
   [junit4]   2>       "actions":[
   [junit4]   2>         {
   [junit4]   2>           "name":"inactive_shard_plan",
   [junit4]   2>           "class":"solr.InactiveShardPlanAction"},
   [junit4]   2>         {
   [junit4]   2>           "name":"execute_plan",
   [junit4]   2>           "class":"solr.ExecutePlanAction"}],
   [junit4]   2>       "enabled":true},
   [junit4]   2>     ".auto_add_replicas":{
   [junit4]   2>       "name":".auto_add_replicas",
   [junit4]   2>       "event":"nodeLost",
   [junit4]   2>       "waitFor":120,
   [junit4]   2>       "actions":[
   [junit4]   2>         {
   [junit4]   2>           "name":"auto_add_replicas_plan",
   [junit4]   2>           "class":"solr.AutoAddReplicasPlanAction"},
   [junit4]   2>         {
   [junit4]   2>           "name":"execute_plan",
   [junit4]   2>           "class":"solr.ExecutePlanAction"}],
   [junit4]   2>       "enabled":true}},
   [junit4]   2>   "listeners":{
   [junit4]   2>     ".scheduled_maintenance.system":{
   [junit4]   2>       "trigger":".scheduled_maintenance",
   [junit4]   2>       "afterAction":[],
   [junit4]   2>       "stage":[
   [junit4]   2>         "STARTED",
   [junit4]   2>         "ABORTED",
   [junit4]   2>         "SUCCEEDED",
   [junit4]   2>         "FAILED",
   [junit4]   2>         "BEFORE_ACTION",
   [junit4]   2>         "AFTER_ACTION",
   [junit4]   2>         "IGNORED"],
   [junit4]   2>       "class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
   [junit4]   2>       "beforeAction":[]},
   [junit4]   2>     ".auto_add_replicas.system":{
   [junit4]   2>       "trigger":".auto_add_replicas",
   [junit4]   2>       "afterAction":[],
   [junit4]   2>       "stage":[
   [junit4]   2>         "STARTED",
   [junit4]   2>         "ABORTED",
   [junit4]   2>         "SUCCEEDED",
   [junit4]   2>         "FAILED",
   [junit4]   2>         "BEFORE_ACTION",
   [junit4]   2>         "AFTER_ACTION",
   [junit4]   2>         "IGNORED"],
   [junit4]   2>       "class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
   [junit4]   2>       "beforeAction":[]}},
   [junit4]   2>   "properties":{}}
   [junit4]   2> 1436007 DEBUG (ScheduledTrigger-2330-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1436008 INFO  (TEST-ScheduledMaintenanceTriggerTest.testTriggerDefaults-seed#[F057BA3F9E752B32]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testTriggerDefaults
   [junit4]   2> 1436008 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1436009 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1436009 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1436009 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1436009 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1436019 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1436019 DEBUG (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1436137 INFO  (jetty-closer-1050-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@5e762bfb{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1436141 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=554313082
   [junit4]   2> 1436178 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1436178 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4e084578: rootName = solr_35194, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@15b3e449
   [junit4]   2> 1436195 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1436195 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2e8995e9: rootName = solr_35194, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@6328312c
   [junit4]   2> 1436251 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1436251 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3d421042: rootName = solr_35194, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2946d62e
   [junit4]   2> 1436289 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:35194_solr
   [junit4]   2> 1436291 INFO  (zkCallback-1059-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1436291 INFO  (zkCallback-1066-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1436291 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1436320 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1436320 INFO  (jetty-closer-1050-thread-1) [    ] o.a.s.c.Overseer Overseer (id=72838688041664515-127.0.0.1:35194_solr-n_0000000000) closing
   [junit4]   2> 1436320 INFO  (OverseerStateUpdate-72838688041664515-127.0.0.1:35194_solr-n_0000000000) [    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:35194_solr
   [junit4]   2> 1436361 WARN  (OverseerAutoScalingTriggerThread-72838688041664515-127.0.0.1:35194_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1436361 DEBUG (jetty-closer-1050-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1436361 DEBUG (jetty-closer-1050-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1436361 DEBUG (jetty-closer-1050-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1436361 DEBUG (jetty-closer-1050-thread-1) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1436362 DEBUG (jetty-closer-1050-thread-1) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1436362 DEBUG (jetty-closer-1050-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1436364 INFO  (jetty-closer-1050-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@47a76f97{/solr,null,UNAVAILABLE}
   [junit4]   2> 1436364 INFO  (jetty-closer-1050-thread-1) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1436413 ERROR (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1436414 INFO  (SUITE-ScheduledMaintenanceTriggerTest-seed#[F057BA3F9E752B32]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46204 46204
   [junit4]   2> 1436521 INFO  (Thread-1050) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46204 46204
   [junit4]   2> 1436568 WARN  (Thread-1050) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/aliases.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/clusterstate.json
   [junit4]   2> 3 /solr/clusterprops.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 3 /solr/live_nodes
   [junit4]   2> 3 /solr/collections
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.ScheduledMaintenanceTriggerTest_F057BA3F9E752B32-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1819, maxMBSortInHeap=7.400451758482797, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@3a1d7183), locale=uk-UA, timezone=Indian/Mahe
   [junit4]   2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=248162552,total=529530880
   [junit4]   2> NOTE: All tests run in this JVM: [SpatialRPTFieldTypeTest, TestManagedResourceStorage, MetricsConfigTest, TestApiFramework, TestMultiWordSynonyms, DocValuesTest, ReplicaListTransformerTest, TestFieldCacheReopen, TestPolicyCloud, AutoAddReplicasPlanActionTest, TestDistributedMissingSort, HttpPartitionTest, DistributedQueueTest, ActionThrottleTest, ClusterStateTest, TestPolicyCloud, CircularListTest, SolrRrdBackendFactoryTest, TestCloudJSONFacetJoinDomain, CoreAdminRequestStatusTest, TestRangeQuery, TestSafeXMLParsing, DistributedSuggestComponentTest, AddSchemaFieldsUpdateProcessorFactoryTest, RegexBoostProcessorTest, UtilsToolTest, ResourceLoaderTest, SuggestComponentContextFilterQueryTest, TestRandomDVFaceting, OpenExchangeRatesOrgProviderTest, TestCorePropertiesReload, ConnectionManagerTest, DeleteReplicaTest, TestSkipOverseerOperations, UniqFieldsUpdateProcessorFactoryTest, AnalyticsMergeStrategyTest, MetricsHandlerTest, BaseCdcrDistributedZkTest, SpatialFilterTest, SearchRateTriggerIntegrationTest, BasicDistributedZk2Test, TestMultiValuedNumericRangeQuery, FileBasedSpellCheckerTest, ShowFileRequestHandlerTest, TestJmxIntegration, ScheduledMaintenanceTriggerTest]
   [junit4] Completed [197/821 (2!)] on J2 in 5.59s, 2 tests, 1 failure <<< FAILURES!

[...truncated 52480 lines...]
[asciidoctor:convert] asciidoctor: ERROR: about-this-guide.adoc: line 1: invalid part, must have at least one section (e.g., chapter, appendix, etc.)
[asciidoctor:convert] asciidoctor: ERROR: solr-glossary.adoc: line 1: invalid part, must have at least one section (e.g., chapter, appendix, etc.)
     [java] Processed 2243 links (1792 relative) to 3127 anchors in 247 files
     [echo] Validated Links & Anchors via: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-ref-guide/bare-bones-html/

-documentation-lint:
    [jtidy] Checking for broken html (such as invalid tags)...
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/build/jtidy_tmp
     [echo] Checking for broken links...
     [exec]
     [exec] Crawl/parse...
     [exec]
     [exec] Verify...
     [echo] Checking for malformed docs...

jar-checksums:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

common.resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1602248555
     [copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1602248555
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1602248555

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1655124739
     [copy] Copying 247 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1655124739
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1655124739

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 28ms :: artifacts dl 4ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
        ---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 211 minutes 32 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 82 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/82/

3 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMixedBounds

Error Message:
failed to create testMixedBounds_collection Live Nodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr] Last available state: DocCollection(testMixedBounds_collection//clusterstate.json/29)={   "replicationFactor":"1",   "pullReplicas":"0",   "router":{"name":"compositeId"},   "maxShardsPerNode":"2",   "autoAddReplicas":"false",   "nrtReplicas":"2",   "tlogReplicas":"0",   "autoCreated":"true",   "shards":{     "shard2":{       "replicas":{         "core_node3":{           "core":"testMixedBounds_collection_shard2_replica_n3",           "leader":"true",           "SEARCHER.searcher.maxDoc":0,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":10240,           "node_name":"127.0.0.1:10001_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":9.5367431640625E-6,           "SEARCHER.searcher.numDocs":0},         "core_node4":{           "core":"testMixedBounds_collection_shard2_replica_n4",           "SEARCHER.searcher.maxDoc":0,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":10240,           "node_name":"127.0.0.1:10000_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":9.5367431640625E-6,           "SEARCHER.searcher.numDocs":0}},       "range":"0-7fffffff",       "state":"active"},     "shard1":{       "stateTimestamp":"1529724483343326950",       "replicas":{         "core_node1":{           "core":"testMixedBounds_collection_shard1_replica_n1",           "leader":"true",           "SEARCHER.searcher.maxDoc":495,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":257740,           "node_name":"127.0.0.1:10001_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":2.4003908038139343E-4,           "SEARCHER.searcher.numDocs":495},         "core_node2":{           "core":"testMixedBounds_collection_shard1_replica_n2",           "SEARCHER.searcher.maxDoc":495,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":257740,           "node_name":"127.0.0.1:10000_solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":2.4003908038139343E-4,           "SEARCHER.searcher.numDocs":495}},       "range":"80000000-ffffffff",       "state":"inactive"},     "shard1_1":{       "parent":"shard1",       "stateTimestamp":"1529724483377442800",       "range":"c0000000-ffffffff",       "state":"active",       "replicas":{         "core_node10":{           "leader":"true",           "core":"testMixedBounds_collection_shard1_1_replica1",           "SEARCHER.searcher.maxDoc":247,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":133740,           "node_name":"127.0.0.1:10000_solr",           "base_url":"http://127.0.0.1:10000/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":1.245550811290741E-4,           "SEARCHER.searcher.numDocs":247},         "core_node9":{           "core":"testMixedBounds_collection_shard1_1_replica0",           "SEARCHER.searcher.maxDoc":247,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":133740,           "node_name":"127.0.0.1:10001_solr",           "base_url":"http://127.0.0.1:10001/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":1.245550811290741E-4,           "SEARCHER.searcher.numDocs":247}}},     "shard1_0":{       "parent":"shard1",       "stateTimestamp":"1529724483377075950",       "range":"80000000-bfffffff",       "state":"active",       "replicas":{         "core_node7":{           "leader":"true",           "core":"testMixedBounds_collection_shard1_0_replica0",           "SEARCHER.searcher.maxDoc":248,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":144480,           "node_name":"127.0.0.1:10001_solr",           "base_url":"http://127.0.0.1:10001/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":1.3455748558044434E-4,           "SEARCHER.searcher.numDocs":248},         "core_node8":{           "core":"testMixedBounds_collection_shard1_0_replica1",           "SEARCHER.searcher.maxDoc":248,           "SEARCHER.searcher.deletedDocs":0,           "INDEX.sizeInBytes":144480,           "node_name":"127.0.0.1:10000_solr",           "base_url":"http://127.0.0.1:10000/solr",           "state":"active",           "type":"NRT",           "INDEX.sizeInGB":1.3455748558044434E-4,           "SEARCHER.searcher.numDocs":248}}}}}

Stack Trace:
java.lang.AssertionError: failed to create testMixedBounds_collection
Live Nodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
Last available state: DocCollection(testMixedBounds_collection//clusterstate.json/29)={
  "replicationFactor":"1",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "shards":{
    "shard2":{
      "replicas":{
        "core_node3":{
          "core":"testMixedBounds_collection_shard2_replica_n3",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10001_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0},
        "core_node4":{
          "core":"testMixedBounds_collection_shard2_replica_n4",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10000_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0}},
      "range":"0-7fffffff",
      "state":"active"},
    "shard1":{
      "stateTimestamp":"1529724483343326950",
      "replicas":{
        "core_node1":{
          "core":"testMixedBounds_collection_shard1_replica_n1",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":495,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":257740,
          "node_name":"127.0.0.1:10001_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.4003908038139343E-4,
          "SEARCHER.searcher.numDocs":495},
        "core_node2":{
          "core":"testMixedBounds_collection_shard1_replica_n2",
          "SEARCHER.searcher.maxDoc":495,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":257740,
          "node_name":"127.0.0.1:10000_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.4003908038139343E-4,
          "SEARCHER.searcher.numDocs":495}},
      "range":"80000000-ffffffff",
      "state":"inactive"},
    "shard1_1":{
      "parent":"shard1",
      "stateTimestamp":"1529724483377442800",
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node10":{
          "leader":"true",
          "core":"testMixedBounds_collection_shard1_1_replica1",
          "SEARCHER.searcher.maxDoc":247,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":133740,
          "node_name":"127.0.0.1:10000_solr",
          "base_url":"http://127.0.0.1:10000/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.245550811290741E-4,
          "SEARCHER.searcher.numDocs":247},
        "core_node9":{
          "core":"testMixedBounds_collection_shard1_1_replica0",
          "SEARCHER.searcher.maxDoc":247,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":133740,
          "node_name":"127.0.0.1:10001_solr",
          "base_url":"http://127.0.0.1:10001/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.245550811290741E-4,
          "SEARCHER.searcher.numDocs":247}}},
    "shard1_0":{
      "parent":"shard1",
      "stateTimestamp":"1529724483377075950",
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "leader":"true",
          "core":"testMixedBounds_collection_shard1_0_replica0",
          "SEARCHER.searcher.maxDoc":248,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":144480,
          "node_name":"127.0.0.1:10001_solr",
          "base_url":"http://127.0.0.1:10001/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.3455748558044434E-4,
          "SEARCHER.searcher.numDocs":248},
        "core_node8":{
          "core":"testMixedBounds_collection_shard1_0_replica1",
          "SEARCHER.searcher.maxDoc":248,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":144480,
          "node_name":"127.0.0.1:10000_solr",
          "base_url":"http://127.0.0.1:10000/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.3455748558044434E-4,
          "SEARCHER.searcher.numDocs":248}}}}}
        at __randomizedtesting.SeedInfo.seed([634A4A5FCDAB9F5E:69C9F5F280109404]:0)
        at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:69)
        at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMixedBounds(IndexSizeTriggerTest.java:582)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.TimeoutException: last state: DocCollection(testMixedBounds_collection//clusterstate.json/29)={
  "replicationFactor":"1",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"2",
  "autoAddReplicas":"false",
  "nrtReplicas":"2",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "shards":{
    "shard2":{
      "replicas":{
        "core_node3":{
          "core":"testMixedBounds_collection_shard2_replica_n3",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10001_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0},
        "core_node4":{
          "core":"testMixedBounds_collection_shard2_replica_n4",
          "SEARCHER.searcher.maxDoc":0,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":10240,
          "node_name":"127.0.0.1:10000_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":9.5367431640625E-6,
          "SEARCHER.searcher.numDocs":0}},
      "range":"0-7fffffff",
      "state":"active"},
    "shard1":{
      "stateTimestamp":"1529724483343326950",
      "replicas":{
        "core_node1":{
          "core":"testMixedBounds_collection_shard1_replica_n1",
          "leader":"true",
          "SEARCHER.searcher.maxDoc":495,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":257740,
          "node_name":"127.0.0.1:10001_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.4003908038139343E-4,
          "SEARCHER.searcher.numDocs":495},
        "core_node2":{
          "core":"testMixedBounds_collection_shard1_replica_n2",
          "SEARCHER.searcher.maxDoc":495,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":257740,
          "node_name":"127.0.0.1:10000_solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":2.4003908038139343E-4,
          "SEARCHER.searcher.numDocs":495}},
      "range":"80000000-ffffffff",
      "state":"inactive"},
    "shard1_1":{
      "parent":"shard1",
      "stateTimestamp":"1529724483377442800",
      "range":"c0000000-ffffffff",
      "state":"active",
      "replicas":{
        "core_node10":{
          "leader":"true",
          "core":"testMixedBounds_collection_shard1_1_replica1",
          "SEARCHER.searcher.maxDoc":247,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":133740,
          "node_name":"127.0.0.1:10000_solr",
          "base_url":"http://127.0.0.1:10000/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.245550811290741E-4,
          "SEARCHER.searcher.numDocs":247},
        "core_node9":{
          "core":"testMixedBounds_collection_shard1_1_replica0",
          "SEARCHER.searcher.maxDoc":247,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":133740,
          "node_name":"127.0.0.1:10001_solr",
          "base_url":"http://127.0.0.1:10001/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.245550811290741E-4,
          "SEARCHER.searcher.numDocs":247}}},
    "shard1_0":{
      "parent":"shard1",
      "stateTimestamp":"1529724483377075950",
      "range":"80000000-bfffffff",
      "state":"active",
      "replicas":{
        "core_node7":{
          "leader":"true",
          "core":"testMixedBounds_collection_shard1_0_replica0",
          "SEARCHER.searcher.maxDoc":248,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":144480,
          "node_name":"127.0.0.1:10001_solr",
          "base_url":"http://127.0.0.1:10001/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.3455748558044434E-4,
          "SEARCHER.searcher.numDocs":248},
        "core_node8":{
          "core":"testMixedBounds_collection_shard1_0_replica1",
          "SEARCHER.searcher.maxDoc":248,
          "SEARCHER.searcher.deletedDocs":0,
          "INDEX.sizeInBytes":144480,
          "node_name":"127.0.0.1:10000_solr",
          "base_url":"http://127.0.0.1:10000/solr",
          "state":"active",
          "type":"NRT",
          "INDEX.sizeInGB":1.3455748558044434E-4,
          "SEARCHER.searcher.numDocs":248}}}}}
        at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:111)
        at org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:63)
        ... 40 more


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger

Error Message:
number of ops expected:<2> but was:<1>

Stack Trace:
java.lang.AssertionError: number of ops expected:<2> but was:<1>
        at __randomizedtesting.SeedInfo.seed([634A4A5FCDAB9F5E:817CDD5464EC73]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:189)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeLostTriggerRestoreState

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
        at __randomizedtesting.SeedInfo.seed([634A4A5FCDAB9F5E:48B59F0457D38A8E]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeLostTriggerRestoreState(TestTriggerIntegration.java:324)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 14316 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_634A4A5FCDAB9F5E-001/init-core-data-001
   [junit4]   2> 3688690 WARN  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=8 numCloses=8
   [junit4]   2> 3688690 INFO  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 3688692 INFO  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 3688693 INFO  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 3688693 INFO  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_634A4A5FCDAB9F5E-001/tempDir-001
   [junit4]   2> 3688694 INFO  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3688694 INFO  (Thread-5827) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3688694 INFO  (Thread-5827) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3688696 ERROR (Thread-5827) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 3688794 INFO  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:53190
   [junit4]   2> 3688796 INFO  (zkConnectionManagerCallback-7590-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3688807 INFO  (jetty-launcher-7587-thread-1) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 3688808 INFO  (jetty-launcher-7587-thread-2) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 3688809 INFO  (jetty-launcher-7587-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 3688809 INFO  (jetty-launcher-7587-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 3688809 INFO  (jetty-launcher-7587-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 3688809 INFO  (jetty-launcher-7587-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4c966ddf{/solr,null,AVAILABLE}
   [junit4]   2> 3688810 INFO  (jetty-launcher-7587-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@696914b5{SSL,[ssl, http/1.1]}{127.0.0.1:51482}
   [junit4]   2> 3688810 INFO  (jetty-launcher-7587-thread-1) [    ] o.e.j.s.Server Started @3688973ms
   [junit4]   2> 3688810 INFO  (jetty-launcher-7587-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51482}
   [junit4]   2> 3688810 ERROR (jetty-launcher-7587-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3688810 INFO  (jetty-launcher-7587-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 3688810 INFO  (jetty-launcher-7587-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 3688811 INFO  (jetty-launcher-7587-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3688811 INFO  (jetty-launcher-7587-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3688811 INFO  (jetty-launcher-7587-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-23T03:27:12.390Z
   [junit4]   2> 3688817 INFO  (jetty-launcher-7587-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 3688817 INFO  (jetty-launcher-7587-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 3688817 INFO  (jetty-launcher-7587-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 3688817 INFO  (zkConnectionManagerCallback-7592-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3688818 INFO  (jetty-launcher-7587-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3688818 INFO  (jetty-launcher-7587-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1a239678{/solr,null,AVAILABLE}
   [junit4]   2> 3688819 INFO  (jetty-launcher-7587-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2e12bbf1{SSL,[ssl, http/1.1]}{127.0.0.1:50167}
   [junit4]   2> 3688819 INFO  (jetty-launcher-7587-thread-2) [    ] o.e.j.s.Server Started @3688981ms
   [junit4]   2> 3688819 INFO  (jetty-launcher-7587-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=50167}
   [junit4]   2> 3688819 ERROR (jetty-launcher-7587-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3688819 INFO  (jetty-launcher-7587-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 3688819 INFO  (jetty-launcher-7587-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 3688819 INFO  (jetty-launcher-7587-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3688819 INFO  (jetty-launcher-7587-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3688819 INFO  (jetty-launcher-7587-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-23T03:27:12.398Z
   [junit4]   2> 3688824 INFO  (zkConnectionManagerCallback-7594-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3688824 INFO  (jetty-launcher-7587-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3688988 INFO  (jetty-launcher-7587-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53190/solr
   [junit4]   2> 3688996 INFO  (zkConnectionManagerCallback-7598-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3688998 INFO  (zkConnectionManagerCallback-7600-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3689177 INFO  (jetty-launcher-7587-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53190/solr
   [junit4]   2> 3689181 INFO  (zkConnectionManagerCallback-7606-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3689185 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x106faba7c630005, likely client has closed socket
   [junit4]   2> 3689195 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3689196 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51482_solr
   [junit4]   2> 3689197 INFO  (zkConnectionManagerCallback-7608-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3689197 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.c.Overseer Overseer (id=74022122755915780-127.0.0.1:51482_solr-n_0000000000) starting
   [junit4]   2> 3689209 INFO  (zkConnectionManagerCallback-7615-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3689209 INFO  (jetty-launcher-7587-thread-2) [n:127.0.0.1:50167_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3689210 INFO  (jetty-launcher-7587-thread-2) [n:127.0.0.1:50167_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50167_solr
   [junit4]   2> 3689211 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53190/solr ready
   [junit4]   2> 3689213 INFO  (zkCallback-7599-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3689213 INFO  (zkCallback-7607-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3689216 INFO  (zkCallback-7614-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3689219 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3689219 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51482_solr
   [junit4]   2> 3689224 INFO  (zkCallback-7607-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3689224 INFO  (zkCallback-7599-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3689224 INFO  (zkCallback-7614-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3689237 DEBUG (OverseerAutoScalingTriggerThread-74022122755915780-127.0.0.1:51482_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 3689238 DEBUG (OverseerAutoScalingTriggerThread-74022122755915780-127.0.0.1:51482_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 3689239 DEBUG (OverseerAutoScalingTriggerThread-74022122755915780-127.0.0.1:51482_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 3689239 DEBUG (OverseerAutoScalingTriggerThread-74022122755915780-127.0.0.1:51482_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 3689239 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 3689249 INFO  (zkConnectionManagerCallback-7621-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3689250 INFO  (jetty-launcher-7587-thread-2) [n:127.0.0.1:50167_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 3689253 DEBUG (OverseerAutoScalingTriggerThread-74022122755915780-127.0.0.1:51482_solr-n_0000000000) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:50167_solr, 127.0.0.1:51482_solr]
   [junit4]   2> 3689253 INFO  (jetty-launcher-7587-thread-2) [n:127.0.0.1:50167_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53190/solr ready
   [junit4]   2> 3689260 INFO  (jetty-launcher-7587-thread-2) [n:127.0.0.1:50167_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 3689273 DEBUG (OverseerAutoScalingTriggerThread-74022122755915780-127.0.0.1:51482_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3689273 DEBUG (OverseerAutoScalingTriggerThread-74022122755915780-127.0.0.1:51482_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 3689283 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51482.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c83a9e
   [junit4]   2> 3689287 DEBUG (ScheduledTrigger-12524-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689295 INFO  (jetty-launcher-7587-thread-2) [n:127.0.0.1:50167_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50167.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c83a9e
   [junit4]   2> 3689298 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51482.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c83a9e
   [junit4]   2> 3689299 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51482.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c83a9e
   [junit4]   2> 3689300 INFO  (jetty-launcher-7587-thread-1) [n:127.0.0.1:51482_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_634A4A5FCDAB9F5E-001/tempDir-001/node1/.
   [junit4]   2> 3689315 INFO  (jetty-launcher-7587-thread-2) [n:127.0.0.1:50167_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50167.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c83a9e
   [junit4]   2> 3689315 INFO  (jetty-launcher-7587-thread-2) [n:127.0.0.1:50167_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50167.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@c83a9e
   [junit4]   2> 3689317 INFO  (jetty-launcher-7587-thread-2) [n:127.0.0.1:50167_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.IndexSizeTriggerTest_634A4A5FCDAB9F5E-001/tempDir-001/node2/.
   [junit4]   2> 3689379 INFO  (zkConnectionManagerCallback-7624-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3689382 INFO  (zkConnectionManagerCallback-7629-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3689383 INFO  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 3689384 INFO  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53190/solr ready
   [junit4]   2> 3689439 DEBUG (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
   [junit4]   2> 3689440 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 3689440 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 3689441 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 3689441 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 3689447 INFO  (SUITE-IndexSizeTriggerTest-seed#[634A4A5FCDAB9F5E]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 3689448 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 3689448 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3689448 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 3689449 DEBUG (ScheduledTrigger-12537-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689470 DEBUG (ScheduledTrigger-12537-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689491 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689513 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689533 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689553 DEBUG (ScheduledTrigger-12537-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689573 DEBUG (ScheduledTrigger-12537-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689593 DEBUG (ScheduledTrigger-12537-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689614 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689634 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689655 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689675 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689693 INFO  (TEST-IndexSizeTriggerTest.testMixedBounds-seed#[634A4A5FCDAB9F5E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testMixedBounds
   [junit4]   2> 3689695 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689717 DEBUG (ScheduledTrigger-12537-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689738 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689760 DEBUG (ScheduledTrigger-12537-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689779 DEBUG (simCloudManagerPool-12536-thread-7) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testMixedBounds_collection / shard1: {"core_node1":{
   [junit4]   2>     "core":"testMixedBounds_collection_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testMixedBounds_collection"}}
   [junit4]   2> 3689780 DEBUG (ScheduledTrigger-12537-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689800 DEBUG (ScheduledTrigger-12537-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689803 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689804 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689804 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689804 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689815 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689816 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689816 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689816 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689816 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689817 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689817 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689820 DEBUG (ScheduledTrigger-12537-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689824 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689824 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689830 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689831 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689831 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689832 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689833 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689833 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689833 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689834 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689834 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689835 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689835 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689835 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689836 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689836 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689837 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689838 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689838 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689839 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689841 DEBUG (ScheduledTrigger-12537-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3689855 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10001_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n4",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10000_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"0-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 3689857 DEBUG (simCloudManagerPool-12536-thread-8) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard2:{
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node3":{
   [junit4]   2>       "core":"testMixedBounds_collection_shard2_replica_n3",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":

[...truncated too long message...]

:: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null475742672
     [copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null475742672
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null475742672

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null251273796
     [copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null251273796
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null251273796

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 44ms :: artifacts dl 2ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
        ---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 176 minutes 29 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 83 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/83/

4 tests failed.
FAILED:  org.apache.solr.cloud.AddReplicaTest.test

Error Message:
core_node6:{"core":"addreplicatest_coll_shard1_replica_n5","base_url":"https://127.0.0.1:43801/solr","node_name":"127.0.0.1:43801_solr","state":"active","type":"NRT","force_set_state":"false"}

Stack Trace:
java.lang.AssertionError: core_node6:{"core":"addreplicatest_coll_shard1_replica_n5","base_url":"https://127.0.0.1:43801/solr","node_name":"127.0.0.1:43801_solr","state":"active","type":"NRT","force_set_state":"false"}
        at __randomizedtesting.SeedInfo.seed([C03129E07D75B94:84572D44A92B366C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.AddReplicaTest.test(AddReplicaTest.java:85)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration

Error Message:
did not finish processing in time

Stack Trace:
java.lang.AssertionError: did not finish processing in time
        at __randomizedtesting.SeedInfo.seed([C03129E07D75B94:5FBA502EE5C6CE6E]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration(IndexSizeTriggerTest.java:427)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration

Error Message:
events: [CapturedEvent{timestamp=30199874855567514, stage=STARTED, actionName='null', event={   "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",   "source":"index_size_trigger2",   "eventTime":30199869746650314,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30199874847334314,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199876032314964, stage=BEFORE_ACTION, actionName='compute_plan', event={   "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",   "source":"index_size_trigger2",   "eventTime":30199869746650314,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30199874847334314,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199876087984514, stage=AFTER_ACTION, actionName='compute_plan', event={   "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",   "source":"index_size_trigger2",   "eventTime":30199869746650314,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30199874847334314,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199876110009064, stage=BEFORE_ACTION, actionName='execute_plan', event={   "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",   "source":"index_size_trigger2",   "eventTime":30199869746650314,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30199874847334314,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199883381681164, stage=AFTER_ACTION, actionName='execute_plan', event={   "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",   "source":"index_size_trigger2",   "eventTime":30199869746650314,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30199874847334314,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger2/6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n/0] in completed tasks},success=}], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199883412704014, stage=SUCCEEDED, actionName='null', event={   "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",   "source":"index_size_trigger2",   "eventTime":30199869746650314,   "eventType":"INDEXSIZE",   "properties":{     "__start__":1,     "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{},     "_enqueue_time_":30199874847334314,     "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199893725932164, stage=STARTED, actionName='null', event={   "id":"6b4aa37dfe086cTd0li4wkf29wzmsxbhziqx0g8q",   "source":"index_size_trigger2",   "eventTime":30199888561965164,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"base_url\":\"http://127.0.0.1:10007/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10008_solr\",\n    \"base_url\":\"http://127.0.0.1:10008/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":30199893713992164,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199893847188214, stage=BEFORE_ACTION, actionName='compute_plan', event={   "id":"6b4aa37dfe086cTd0li4wkf29wzmsxbhziqx0g8q",   "source":"index_size_trigger2",   "eventTime":30199888561965164,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"base_url\":\"http://127.0.0.1:10007/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10008_solr\",\n    \"base_url\":\"http://127.0.0.1:10008/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":30199893713992164,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199893992554314, stage=AFTER_ACTION, actionName='compute_plan', event={   "id":"6b4aa37dfe086cTd0li4wkf29wzmsxbhziqx0g8q",   "source":"index_size_trigger2",   "eventTime":30199888561965164,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"base_url\":\"http://127.0.0.1:10007/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10008_solr\",\n    \"base_url\":\"http://127.0.0.1:10008/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":30199893713992164,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199894023424714, stage=BEFORE_ACTION, actionName='execute_plan', event={   "id":"6b4aa37dfe086cTd0li4wkf29wzmsxbhziqx0g8q",   "source":"index_size_trigger2",   "eventTime":30199888561965164,   "eventType":"INDEXSIZE",   "properties":{     "__start__":2,     "aboveSize":{"testSplitIntegration_collection":[         "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"base_url\":\"http://127.0.0.1:10007/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",         "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10008_solr\",\n    \"base_url\":\"http://127.0.0.1:10008/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},     "_enqueue_time_":30199893713992164,     "requestedOps":[       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",       "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={   "trigger":"index_size_trigger2",   "stage":[     "STARTED",     "ABORTED",     "SUCCEEDED",     "FAILED"],   "afterAction":[     "compute_plan",     "execute_plan"],   "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",   "beforeAction":[     "compute_plan",     "execute_plan"]}, message='null'}] expected:<6> but was:<10>

Stack Trace:
java.lang.AssertionError: events: [CapturedEvent{timestamp=30199874855567514, stage=STARTED, actionName='null', event={
  "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",
  "source":"index_size_trigger2",
  "eventTime":30199869746650314,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30199874847334314,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199876032314964, stage=BEFORE_ACTION, actionName='compute_plan', event={
  "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",
  "source":"index_size_trigger2",
  "eventTime":30199869746650314,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30199874847334314,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199876087984514, stage=AFTER_ACTION, actionName='compute_plan', event={
  "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",
  "source":"index_size_trigger2",
  "eventTime":30199869746650314,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30199874847334314,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199876110009064, stage=BEFORE_ACTION, actionName='execute_plan', event={
  "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",
  "source":"index_size_trigger2",
  "eventTime":30199869746650314,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30199874847334314,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199883381681164, stage=AFTER_ACTION, actionName='execute_plan', event={
  "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",
  "source":"index_size_trigger2",
  "eventTime":30199869746650314,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30199874847334314,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.async=index_size_trigger2/6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n/0, params.waitForFinalState=true, params.collection=testSplitIntegration_collection, params.shard=shard1}], properties.BEFORE_ACTION=[compute_plan, execute_plan], properties.responses=[{status={state=completed,msg=found [index_size_trigger2/6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n/0] in completed tasks},success=}], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan, execute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199883412704014, stage=SUCCEEDED, actionName='null', event={
  "id":"6b4a9f1c831ccaTd0li4wkf29wzmsxbhziqx0g8n",
  "source":"index_size_trigger2",
  "eventTime":30199869746650314,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":1,
    "aboveSize":{"testSplitIntegration_collection":["{\"core_node1\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":25,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":22740,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":25,\n    \"__bytes__\":22740,\n    \"core\":\"testSplitIntegration_collection_shard1_replica_n1\",\n    \"__docs__\":25,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.117827534675598E-5,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{},
    "_enqueue_time_":30199874847334314,
    "requestedOps":["Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1\"}]}}"]}}, context={}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199893725932164, stage=STARTED, actionName='null', event={
  "id":"6b4aa37dfe086cTd0li4wkf29wzmsxbhziqx0g8q",
  "source":"index_size_trigger2",
  "eventTime":30199888561965164,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"base_url\":\"http://127.0.0.1:10007/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10008_solr\",\n    \"base_url\":\"http://127.0.0.1:10008/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "_enqueue_time_":30199893713992164,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199893847188214, stage=BEFORE_ACTION, actionName='compute_plan', event={
  "id":"6b4aa37dfe086cTd0li4wkf29wzmsxbhziqx0g8q",
  "source":"index_size_trigger2",
  "eventTime":30199888561965164,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"base_url\":\"http://127.0.0.1:10007/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10008_solr\",\n    \"base_url\":\"http://127.0.0.1:10008/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "_enqueue_time_":30199893713992164,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199893992554314, stage=AFTER_ACTION, actionName='compute_plan', event={
  "id":"6b4aa37dfe086cTd0li4wkf29wzmsxbhziqx0g8q",
  "source":"index_size_trigger2",
  "eventTime":30199888561965164,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"base_url\":\"http://127.0.0.1:10007/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10008_solr\",\n    \"base_url\":\"http://127.0.0.1:10008/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "_enqueue_time_":30199893713992164,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}, CapturedEvent{timestamp=30199894023424714, stage=BEFORE_ACTION, actionName='execute_plan', event={
  "id":"6b4aa37dfe086cTd0li4wkf29wzmsxbhziqx0g8q",
  "source":"index_size_trigger2",
  "eventTime":30199888561965164,
  "eventType":"INDEXSIZE",
  "properties":{
    "__start__":2,
    "aboveSize":{"testSplitIntegration_collection":[
        "{\"core_node10\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":12,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":16240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"base_url\":\"http://127.0.0.1:10007/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":12,\n    \"__bytes__\":16240,\n    \"__docs__\":12,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":1.5124678611755371E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_1_replica1\",\n    \"shard\":\"shard1_1\",\n    \"collection\":\"testSplitIntegration_collection\"}}",
        "{\"core_node7\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":13,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":26980,\n    \"node_name\":\"127.0.0.1:10008_solr\",\n    \"base_url\":\"http://127.0.0.1:10008/solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":13,\n    \"__bytes__\":26980,\n    \"__docs__\":13,\n    \"violationType\":\"aboveDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":2.512708306312561E-5,\n    \"core\":\"testSplitIntegration_collection_shard1_0_replica0\",\n    \"shard\":\"shard1_0\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "belowSize":{"testSplitIntegration_collection":["{\"core_node3\":{\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"node_name\":\"127.0.0.1:10007_solr\",\n    \"type\":\"NRT\",\n    \"SEARCHER.searcher.numDocs\":0,\n    \"__bytes__\":10240,\n    \"core\":\"testSplitIntegration_collection_shard2_replica_n3\",\n    \"__docs__\":0,\n    \"violationType\":\"belowDocs\",\n    \"state\":\"active\",\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"shard\":\"shard2\",\n    \"collection\":\"testSplitIntegration_collection\"}}"]},
    "_enqueue_time_":30199893713992164,
    "requestedOps":[
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_1\"}]}}",
      "Op{action=SPLITSHARD, hints={COLL_SHARD=[{\n  \"first\":\"testSplitIntegration_collection\",\n  \"second\":\"shard1_0\"}]}}"]}}, context={properties.operations=[{class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_1}, {class=org.apache.solr.client.solrj.request.CollectionAdminRequest$SplitShard, method=GET, params.action=SPLITSHARD, params.collection=testSplitIntegration_collection, params.shard=shard1_0}], properties.BEFORE_ACTION=[compute_plan, execute_plan], source=index_size_trigger2, properties.AFTER_ACTION=[compute_plan]}, config={
  "trigger":"index_size_trigger2",
  "stage":[
    "STARTED",
    "ABORTED",
    "SUCCEEDED",
    "FAILED"],
  "afterAction":[
    "compute_plan",
    "execute_plan"],
  "class":"org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest$CapturingTriggerListener",
  "beforeAction":[
    "compute_plan",
    "execute_plan"]}, message='null'}] expected:<6> but was:<10>
        at __randomizedtesting.SeedInfo.seed([C03129E07D75B94:358DABDE2828926A]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testSplitIntegration(IndexSizeTriggerTest.java:316)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode

Error Message:
trigger did not fire

Stack Trace:
java.lang.AssertionError: trigger did not fire
        at __randomizedtesting.SeedInfo.seed([C03129E07D75B94:ABEC0F3DC89AD48C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode(TestLargeCluster.java:308)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13045 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestLargeCluster_C03129E07D75B94-001/init-core-data-001
   [junit4]   2> 1531590 WARN  (SUITE-TestLargeCluster-seed#[C03129E07D75B94]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 1531591 INFO  (SUITE-TestLargeCluster-seed#[C03129E07D75B94]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1531592 INFO  (SUITE-TestLargeCluster-seed#[C03129E07D75B94]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1531601 INFO  (SUITE-TestLargeCluster-seed#[C03129E07D75B94]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1531637 DEBUG (SUITE-TestLargeCluster-seed#[C03129E07D75B94]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
   [junit4]   2> 1531638 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1531638 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 1531639 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 1531639 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 1531644 INFO  (SUITE-TestLargeCluster-seed#[C03129E07D75B94]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1531669 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
   [junit4]   2> 1531671 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1531671 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 1531678 DEBUG (ScheduledTrigger-3170-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 1531685 INFO  (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSearchRate
   [junit4]   2> 1531686 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 1531686 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 1531687 INFO  (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1531688 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 1531688 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1531688 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException: null
   [junit4]   2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
   [junit4]   2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
   [junit4]   2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
   [junit4]   2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
   [junit4]   2> 1531688 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 1531688 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 1531688 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 1531688 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 1531689 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 1531689 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1531692 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 1531693 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 1531696 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 1531696 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 1531697 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
   [junit4]   2> 1531698 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1531698 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 1531698 DEBUG (ScheduledTrigger-3175-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 1531722 DEBUG (ScheduledTrigger-3175-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 1531742 DEBUG (ScheduledTrigger-3175-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 1531763 DEBUG (ScheduledTrigger-3175-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 1531783 DEBUG (ScheduledTrigger-3175-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 1531803 DEBUG (ScheduledTrigger-3175-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 1531816 DEBUG (simCloudManagerPool-3174-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1531816 DEBUG (simCloudManagerPool-3174-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 1531817 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 1531818 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1531818 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 1531823 DEBUG (ScheduledTrigger-3175-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 1531842 DEBUG (simCloudManagerPool-3174-thread-2) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1531842 DEBUG (simCloudManagerPool-3174-thread-2) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 1531842 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 1531843 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 1531843 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1531843 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 1532096 DEBUG (simCloudManagerPool-3174-thread-26) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSearchRate / shard2: {"core_node18":{
   [junit4]   2>     "core":"testSearchRate_shard2_replica_n18",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10003_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard2",
   [junit4]   2>     "collection":"testSearchRate"}}
   [junit4]   2> 1532096 DEBUG (simCloudManagerPool-3174-thread-27) [    ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSearchRate / shard1: {"core_node1":{
   [junit4]   2>     "core":"testSearchRate_shard1_replica_n1",
   [junit4]   2>     "leader":"true",
   [junit4]   2>     "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>     "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>     "INDEX.sizeInBytes":10240,
   [junit4]   2>     "node_name":"127.0.0.1:10079_solr",
   [junit4]   2>     "state":"active",
   [junit4]   2>     "type":"NRT",
   [junit4]   2>     "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>     "SEARCHER.searcher.numDocs":0,
   [junit4]   2>     "shard":"shard1",
   [junit4]   2>     "collection":"testSearchRate"}}
   [junit4]   2> 1532102 INFO  (TEST-TestLargeCluster.testSearchRate-seed#[C03129E07D75B94]) [    ] o.a.s.c.a.s.TestLargeCluster Ready after 462 ms
   [junit4]   2> 1532110 DEBUG (simCloudManagerPool-3174-thread-28) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1532111 DEBUG (simCloudManagerPool-3174-thread-28) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 1532111 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 1532111 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 1532111 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1532112 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 1532115 WARN  (ScheduledTrigger-3175-thread-2) [    ] o.a.s.c.a.s.SimNodeStateProvider Invalid registry name: solr.core..system.shard1.replica_n1
   [junit4]   2> 1532128 DEBUG (simCloudManagerPool-3174-thread-29) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1532129 DEBUG (simCloudManagerPool-3174-thread-29) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 1532129 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 1532130 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 1532130 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1532130 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
   [junit4]   2> 1532138 WARN  (ScheduledTrigger-3175-thread-3) [    ] o.a.s.c.a.s.SimNodeStateProvider Invalid registry name: solr.core..system.shard1.replica_n1
   [junit4]   2> 1532160 WARN  (ScheduledTrigger-3175-thread-3) [    ] o.a.s.c.a.s.SimNodeStateProvider Invalid registry name: solr.core..system.shard1.replica_n1
   [junit4]   2> 1532182 WARN  (ScheduledTrigger-3175-thread-3) [    ] o.a.s.c.a.s.SimNodeStateProvider Invalid registry name: solr.core..system.shard1.replica_n1
   [junit4]   2> 1532205 WARN  (ScheduledTrigger-3175-thread-3) [    ] o.a.s.c.a.s.SimNodeStateProvider Invalid registry name: solr.core..system.shard1.replica_n1
   [junit4]   2> 1532228 WARN  (ScheduledTrigger-3175-thread-3) [    ] o.a.s.c.a.s.SimNodeStateProvider Invalid registry name: solr.core..system.shard1.replica_n1
   [junit4]   2> 1532235 DEBUG (ScheduledTrigger-3175-thread-3) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"6aec54b8ab5e51Td0li4wkf29wzmsxbhziqx06ni",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime":30096196150976081,
   [junit4]   2>   "eventType":"SEARCHRATE",
   [junit4]   2>   "properties":{
   [junit4]   2>     "hotReplicas":[
   [junit4]   2>       "{\"core_node1\":{\n    \"core\":\"testSearchRate_shard1_replica_n1\",\n    \"leader\":\"true\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10079_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}",
   [junit4]   2>       "{\"core_node4\":{\n    \"core\":\"testSearchRate_shard1_replica_n4\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10070_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}",
   [junit4]   2>       "{\"core_node9\":{\n    \"core\":\"testSearchRate_shard1_replica_n9\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10038_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}",
   [junit4]   2>       "{\"core_node5\":{\n    \"core\":\"testSearchRate_shard1_replica_n5\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10032_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}",
   [junit4]   2>       "{\"core_node3\":{\n    \"core\":\"testSearchRate_shard1_replica_n3\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10093_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}",
   [junit4]   2>       "{\"core_node10\":{\n    \"core\":\"testSearchRate_shard1_replica_n10\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10083_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}",
   [junit4]   2>       "{\"core_node6\":{\n    \"core\":\"testSearchRate_shard1_replica_n6\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10048_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}",
   [junit4]   2>       "{\"core_node8\":{\n    \"core\":\"testSearchRate_shard1_replica_n8\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10064_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}",
   [junit4]   2>       "{\"core_node2\":{\n    \"core\":\"testSearchRate_shard1_replica_n2\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10050_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}",
   [junit4]   2>       "{\"core_node7\":{\n    \"core\":\"testSearchRate_shard1_replica_n7\",\n    \"SEARCHER.searcher.maxDoc\":0,\n    \"SEARCHER.searcher.deletedDocs\":0,\n    \"INDEX.sizeInBytes\":10240,\n    \"rate\":4.0,\n    \"node_name\":\"127.0.0.1:10072_solr\",\n    \"state\":\"active\",\n    \"type\":\"NRT\",\n    \"QUERY./select.requestTimes:1minRate\":4,\n    \"INDEX.sizeInGB\":9.5367431640625E-6,\n    \"SEARCHER.searcher.numDocs\":0,\n    \"shard\":\"shard1\",\n    \"collection\":\"testSearchRate\"}}"],
   [junit4]   2>     "coldNodes":{},
   [junit4]   2>     "coldShards":{},
   [junit4]   2>     "coldReplicas":[],
   [junit4]   2>     "coldCollections":{},
   [junit4]   2>     "hotShards":{"testSearchRate":{"shard1":40.0}},
   [junit4]   2>     "hotCollections":{"testSearchRate":40.0},
   [junit4]   2>     "hotNodes":{
   [junit4]   2>       "127.0.0.1:10050_solr":4.0,
   [junit4]   2>       "127.0.0.1:10079_solr":4.0,
   [junit4]   2>       "127.0.0.1:10070_solr":4.0,
   [junit4]   2>       "127.0.0.1:10038_solr":4.0,
   [junit4]   2>       "127.0.0.1:10083_solr":4.0,
   [junit4]   2>       "127.0.0.1:10032_solr":4.0,
   [junit4]   2>       "127.0.0.1:10093_solr":4.0,
   [junit4]   2>       "127.0.0.1:10048_solr":4.0,
   [junit4]   2>       "127.0.0.1:10072_solr":4.0,
   [junit4]   2>       "127.0.0.1:10064_solr":4.0},
   [junit4]   2>     "requestedOps":[
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"testSearchRate\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"testSearchRate\",\n  \"second\":\"shard1\"}]}}",
   [junit4]   2>       "Op{action=ADDREPLICA, hints={COLL_SHARD=[{\n  \"first\":\"testSearchRate\",\n  \"second\":\"shard1\"}]}}"]}}
   [junit4]   2> 1532237 DEBUG (ScheduledTrigger-3175-thread-3) [    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [search_rate_trigger]
   [junit4]   2> 1532290 DEBUG (simCloudManagerPool-3174-thread-30) [    ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
   [junit4]   2>   "replicas":{"core_node1":{
   [junit4]   2>       "core":".system_shard1_replica_n1",
   [junit4]   2>       "SEARCHER.searcher.maxDoc":0,
   [junit4]   2>       "SEARCHER.searcher.deletedDocs":0,
   [junit4]   2>       "INDEX.sizeInBytes":10240,
   [junit4]   2>       "node_name":"127.0.0.1:10040_solr",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "type":"NRT",
   [junit4]   2>       "INDEX.sizeInGB":9.5367431640625E-6,
   [junit4]   2>       "SEARCHER.searcher.numDocs":0}},
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active"}
   [junit4]   2> 1532301 DEBUG (AutoscalingActionExecutor-3176-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"6aec54b8ab5e51Td0li4wkf29wzmsxbhziqx06ni",
   [junit4]   2>   "source":"search_rate_trigger",
   [junit4]   2>   "eventTime"

[...truncated too long message...]

file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1547893374
     [copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1547893374
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null1547893374

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null790984326
     [copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null790984326
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null790984326

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 84ms :: artifacts dl 6ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
        ---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 181 minutes 28 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 84 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/84/

2 tests failed.
FAILED:  org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest.testBasic

Error Message:
{} expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: {} expected:<1> but was:<0>
        at __randomizedtesting.SeedInfo.seed([60DC9B6165AE25D3:CB268674BA72A3FD]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest.testBasic(SolrRrdBackendFactoryTest.java:90)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.client.solrj.io.stream.StreamDecoratorTest.testUpdateStream

Error Message:
Error from server at http://127.0.0.1:33216/solr: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qnr-0000000043

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33216/solr: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qnr-0000000043
        at __randomizedtesting.SeedInfo.seed([3A3F22E5E426824E:F278E69FED2E7F1E]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.client.solrj.io.stream.StreamDecoratorTest.testUpdateStream(StreamDecoratorTest.java:2394)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12780 lines...]
   [junit4] Suite: org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest
   [junit4]   2> 1111477 INFO  (SUITE-SolrRrdBackendFactoryTest-seed#[60DC9B6165AE25D3]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.metrics.rrd.SolrRrdBackendFactoryTest_60DC9B6165AE25D3-001/init-core-data-001
   [junit4]   2> 1111478 WARN  (SUITE-SolrRrdBackendFactoryTest-seed#[60DC9B6165AE25D3]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=27 numCloses=27
   [junit4]   2> 1111478 INFO  (SUITE-SolrRrdBackendFactoryTest-seed#[60DC9B6165AE25D3]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1111479 INFO  (SUITE-SolrRrdBackendFactoryTest-seed#[60DC9B6165AE25D3]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1111552 INFO  (TEST-SolrRrdBackendFactoryTest.testBasic-seed#[60DC9B6165AE25D3]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testBasic
   [junit4]   2> 1111619 INFO  (TEST-SolrRrdBackendFactoryTest.testBasic-seed#[60DC9B6165AE25D3]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testBasic
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SolrRrdBackendFactoryTest -Dtests.method=testBasic -Dtests.seed=60DC9B6165AE25D3 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=zh-CN -Dtests.timezone=Africa/Ouagadougou -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.16s J2 | SolrRrdBackendFactoryTest.testBasic <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: {} expected:<1> but was:<0>
   [junit4]    > at __randomizedtesting.SeedInfo.seed([60DC9B6165AE25D3:CB268674BA72A3FD]:0)
   [junit4]    > at org.apache.solr.metrics.rrd.SolrRrdBackendFactoryTest.testBasic(SolrRrdBackendFactoryTest.java:90)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J2/temp/solr.metrics.rrd.SolrRrdBackendFactoryTest_60DC9B6165AE25D3-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1686, maxMBSortInHeap=6.46385235553262, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@740cfbbf), locale=zh-CN, timezone=Africa/Ouagadougou
   [junit4]   2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=240570696,total=527433728
   [junit4]   2> NOTE: All tests run in this JVM: [TestZkAclsWithHadoopAuth, BasicAuthStandaloneTest, TestTolerantUpdateProcessorRandomCloud, TestPullReplicaErrorHandling, TestSortableTextField, TestQueryTypes, TestInfoStreamLogging, SolrTestCaseJ4Test, TestSafeXMLParsing, TestManagedSynonymGraphFilterFactory, UnloadDistributedZkTest, TestPayloadCheckQParserPlugin, TestCloudPseudoReturnFields, DeleteInactiveReplicaTest, TimeZoneUtilsTest, CollectionTooManyReplicasTest, SearchRateTriggerIntegrationTest, TestUniqueKeyFieldResource, HdfsBasicDistributedZkTest, TestUpdate, PreAnalyzedFieldManagedSchemaCloudTest, AssignBackwardCompatibilityTest, PeerSyncWithIndexFingerprintCachingTest, BufferStoreTest, TestReqParamsAPI, TestNumericTokenStream, SpellingQueryConverterTest, TestCodecSupport, SolrCmdDistributorTest, TestSolrConfigHandler, TestManagedSchema, RecoveryZkTest, DirectSolrSpellCheckerTest, SearchHandlerTest, ChaosMonkeySafeLeaderWithPullReplicasTest, SolrRrdBackendFactoryTest]
   [junit4] Completed [123/821 (1!)] on J2 in 0.46s, 1 test, 1 failure <<< FAILURES!

[...truncated 3289 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.io.stream.StreamDecoratorTest
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/init-core-data-001
   [junit4]   2> 14693 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 14695 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 14695 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 14699 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001
   [junit4]   2> 14714 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 14719 INFO  (Thread-2) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 14719 INFO  (Thread-2) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 14819 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:35628
   [junit4]   2> 14834 ERROR (Thread-2) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 14934 INFO  (zkConnectionManagerCallback-6-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 15014 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x102dda6e7570000, likely client has closed socket
   [junit4]   2> 15430 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 15437 INFO  (jetty-launcher-3-thread-3) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 15437 INFO  (jetty-launcher-3-thread-4) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 15444 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
   [junit4]   2> 15549 INFO  (jetty-launcher-3-thread-3) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 15549 INFO  (jetty-launcher-3-thread-3) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 15553 INFO  (jetty-launcher-3-thread-3) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 15573 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 15573 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 15573 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 15588 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e29c6dc{/solr,null,AVAILABLE}
   [junit4]   2> 15589 INFO  (jetty-launcher-3-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3fe18418{/solr,null,AVAILABLE}
   [junit4]   2> 15633 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 15633 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 15634 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 15645 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@53cac00c{/solr,null,AVAILABLE}
   [junit4]   2> 15869 INFO  (jetty-launcher-3-thread-4) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 15869 INFO  (jetty-launcher-3-thread-4) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 15869 INFO  (jetty-launcher-3-thread-4) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 15873 INFO  (jetty-launcher-3-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3521fd2a{/solr,null,AVAILABLE}
   [junit4]   2> 15878 INFO  (jetty-launcher-3-thread-4) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4e218bb6{HTTP/1.1,[http/1.1]}{127.0.0.1:33216}
   [junit4]   2> 15878 INFO  (jetty-launcher-3-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5bc5beae{HTTP/1.1,[http/1.1]}{127.0.0.1:40736}
   [junit4]   2> 15878 INFO  (jetty-launcher-3-thread-4) [    ] o.e.j.s.Server Started @15921ms
   [junit4]   2> 15878 INFO  (jetty-launcher-3-thread-3) [    ] o.e.j.s.Server Started @15921ms
   [junit4]   2> 15878 INFO  (jetty-launcher-3-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40736}
   [junit4]   2> 15884 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2bffa70{HTTP/1.1,[http/1.1]}{127.0.0.1:35748}
   [junit4]   2> 15885 INFO  (jetty-launcher-3-thread-2) [    ] o.e.j.s.Server Started @15928ms
   [junit4]   2> 15885 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35748}
   [junit4]   2> 15885 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@24b796ea{HTTP/1.1,[http/1.1]}{127.0.0.1:34830}
   [junit4]   2> 15885 INFO  (jetty-launcher-3-thread-1) [    ] o.e.j.s.Server Started @15929ms
   [junit4]   2> 15885 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34830}
   [junit4]   2> 15878 INFO  (jetty-launcher-3-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33216}
   [junit4]   2> 15921 INFO  (jetty-launcher-3-thread-3) [    ] o.a.s.u.c.SSLCredentialProviderFactory Processing SSL Credential Provider chain: env;sysprop
   [junit4]   2> 15925 ERROR (jetty-launcher-3-thread-4) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 15925 INFO  (jetty-launcher-3-thread-4) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 15926 INFO  (jetty-launcher-3-thread-4) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 15926 INFO  (jetty-launcher-3-thread-4) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 15926 INFO  (jetty-launcher-3-thread-4) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 15926 INFO  (jetty-launcher-3-thread-4) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-26T00:00:37.744Z
   [junit4]   2> 15927 INFO  (jetty-launcher-3-thread-4) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 15931 ERROR (jetty-launcher-3-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 15931 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 15931 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 15931 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 15931 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 15931 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-26T00:00:37.749Z
   [junit4]   2> 15941 ERROR (jetty-launcher-3-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 15941 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 15941 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 15941 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 15941 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 15941 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-26T00:00:37.759Z
   [junit4]   2> 15946 ERROR (jetty-launcher-3-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 15946 INFO  (jetty-launcher-3-thread-3) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 15946 INFO  (jetty-launcher-3-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 15946 INFO  (jetty-launcher-3-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 15946 INFO  (jetty-launcher-3-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 15946 INFO  (jetty-launcher-3-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-26T00:00:37.764Z
   [junit4]   2> 15977 INFO  (zkConnectionManagerCallback-14-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 15993 INFO  (zkConnectionManagerCallback-10-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 16017 INFO  (zkConnectionManagerCallback-12-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 16019 INFO  (zkConnectionManagerCallback-8-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 16019 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 16019 INFO  (jetty-launcher-3-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 16025 INFO  (jetty-launcher-3-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 16027 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 16100 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x102dda6e7570004, likely client has closed socket
   [junit4]   2> 16474 INFO  (jetty-launcher-3-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35628/solr
   [junit4]   2> 16539 INFO  (jetty-launcher-3-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35628/solr
   [junit4]   2> 16588 INFO  (zkConnectionManagerCallback-20-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 16629 INFO  (zkConnectionManagerCallback-22-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 16774 INFO  (jetty-launcher-3-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35628/solr
   [junit4]   2> 16791 INFO  (zkConnectionManagerCallback-26-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 16809 INFO  (zkConnectionManagerCallback-25-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 16826 INFO  (zkConnectionManagerCallback-30-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 16905 INFO  (zkConnectionManagerCallback-32-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 17039 WARN  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
   [junit4]   2> 17059 WARN  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
   [junit4]   2> 17063 WARN  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
   [junit4]   2> 17099 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 17102 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34830_solr
   [junit4]   2> 17103 INFO  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 17105 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.c.Overseer Overseer (id=72864252906897418-127.0.0.1:34830_solr-n_0000000000) starting
   [junit4]   2> 17110 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 17114 INFO  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33216_solr
   [junit4]   2> 17123 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35748_solr
   [junit4]   2> 17172 INFO  (zkCallback-24-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 17173 INFO  (zkCallback-31-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 17173 INFO  (zkCallback-23-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 17213 INFO  (jetty-launcher-3-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35628/solr
   [junit4]   2> 17245 INFO  (zkConnectionManagerCallback-42-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 17325 INFO  (zkConnectionManagerCallback-59-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 17328 INFO  (zkConnectionManagerCallback-54-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 17335 INFO  (zkConnectionManagerCallback-55-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 17343 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 17344 INFO  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 17352 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35628/solr ready
   [junit4]   2> 17357 INFO  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35628/solr ready
   [junit4]   2> 17358 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 17360 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35628/solr ready
   [junit4]   2> 17371 INFO  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 17352 INFO  (zkConnectionManagerCallback-47-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 17399 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 17489 WARN  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
   [junit4]   2> 17490 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 17494 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 17500 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 17500 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40736_solr
   [junit4]   2> 17526 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35748.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 17533 INFO  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33216.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 17533 INFO  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33216.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 17533 INFO  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33216.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 17541 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35748.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 17541 INFO  (zkCallback-31-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 17541 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35748.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 17558 INFO  (zkCallback-53-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 17558 INFO  (zkCallback-46-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 17558 INFO  (zkCallback-58-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 17565 INFO  (jetty-launcher-3-thread-2) [n:127.0.0.1:35748_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node2/.
   [junit4]   2> 17566 INFO  (zkCallback-23-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 17567 INFO  (zkCallback-24-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 17571 INFO  (zkCallback-52-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 17584 INFO  (jetty-launcher-3-thread-4) [n:127.0.0.1:33216_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node4/.
   [junit4]   2> 17655 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 17655 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34830_solr
   [junit4]   2> 17662 INFO  (zkConnectionManagerCallback-68-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 17678 INFO  (zkCallback-53-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 17679 INFO  (zkCallback-23-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 17679 INFO  (zkCallback-52-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 17680 INFO  (zkCallback-31-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 17681 INFO  (zkCallback-24-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 17681 INFO  (zkCallback-46-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 17685 INFO  (zkCallback-58-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 17692 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 17695 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35628/solr ready
   [junit4]   2> 17709 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 17857 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 17915 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40736.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 17980 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34830.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 18000 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40736.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 18000 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40736.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 18006 INFO  (jetty-launcher-3-thread-3) [n:127.0.0.1:40736_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node3/.
   [junit4]   2> 18035 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34830.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 18036 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34830.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 18084 INFO  (jetty-launcher-3-thread-1) [n:127.0.0.1:34830_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node1/.
   [junit4]   2> 18313 INFO  (zkConnectionManagerCallback-72-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 18349 INFO  (zkConnectionManagerCallback-77-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 18351 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 18353 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35628/solr ready
   [junit4]   2> 18752 INFO  (qtp2112503148-97) [n:127.0.0.1:33216_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 18801 INFO  (OverseerThreadFactory-49-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 18967 INFO  (OverseerStateUpdate-72864252906897418-127.0.0.1:34830_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:35748/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 18993 INFO  (OverseerStateUpdate-72864252906897418-127.0.0.1:34830_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"collection1_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:34830/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 19201 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr    x:collection1_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 19202 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr    x:collection1_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 19207 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr    x:collection1_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 20268 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 20274 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 20302 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.IndexSchema [collection1_shard2_replica_n2] Schema name=test
   [junit4]   2> 20313 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema [collection1_shard1_replica_n1] Schema name=test
   [junit4]   2> 20510 WARN  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.GeoHashField]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 20533 WARN  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.LatLonType]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 20533 WARN  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.LatLonType]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 20543 WARN  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 20543 WARN  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 20544 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 20554 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n1' using configuration from collection collection1, trusted=true
   [junit4]   2> 20572 WARN  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 20572 WARN  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 20573 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 20573 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n2' using configuration from collection collection1, trusted=true
   [junit4]   2> 20577 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35748.solr.core.collection1.shard1.replica_n1' (registry 'solr.core.collection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 20577 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 20577 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [[collection1_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node2/collection1_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node2/./collection1_shard1_replica_n1/data/]
   [junit4]   2> 20579 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34830.solr.core.collection1.shard2.replica_n2' (registry 'solr.core.collection1.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 20580 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 20580 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrCore [[collection1_shard2_replica_n2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node1/collection1_shard2_replica_n2], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node1/./collection1_shard2_replica_n2/data/]
   [junit4]   2> 20742 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 20742 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 20744 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 20744 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 20747 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7b5419a3[collection1_shard1_replica_n1] main]
   [junit4]   2> 20754 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 20754 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 20756 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 20756 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 20758 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@64c2290f[collection1_shard2_replica_n2] main]
   [junit4]   2> 20760 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 20761 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 20763 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 20763 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 20764 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 20767 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604291125664808960
   [junit4]   2> 20779 INFO  (searcherExecutor-58-thread-1-processing-n:127.0.0.1:34830_solr x:collection1_shard2_replica_n2 c:collection1 s:shard2 r:core_node4) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrCore [collection1_shard2_replica_n2] Registered new searcher Searcher@64c2290f[collection1_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 20780 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 20781 INFO  (searcherExecutor-57-thread-1-processing-n:127.0.0.1:35748_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [collection1_shard1_replica_n1] Registered new searcher Searcher@7b5419a3[collection1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 20783 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604291125680537600
   [junit4]   2> 20833 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node3=0}, version=0}
   [junit4]   2> 20850 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node4=0}, version=0}
   [junit4]   2> 20884 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 20884 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 20885 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34830/solr/collection1_shard2_replica_n2/
   [junit4]   2> 20887 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 20887 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:34830/solr/collection1_shard2_replica_n2/ has no replicas
   [junit4]   2> 20887 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 20906 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 20906 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 20906 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35748/solr/collection1_shard1_replica_n1/
   [junit4]   2> 20907 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 20910 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:35748/solr/collection1_shard1_replica_n1/ has no replicas
   [junit4]   2> 20910 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 20952 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35748/solr/collection1_shard1_replica_n1/ shard1
   [junit4]   2> 20954 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34830/solr/collection1_shard2_replica_n2/ shard2
   [junit4]   2> 21121 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 21121 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 21129 INFO  (qtp2089967312-69) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1927
   [junit4]   2> 21138 INFO  (qtp1190851269-88) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1940
   [junit4]   2> 21202 INFO  (qtp2112503148-97) [n:127.0.0.1:33216_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 21232 INFO  (zkCallback-24-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 21234 INFO  (zkCallback-31-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 22205 INFO  (qtp2112503148-97) [n:127.0.0.1:33216_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=3455
   [junit4]   2> 22214 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):90
   [junit4]   2> 22244 INFO  (SUITE-StreamDecoratorTest-seed#[3A3F22E5E426824E]-worker) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: collection1
   [junit4]   2> 22314 INFO  (TEST-StreamDecoratorTest.testParallelDaemonCommitStream-seed#[3A3F22E5E426824E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testParallelDaemonCommitStream
   [junit4]   2> 22461 INFO  (qtp1190851269-72) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node3=1}, version=1}
   [junit4]   2> 22462 INFO  (qtp1190851269-72) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=TOLEADER&distrib.from=http://127.0.0.1:34830/solr/collection1_shard2_replica_n2/&wt=javabin&version=2}{deleteByQuery=*:* (-1604291127428513792)} 0 14
   [junit4]   2> 22483 INFO  (qtp1190851269-87) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1604291127464165376,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 22483 INFO  (qtp1190851269-87) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 22487 INFO  (qtp2089967312-65) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1604291127468359680,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 22487 INFO  (qtp2089967312-65) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 22494 INFO  (qtp1190851269-87) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 22494 INFO  (qtp1190851269-87) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:34830/solr/collection1_shard2_replica_n2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 11
   [junit4]   2> 22499 INFO  (qtp2089967312-65) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 22499 INFO  (qtp2089967312-65) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:34830/solr/collection1_shard2_replica_n2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 12
   [junit4]   2> 22503 INFO  (qtp2089967312-64) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node4=1}, version=1}
   [junit4]   2> 22504 INFO  (qtp2089967312-64) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n2]  webapp=/solr path=/update params={_stateVer_=collection1:4&commit=true&wt=javabin&version=2}{deleteByQuery=*:* (-1604291127403347968),commit=} 0 172
   [junit4]   2> 22506 INFO  (qtp2112503148-92) [n:127.0.0.1:33216_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf&name=parallelDestinationCollection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 22519 INFO  (OverseerCollectionConfigSetProcessor-72864252906897418-127.0.0.1:34830_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 22519 INFO  (OverseerThreadFactory-49-thread-2) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection parallelDestinationCollection1
   [junit4]   2> 22743 INFO  (OverseerStateUpdate-72864252906897418-127.0.0.1:34830_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"parallelDestinationCollection1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"parallelDestinationCollection1_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:33216/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 22747 INFO  (OverseerStateUpdate-72864252906897418-127.0.0.1:34830_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"parallelDestinationCollection1",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"parallelDestinationCollection1_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:35748/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 22950 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr    x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=parallelDestinationCollection1_shard2_replica_n2&action=CREATE&numShards=2&collection=parallelDestinationCollection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 22951 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr    x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=parallelDestinationCollection1_shard1_replica_n1&action=CREATE&numShards=2&collection=parallelDestinationCollection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 22954 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr    x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 23993 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 23995 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 24034 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.s.IndexSchema [parallelDestinationCollection1_shard2_replica_n2] Schema name=test
   [junit4]   2> 24043 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.s.IndexSchema [parallelDestinationCollection1_shard1_replica_n1] Schema name=test
   [junit4]   2> 24188 WARN  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 24188 WARN  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 24188 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 24189 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'parallelDestinationCollection1_shard2_replica_n2' using configuration from collection parallelDestinationCollection1, trusted=true
   [junit4]   2> 24189 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35748.solr.core.parallelDestinationCollection1.shard2.replica_n2' (registry 'solr.core.parallelDestinationCollection1.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 24189 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 24190 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.SolrCore [[parallelDestinationCollection1_shard2_replica_n2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node2/parallelDestinationCollection1_shard2_replica_n2], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node2/./parallelDestinationCollection1_shard2_replica_n2/data/]
   [junit4]   2> 24339 WARN  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 24339 WARN  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 24339 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 24339 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'parallelDestinationCollection1_shard1_replica_n1' using configuration from collection parallelDestinationCollection1, trusted=true
   [junit4]   2> 24361 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33216.solr.core.parallelDestinationCollection1.shard1.replica_n1' (registry 'solr.core.parallelDestinationCollection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5eb5ee8e
   [junit4]   2> 24361 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 24362 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.SolrCore [[parallelDestinationCollection1_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node4/parallelDestinationCollection1_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamDecoratorTest_3A3F22E5E426824E-001/tempDir-001/node4/./parallelDestinationCollection1_shard1_replica_n1/data/]
   [junit4]   2> 24364 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 24364 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 24366 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 24366 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 24369 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@5a49dcb7[parallelDestinationCollection1_shard2_replica_n2] main]
   [junit4]   2> 24371 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 24385 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 24386 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 24417 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604291129492111360
   [junit4]   2> 24435 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/parallelDestinationCollection1/terms/shard2 to Terms{values={core_node4=0}, version=0}
   [junit4]   2> 24440 INFO  (searcherExecutor-67-thread-1-processing-n:127.0.0.1:35748_solr x:parallelDestinationCollection1_shard2_replica_n2 c:parallelDestinationCollection1 s:shard2 r:core_node4) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.SolrCore [parallelDestinationCollection1_shard2_replica_n2] Registered new searcher Searcher@5a49dcb7[parallelDestinationCollection1_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 24445 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 24445 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 24445 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35748/solr/parallelDestinationCollection1_shard2_replica_n2/
   [junit4]   2> 24445 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 24446 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:35748/solr/parallelDestinationCollection1_shard2_replica_n2/ has no replicas
   [junit4]   2> 24446 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 24451 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35748/solr/parallelDestinationCollection1_shard2_replica_n2/ shard2
   [junit4]   2> 24559 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 24559 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 24560 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 24560 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 24562 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@85d3657[parallelDestinationCollection1_shard1_replica_n1] main]
   [junit4]   2> 24564 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 24564 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 24565 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 24565 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604291129647300608
   [junit4]   2> 24568 INFO  (searcherExecutor-69-thread-1-processing-n:127.0.0.1:33216_solr x:parallelDestinationCollection1_shard1_replica_n1 c:parallelDestinationCollection1 s:shard1 r:core_node3) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.SolrCore [parallelDestinationCollection1_shard1_replica_n1] Registered new searcher Searcher@85d3657[parallelDestinationCollection1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 24577 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/parallelDestinationCollection1/terms/shard1 to Terms{values={core_node3=0}, version=0}
   [junit4]   2> 24584 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 24584 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 24584 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33216/solr/parallelDestinationCollection1_shard1_replica_n1/
   [junit4]   2> 24585 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 24585 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:33216/solr/parallelDestinationCollection1_shard1_replica_n1/ has no replicas
   [junit4]   2> 24585 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 24595 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33216/solr/parallelDestinationCollection1_shard1_replica_n1/ shard1
   [junit4]   2> 24605 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 24608 INFO  (qtp1190851269-84) [n:127.0.0.1:35748_solr c:parallelDestinationCollection1 s:shard2 r:core_node4 x:parallelDestinationCollection1_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=parallelDestinationCollection1_shard2_replica_n2&action=CREATE&numShards=2&collection=parallelDestinationCollection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1658
   [junit4]   2> 24712 INFO  (zkCallback-24-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/parallelDestinationCollection1/state.json] for collection [parallelDestinationCollection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 24751 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 24755 INFO  (qtp2112503148-93) [n:127.0.0.1:33216_solr c:parallelDestinationCollection1 s:shard1 r:core_node3 x:parallelDestinationCollection1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=parallelDestinationCollection1_shard1_replica_n1&action=CREATE&numShards=2&collection=parallelDestinationCollection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1804
   [junit4]   2> 24760 INFO  (qtp2112503148-92) [n:127.0.0.1:33216_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 24855 INFO  (zkCallback-23-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/parallelDestinationCollection1/state.json] for collection [parallelDestinationCollection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 24855 INFO  (zkCallback-24-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/parallelDestinationCollection1/state.json] for collection [parallelDestinationCollection1] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 25760 INFO  (qtp2112503148-92) [n:127.0.0.1:33216_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf&name=parallelDestinationCollection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=3254
   [junit4]   2> 25761 INFO  (TEST-StreamDecoratorTest.testParallelDaemonCommitStream-seed#[3A3F22E5E426824E]) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: parallelDestinationCollection1 failOnTimeout:true timeout (sec):90
   [junit4]   2> 25762 INFO  (TEST-StreamDecoratorTest.testParallelDaemonCommitStream-seed#[3A3F22E5E426824E]) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: parallelDestinationCollection1
   [junit4]   2> 25805 INFO  (qtp2089967312-68) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n2]  webapp=/solr path=/update params={_stateVer_=collection1:4&wt=javabin&version=2}{add=[2 (1604291130911883264), 3 (1604291130947534848)]} 0 36
   [junit4]   2> 25807 INFO  (qtp1190851269-87) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1]  webapp=/solr path=/update params={_stateVer_=collection1:4&wt=javabin&version=2}{add=[0 (1604291130908737536), 4 (1604291130947534848), 1 (1604291130948583424)]} 0 40
   [junit4]   2> 25814 INFO  (qtp2089967312-70) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1604291130956972032,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 25814 INFO  (qtp2089967312-70) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@14f261fa commitCommandVersion:1604291130956972032
   [junit4]   2> 25827 INFO  (qtp1190851269-90) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1604291130970603520,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 25830 INFO  (qtp1190851269-90) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@5b44ef3c commitCommandVersion:1604291130970603520
   [junit4]   2> 25901 INFO  (qtp1190851269-90) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@712c34a1[collection1_shard1_replica_n1] main]
   [junit4]   2> 25901 INFO  (qtp2089967312-70) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@301d760b[collection1_shard2_replica_n2] main]
   [junit4]   2> 25910 INFO  (qtp2089967312-70) [n:127.0.0.1:34830_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 25913 INFO  (qtp1190851269-90) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 25921 INFO  (searcherExecutor-57-thread-1-processing-n:127.0.0.1:35748_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:35748_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.S

[...truncated too long message...]

-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null364564582
     [copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null364564582
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/null364564582

resolve-example:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve-server:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/lucene/top-level-ivy-settings.xml

resolve:

jar-checksums:
    [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1362110496
     [copy] Copying 247 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1362110496
   [delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/null1362110496

check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 21ms :: artifacts dl 4ms
        ---------------------------------------------------------------------
        |                  |            modules            ||   artifacts   |
        |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
        ---------------------------------------------------------------------
        |      default     |   8   |   0   |   0   |   0   ||   8   |   0   |
        ---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...

-jenkins-base:

BUILD SUCCESSFUL
Total time: 219 minutes 44 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 85 - Failure

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/85/

6 tests failed.
FAILED:  org.apache.solr.cloud.LeaderVoteWaitTimeoutTest.testMostInSyncReplicasCanWinElection

Error Message:
Timeout waiting for new leader null Live Nodes: [127.0.0.1:40061_solr, 127.0.0.1:42415_solr, 127.0.0.1:47051_solr] Last available state: DocCollection(collection1//collections/collection1/state.json/14)={   "pullReplicas":"0",   "replicationFactor":"3",   "shards":{"shard1":{       "range":"80000000-7fffffff",       "state":"active",       "replicas":{         "core_node62":{           "core":"collection1_shard1_replica_n61",           "base_url":"https://127.0.0.1:42098/solr",           "node_name":"127.0.0.1:42098_solr",           "state":"down",           "type":"NRT",           "force_set_state":"false"},         "core_node64":{           "core":"collection1_shard1_replica_n63",           "base_url":"https://127.0.0.1:42415/solr",           "node_name":"127.0.0.1:42415_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false"},         "core_node66":{           "core":"collection1_shard1_replica_n65",           "base_url":"https://127.0.0.1:47051/solr",           "node_name":"127.0.0.1:47051_solr",           "state":"active",           "type":"NRT",           "force_set_state":"false"}}}},   "router":{"name":"compositeId"},   "maxShardsPerNode":"1",   "autoAddReplicas":"false",   "nrtReplicas":"3",   "tlogReplicas":"0"}

Stack Trace:
java.lang.AssertionError: Timeout waiting for new leader
null
Live Nodes: [127.0.0.1:40061_solr, 127.0.0.1:42415_solr, 127.0.0.1:47051_solr]
Last available state: DocCollection(collection1//collections/collection1/state.json/14)={
  "pullReplicas":"0",
  "replicationFactor":"3",
  "shards":{"shard1":{
      "range":"80000000-7fffffff",
      "state":"active",
      "replicas":{
        "core_node62":{
          "core":"collection1_shard1_replica_n61",
          "base_url":"https://127.0.0.1:42098/solr",
          "node_name":"127.0.0.1:42098_solr",
          "state":"down",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node64":{
          "core":"collection1_shard1_replica_n63",
          "base_url":"https://127.0.0.1:42415/solr",
          "node_name":"127.0.0.1:42415_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node66":{
          "core":"collection1_shard1_replica_n65",
          "base_url":"https://127.0.0.1:47051/solr",
          "node_name":"127.0.0.1:47051_solr",
          "state":"active",
          "type":"NRT",
          "force_set_state":"false"}}}},
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"3",
  "tlogReplicas":"0"}
        at __randomizedtesting.SeedInfo.seed([CF82A18BD330C561:679EBD311170F14B]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:278)
        at org.apache.solr.cloud.LeaderVoteWaitTimeoutTest.testMostInSyncReplicasCanWinElection(LeaderVoteWaitTimeoutTest.java:189)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey

Error Message:
We think that split was successful but sub-shard states were not updated even after 2 minutes.

Stack Trace:
java.lang.AssertionError: We think that split was successful but sub-shard states were not updated even after 2 minutes.
        at __randomizedtesting.SeedInfo.seed([CF82A18BD330C561:44A5725A92366EE5]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:555)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.AutoScalingHandlerTest.testReadApi

Error Message:
expected:<2> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<2> but was:<0>
        at __randomizedtesting.SeedInfo.seed([CF82A18BD330C561:98AB5A3E08C2277A]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at org.apache.solr.cloud.autoscaling.AutoScalingHandlerTest.testReadApi(AutoScalingHandlerTest.java:845)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger

Error Message:
number of ops expected:<2> but was:<1>

Stack Trace:
java.lang.AssertionError: number of ops expected:<2> but was:<1>
        at __randomizedtesting.SeedInfo.seed([CF82A18BD330C561:AC4997094AFFB64C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testTrigger(IndexSizeTriggerTest.java:189)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.testDeleteNode

Error Message:
unexpected DELETENODE status: {responseHeader={status=0,QTime=8},status={state=notfound,msg=Did not find [search_rate_trigger3/6bced90b5eb068T458wd97sfj1nhkezbh3g0jva1/0] in any tasks queue}}

Stack Trace:
java.lang.AssertionError: unexpected DELETENODE status: {responseHeader={status=0,QTime=8},status={state=notfound,msg=Did not find [search_rate_trigger3/6bced90b5eb068T458wd97sfj1nhkezbh3g0jva1/0] in any tasks queue}}
        at __randomizedtesting.SeedInfo.seed([CF82A18BD330C561:ED106F09E4FA4A1C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.lambda$testDeleteNode$6(SearchRateTriggerIntegrationTest.java:670)
        at java.util.ArrayList.forEach(ArrayList.java:1257)
        at org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.testDeleteNode(SearchRateTriggerIntegrationTest.java:662)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode

Error Message:
trigger did not fire

Stack Trace:
java.lang.AssertionError: trigger did not fire
        at __randomizedtesting.SeedInfo.seed([CF82A18BD330C561:686DBC281C7D4A79]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode(TestLargeCluster.java:308)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12367 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestLargeCluster_CF82A18BD330C561-001/init-core-data-001
   [junit4]   2> 111606 WARN  (SUITE-TestLargeCluster-seed#[CF82A18BD330C561]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=8 numCloses=8
   [junit4]   2> 111606 INFO  (SUITE-TestLargeCluster-seed#[CF82A18BD330C561]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 111608 INFO  (SUITE-TestLargeCluster-seed#[CF82A18BD330C561]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 111610 INFO  (SUITE-TestLargeCluster-seed#[CF82A18BD330C561]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 111678 DEBUG (SUITE-TestLargeCluster-seed#[CF82A18BD330C561]-worker) [    ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
   [junit4]   2> 111687 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 111687 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 111689 INFO  (SUITE-TestLargeCluster-seed#[CF82A18BD330C561]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 111703 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 111703 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 111724 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
   [junit4]   2> 111727 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 111727 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
   [junit4]   2> 111730 DEBUG (ScheduledTrigger-130-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
   [junit4]   2> 111740 INFO  (TEST-TestLargeCluster.testAddNode-seed#[CF82A18BD330C561]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testAddNode
   [junit4]   2> 111740 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[CF82A18BD330C561]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 111740 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 111742 INFO  (TEST-TestLargeCluster.testAddNode-seed#[CF82A18BD330C561]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 111746 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 111746 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 111750 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[CF82A18BD330C561]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
   [junit4]   2> 111750 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[CF82A18BD330C561]) [    ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
   [junit4]   2> 111750 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[CF82A18BD330C561]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
   [junit4]   2> 111750 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[CF82A18BD330C561]) [    ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
   [junit4]   2> 111765 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[CF82A18BD330C561]) [    ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
   [junit4]   2> 111765 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[CF82A18BD330C561]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 111775 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 111775 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 111776 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 111776 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 111776 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr,