[JENKINS] Lucene-Solr-Tests-master - Build # 2242 - Still Failing

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2242 - Still Failing

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2242/

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.LeaderElectionTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.LeaderElectionTest:     1) Thread[id=1249, name=zkConnectionManagerCallback-325-thread-1, state=WAITING, group=TGRP-LeaderElectionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.LeaderElectionTest:
   1) Thread[id=1249, name=zkConnectionManagerCallback-325-thread-1, state=WAITING, group=TGRP-LeaderElectionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([6B76F5E1536ABDFC]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.LeaderElectionTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=1249, name=zkConnectionManagerCallback-325-thread-1, state=WAITING, group=TGRP-LeaderElectionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=1249, name=zkConnectionManagerCallback-325-thread-1, state=WAITING, group=TGRP-LeaderElectionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
        at __randomizedtesting.SeedInfo.seed([6B76F5E1536ABDFC]:0)


FAILED:  org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory

Error Message:
expected:<8> but was:<9>

Stack Trace:
java.lang.AssertionError: expected:<8> but was:<9>
        at __randomizedtesting.SeedInfo.seed([6B76F5E1536ABDFC:68A511CE92242FB]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:268)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 11757 lines...]
   [junit4] Suite: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/init-core-data-001
   [junit4]   2> 0    INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[6B76F5E1536ABDFC]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1443 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[6B76F5E1536ABDFC]-worker) [    ] o.e.j.u.log Logging initialized @20130ms
   [junit4]   2> 1530 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[6B76F5E1536ABDFC]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 2035 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[6B76F5E1536ABDFC]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 2288 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[6B76F5E1536ABDFC]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001
   [junit4]   2> 2378 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[6B76F5E1536ABDFC]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2429 INFO  (Thread-1) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2546 INFO  (Thread-1) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3220 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[6B76F5E1536ABDFC]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:47534
   [junit4]   2> 7463 ERROR (Thread-1) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 8298 INFO  (zkConnectionManagerCallback-4-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 13138 INFO  (jetty-launcher-1-thread-3) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 13140 INFO  (jetty-launcher-1-thread-2) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 13276 INFO  (jetty-launcher-1-thread-1) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 14310 INFO  (jetty-launcher-1-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@25a68c70{/solr,null,AVAILABLE}
   [junit4]   2> 14310 INFO  (jetty-launcher-1-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1e89d64e{/solr,null,AVAILABLE}
   [junit4]   2> 14310 INFO  (jetty-launcher-1-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@393974c8{/solr,null,AVAILABLE}
   [junit4]   2> 14409 INFO  (jetty-launcher-1-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7896be5b{HTTP/1.1,[http/1.1]}{127.0.0.1:39019}
   [junit4]   2> 14410 INFO  (jetty-launcher-1-thread-1) [    ] o.e.j.s.Server Started @33176ms
   [junit4]   2> 14410 INFO  (jetty-launcher-1-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39019}
   [junit4]   2> 14429 INFO  (jetty-launcher-1-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@18abff6e{HTTP/1.1,[http/1.1]}{127.0.0.1:45158}
   [junit4]   2> 14431 INFO  (jetty-launcher-1-thread-2) [    ] o.e.j.s.Server Started @33197ms
   [junit4]   2> 14430 INFO  (jetty-launcher-1-thread-3) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5e220b8{HTTP/1.1,[http/1.1]}{127.0.0.1:36739}
   [junit4]   2> 14431 INFO  (jetty-launcher-1-thread-3) [    ] o.e.j.s.Server Started @33197ms
   [junit4]   2> 14431 INFO  (jetty-launcher-1-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45158}
   [junit4]   2> 14431 INFO  (jetty-launcher-1-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36739}
   [junit4]   2> 14646 ERROR (jetty-launcher-1-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 14646 INFO  (jetty-launcher-1-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 14648 ERROR (jetty-launcher-1-thread-3) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 14648 ERROR (jetty-launcher-1-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 14649 INFO  (jetty-launcher-1-thread-3) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 14649 INFO  (jetty-launcher-1-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 14652 INFO  (jetty-launcher-1-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 14828 INFO  (jetty-launcher-1-thread-3) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 14828 INFO  (jetty-launcher-1-thread-3) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 15321 INFO  (jetty-launcher-1-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 15321 INFO  (jetty-launcher-1-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 15321 INFO  (jetty-launcher-1-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 15360 INFO  (jetty-launcher-1-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-12-28T11:26:30.136Z
   [junit4]   2> 15355 INFO  (jetty-launcher-1-thread-3) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-12-28T11:26:29.645Z
   [junit4]   2> 15450 INFO  (jetty-launcher-1-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-12-28T11:26:30.178Z
   [junit4]   2> 15608 INFO  (jetty-launcher-1-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 15683 INFO  (zkConnectionManagerCallback-6-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 15712 INFO  (zkConnectionManagerCallback-8-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 15713 INFO  (zkConnectionManagerCallback-10-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 15734 INFO  (jetty-launcher-1-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 15755 INFO  (jetty-launcher-1-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 15787 INFO  (jetty-launcher-1-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 23609 INFO  (jetty-launcher-1-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47534/solr
   [junit4]   2> 23628 INFO  (jetty-launcher-1-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47534/solr
   [junit4]   2> 23629 INFO  (jetty-launcher-1-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47534/solr
   [junit4]   2> 24081 INFO  (zkConnectionManagerCallback-20-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 24083 INFO  (zkConnectionManagerCallback-18-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 24100 INFO  (zkConnectionManagerCallback-22-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 24343 INFO  (zkConnectionManagerCallback-24-thread-1-processing-n:127.0.0.1:45158_solr) [n:127.0.0.1:45158_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 24401 INFO  (zkConnectionManagerCallback-26-thread-1-processing-n:127.0.0.1:36739_solr) [n:127.0.0.1:36739_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 24583 INFO  (zkConnectionManagerCallback-28-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 26380 INFO  (jetty-launcher-1-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 26393 INFO  (jetty-launcher-1-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39019_solr
   [junit4]   2> 26530 INFO  (jetty-launcher-1-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.Overseer Overseer (id=99251675344273416-127.0.0.1:39019_solr-n_0000000000) starting
   [junit4]   2> 27506 INFO  (jetty-launcher-1-thread-2) [n:127.0.0.1:45158_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 27544 INFO  (jetty-launcher-1-thread-2) [n:127.0.0.1:45158_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45158_solr
   [junit4]   2> 27657 INFO  (zkCallback-27-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 27783 INFO  (zkCallback-23-thread-1-processing-n:127.0.0.1:45158_solr) [n:127.0.0.1:45158_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 28185 INFO  (jetty-launcher-1-thread-3) [n:127.0.0.1:36739_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 28248 INFO  (jetty-launcher-1-thread-3) [n:127.0.0.1:36739_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 28268 INFO  (jetty-launcher-1-thread-3) [n:127.0.0.1:36739_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36739_solr
   [junit4]   2> 28341 INFO  (zkCallback-27-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 28342 INFO  (zkCallback-23-thread-1-processing-n:127.0.0.1:45158_solr) [n:127.0.0.1:45158_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 28388 INFO  (zkCallback-25-thread-1-processing-n:127.0.0.1:36739_solr) [n:127.0.0.1:36739_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 29876 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 29899 INFO  (jetty-launcher-1-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39019_solr
   [junit4]   2> 29905 INFO  (zkCallback-23-thread-1-processing-n:127.0.0.1:45158_solr) [n:127.0.0.1:45158_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 29905 INFO  (zkCallback-25-thread-1-processing-n:127.0.0.1:36739_solr) [n:127.0.0.1:36739_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 29908 INFO  (zkCallback-27-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 30066 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:36739_solr"} current state version: 0
   [junit4]   2> 30090 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:36739_solr
   [junit4]   2> 30285 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:39019_solr"} current state version: 0
   [junit4]   2> 30344 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:39019_solr
   [junit4]   2> 32032 DEBUG (OverseerAutoScalingTriggerThread-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 32149 DEBUG (OverseerAutoScalingTriggerThread-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 34092 DEBUG (OverseerAutoScalingTriggerThread-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:39019_solr, 127.0.0.1:45158_solr, 127.0.0.1:36739_solr]
   [junit4]   2> 34093 DEBUG (OverseerAutoScalingTriggerThread-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 34093 DEBUG (OverseerAutoScalingTriggerThread-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 34279 DEBUG (OverseerAutoScalingTriggerThread-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 34281 DEBUG (OverseerAutoScalingTriggerThread-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 34376 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 35593 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 36625 DEBUG (ScheduledTrigger-14-thread-2) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 37134 INFO  (jetty-launcher-1-thread-3) [n:127.0.0.1:36739_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36739.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4d21cd5c
   [junit4]   2> 37188 INFO  (jetty-launcher-1-thread-3) [n:127.0.0.1:36739_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36739.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4d21cd5c
   [junit4]   2> 37189 INFO  (jetty-launcher-1-thread-3) [n:127.0.0.1:36739_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36739.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4d21cd5c
   [junit4]   2> 37134 INFO  (jetty-launcher-1-thread-2) [n:127.0.0.1:45158_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45158.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4d21cd5c
   [junit4]   2> 37197 INFO  (jetty-launcher-1-thread-2) [n:127.0.0.1:45158_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45158.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4d21cd5c
   [junit4]   2> 37197 INFO  (jetty-launcher-1-thread-2) [n:127.0.0.1:45158_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45158.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4d21cd5c
   [junit4]   2> 37187 INFO  (jetty-launcher-1-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39019.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4d21cd5c
   [junit4]   2> 37198 INFO  (jetty-launcher-1-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39019.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4d21cd5c
   [junit4]   2> 37198 INFO  (jetty-launcher-1-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39019.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4d21cd5c
   [junit4]   2> 37574 INFO  (jetty-launcher-1-thread-3) [n:127.0.0.1:36739_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node3/.
   [junit4]   2> 37610 INFO  (jetty-launcher-1-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node1/.
   [junit4]   2> 37632 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 37665 INFO  (jetty-launcher-1-thread-2) [n:127.0.0.1:45158_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node2/.
   [junit4]   2> 38636 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 39644 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 40645 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 41687 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 42512 INFO  (zkConnectionManagerCallback-36-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 42602 INFO  (zkConnectionManagerCallback-40-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 42609 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[6B76F5E1536ABDFC]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 42629 INFO  (SUITE-AutoscalingHistoryHandlerTest-seed#[6B76F5E1536ABDFC]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:47534/solr ready
   [junit4]   2> 43068 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 44069 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 45210 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 46210 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 47270 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 48478 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 51196 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 52522 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 52787 INFO  (qtp577311207-36) [n:127.0.0.1:39019_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=3&name=AutoscalingHistoryHandlerTest_collection&nrtReplicas=3&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 53523 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 54528 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 55397 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.CreateCollectionCmd Create collection AutoscalingHistoryHandlerTest_collection
   [junit4]   2> 55398 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.CreateCollectionCmd Only _default config set found, using it.
   [junit4]   2> 55417 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/managed-schema to /configs/AutoscalingHistoryHandlerTest_collection/managed-schema
   [junit4]   2> 55427 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/protwords.txt to /configs/AutoscalingHistoryHandlerTest_collection/protwords.txt
   [junit4]   2> 55439 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/solrconfig.xml to /configs/AutoscalingHistoryHandlerTest_collection/solrconfig.xml
   [junit4]   2> 55458 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/synonyms.txt to /configs/AutoscalingHistoryHandlerTest_collection/synonyms.txt
   [junit4]   2> 55477 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/stopwords.txt to /configs/AutoscalingHistoryHandlerTest_collection/stopwords.txt
   [junit4]   2> 55527 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_it.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/contractions_it.txt
   [junit4]   2> 55539 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/contractions_ca.txt
   [junit4]   2> 55626 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stemdict_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stemdict_nl.txt
   [junit4]   2> 55629 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_no.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_no.txt
   [junit4]   2> 55639 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hy.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_hy.txt
   [junit4]   2> 55657 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_id.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_id.txt
   [junit4]   2> 55658 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hi.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_hi.txt
   [junit4]   2> 55678 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_ga.txt
   [junit4]   2> 55680 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_da.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_da.txt
   [junit4]   2> 55694 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_ja.txt
   [junit4]   2> 55696 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_pt.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_pt.txt
   [junit4]   2> 55721 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hu.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_hu.txt
   [junit4]   2> 55728 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_el.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_el.txt
   [junit4]   2> 55759 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ru.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_ru.txt
   [junit4]   2> 55945 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_tr.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_tr.txt
   [junit4]   2> 55948 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ar.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_ar.txt
   [junit4]   2> 55950 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/userdict_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/userdict_ja.txt
   [junit4]   2> 55952 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_eu.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_eu.txt
   [junit4]   2> 55986 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_it.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_it.txt
   [junit4]   2> 56095 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_cz.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_cz.txt
   [junit4]   2> 56122 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stoptags_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stoptags_ja.txt
   [junit4]   2> 56130 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/contractions_fr.txt
   [junit4]   2> 56150 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_de.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_de.txt
   [junit4]   2> 56165 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fa.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_fa.txt
   [junit4]   2> 56189 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/contractions_ga.txt
   [junit4]   2> 56229 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_ca.txt
   [junit4]   2> 56237 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_nl.txt
   [junit4]   2> 56259 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_bg.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_bg.txt
   [junit4]   2> 56264 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 56271 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_en.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_en.txt
   [junit4]   2> 56385 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_es.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_es.txt
   [junit4]   2> 56393 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_lv.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_lv.txt
   [junit4]   2> 56395 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ro.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_ro.txt
   [junit4]   2> 56403 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_th.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_th.txt
   [junit4]   2> 56405 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_fr.txt
   [junit4]   2> 56420 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_sv.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_sv.txt
   [junit4]   2> 56422 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fi.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_fi.txt
   [junit4]   2> 56452 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/hyphenations_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/hyphenations_ga.txt
   [junit4]   2> 56454 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_gl.txt to /configs/AutoscalingHistoryHandlerTest_collection/lang/stopwords_gl.txt
   [junit4]   2> 56465 INFO  (OverseerThreadFactory-16-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/params.json to /configs/AutoscalingHistoryHandlerTest_collection/params.json
   [junit4]   2> 57149 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "name":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "operation":"create"} current state version: 0
   [junit4]   2> 57149 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.ClusterStateMutator building a new cName: AutoscalingHistoryHandlerTest_collection
   [junit4]   2> 57173 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json
   [junit4]   2> 57288 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 57902 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:45158/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 57904 INFO  (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:45158/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 57923 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{}}
   [junit4]   2> 57923 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"http://127.0.0.1:45158/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 57927 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:36739/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 57927 INFO  (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:36739/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 57942 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{"core_node2":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"http://127.0.0.1:45158/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 57942 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node2":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"http://127.0.0.1:45158/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
   [junit4]   2>       "base_url":"http://127.0.0.1:36739/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 57945 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n5",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:39019/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 57945 INFO  (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n5",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:39019/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 57961 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node2":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"http://127.0.0.1:45158/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
   [junit4]   2>       "base_url":"http://127.0.0.1:36739/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 57961 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.SliceMutator New Slice: shard1:{
   [junit4]   2>   "range":"80000000-7fffffff",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "replicas":{
   [junit4]   2>     "core_node2":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>       "base_url":"http://127.0.0.1:45158/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node4":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
   [junit4]   2>       "base_url":"http://127.0.0.1:36739/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"},
   [junit4]   2>     "core_node6":{
   [junit4]   2>       "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n5",
   [junit4]   2>       "base_url":"http://127.0.0.1:39019/solr",
   [junit4]   2>       "state":"down",
   [junit4]   2>       "node_name":null,
   [junit4]   2>       "type":"NRT"}}}
   [junit4]   2> 58080 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 0
   [junit4]   2> 60264 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 60532 INFO  (qtp577311207-34) [n:127.0.0.1:39019_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=AutoscalingHistoryHandlerTest_collection&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n5&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 60621 INFO  (qtp577311207-34) [n:127.0.0.1:39019_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 61038 INFO  (qtp1212769377-28) [n:127.0.0.1:45158_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=AutoscalingHistoryHandlerTest_collection&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 61134 INFO  (qtp1447694748-45) [n:127.0.0.1:36739_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=AutoscalingHistoryHandlerTest_collection&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n3&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 61181 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n5",
   [junit4]   2>   "core_node_name":"core_node6",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:39019/solr",
   [junit4]   2>   "node_name":"127.0.0.1:39019_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 61258 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n5",
   [junit4]   2>   "core_node_name":"core_node6",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:39019/solr",
   [junit4]   2>   "node_name":"127.0.0.1:39019_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 61258 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node6:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n5","base_url":"http://127.0.0.1:39019/solr","node_name":"127.0.0.1:39019_solr","state":"down","type":"NRT"}
   [junit4]   2> 61259 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node2":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>           "base_url":"http://127.0.0.1:45158/solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
   [junit4]   2>           "base_url":"http://127.0.0.1:36739/solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n5",
   [junit4]   2>           "base_url":"http://127.0.0.1:39019/solr",
   [junit4]   2>           "node_name":"127.0.0.1:39019_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 61185 INFO  (qtp1447694748-45) [n:127.0.0.1:36739_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 61259 INFO  (qtp1212769377-28) [n:127.0.0.1:45158_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 61279 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 61280 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 1
   [junit4]   2> 61281 INFO  (zkCallback-27-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 61281 INFO  (zkCallback-23-thread-1-processing-n:127.0.0.1:45158_solr) [n:127.0.0.1:45158_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 61281 INFO  (zkCallback-25-thread-1-processing-n:127.0.0.1:36739_solr) [n:127.0.0.1:36739_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 62792 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 62817 INFO  (zkCallback-27-thread-2-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 62858 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node2",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:45158/solr",
   [junit4]   2>   "node_name":"127.0.0.1:45158_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 62858 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node2",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:45158/solr",
   [junit4]   2>   "node_name":"127.0.0.1:45158_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 62858 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node2:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1","base_url":"http://127.0.0.1:45158/solr","node_name":"127.0.0.1:45158_solr","state":"down","type":"NRT"}
   [junit4]   2> 62858 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/2)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node2":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>           "base_url":"http://127.0.0.1:45158/solr",
   [junit4]   2>           "node_name":"127.0.0.1:45158_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
   [junit4]   2>           "base_url":"http://127.0.0.1:36739/solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "node_name":null,
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n5",
   [junit4]   2>           "base_url":"http://127.0.0.1:39019/solr",
   [junit4]   2>           "node_name":"127.0.0.1:39019_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 62859 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
   [junit4]   2>   "core_node_name":"core_node4",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:36739/solr",
   [junit4]   2>   "node_name":"127.0.0.1:36739_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 62859 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
   [junit4]   2>   "core_node_name":"core_node4",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:36739/solr",
   [junit4]   2>   "node_name":"127.0.0.1:36739_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"AutoscalingHistoryHandlerTest_collection",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 62859 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node4:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3","base_url":"http://127.0.0.1:36739/solr","node_name":"127.0.0.1:36739_solr","state":"down","type":"NRT"}
   [junit4]   2> 62860 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/2)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"3",
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"3",
   [junit4]   2>   "tlogReplicas":"0",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node2":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
   [junit4]   2>           "base_url":"http://127.0.0.1:45158/solr",
   [junit4]   2>           "node_name":"127.0.0.1:45158_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
   [junit4]   2>           "base_url":"http://127.0.0.1:36739/solr",
   [junit4]   2>           "node_name":"127.0.0.1:36739_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"},
   [junit4]   2>         "core_node6":{
   [junit4]   2>           "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n5",
   [junit4]   2>           "base_url":"http://127.0.0.1:39019/solr",
   [junit4]   2>           "node_name":"127.0.0.1:39019_solr",
   [junit4]   2>           "state":"down",
   [junit4]   2>           "type":"NRT"}}}}}
   [junit4]   2> 62977 DEBUG (OverseerStateUpdate-99251675344273416-127.0.0.1:39019_solr-n_0000000000) [n:127.0.0.1:39019_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 2
   [junit4]   2> 62980 INFO  (zkCallback-23-thread-1-processing-n:127.0.0.1:45158_solr) [n:127.0.0.1:45158_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 62981 INFO  (zkCallback-23-thread-2-processing-n:127.0.0.1:45158_solr) [n:127.0.0.1:45158_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 62985 INFO  (zkCallback-25-thread-1-processing-n:127.0.0.1:36739_solr) [n:127.0.0.1:36739_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 63036 INFO  (zkCallback-27-thread-1-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 63037 INFO  (zkCallback-27-thread-2-processing-n:127.0.0.1:39019_solr) [n:127.0.0.1:39019_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 63037 INFO  (zkCallback-25-thread-2-processing-n:127.0.0.1:36739_solr) [n:127.0.0.1:36739_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 63850 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 64486 INFO  (qtp1447694748-45) [n:127.0.0.1:36739_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
   [junit4]   2> 64486 INFO  (qtp1212769377-28) [n:127.0.0.1:45158_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node2 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
   [junit4]   2> 64494 INFO  (qtp1212769377-28) [n:127.0.0.1:45158_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node2 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 64504 INFO  (qtp1447694748-45) [n:127.0.0.1:36739_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 64508 WARN  (qtp1447694748-45) [n:127.0.0.1:36739_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../contrib/extraction/lib
   [junit4]   2> 64508 WARN  (qtp1447694748-45) [n:127.0.0.1:36739_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../dist
   [junit4]   2> 64533 INFO  (qtp577311207-34) [n:127.0.0.1:39019_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n5] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
   [junit4]   2> 64533 INFO  (qtp577311207-34) [n:127.0.0.1:39019_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n5] o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 64535 WARN  (qtp577311207-34) [n:127.0.0.1:39019_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n5] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n5/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n5/../../../../contrib/extraction/lib
   [junit4]   2> 64590 WARN  (qtp577311207-34) [n:127.0.0.1:39019_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n5] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n5/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n5/../../../../dist
   [junit4]   2> 64590 WARN  (qtp577311207-34) [n:127.0.0.1:39019_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n5] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n5/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n5/../../../../contrib/clustering/lib
   [junit4]   2> 64590 WARN  (qtp577311207-34) [n:127.0.0.1:39019_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n5] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n5/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n5/../../../../dist
   [junit4]   2> 65479 WARN  (qtp577311207-34) [n:127.0.0.1:39019_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n5] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n5/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n5/../../../../contrib/langid/lib
   [junit4]   2> 65480 WARN  (qtp577311207-34) [n:127.0.0.1:39019_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n5] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n5/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n5/../../../../dist
   [junit4]   2> 65480 WARN  (qtp577311207-34) [n:127.0.0.1:39019_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n5] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n5/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n5/../../../../contrib/velocity/lib
   [junit4]   2> 65480 WARN  (qtp577311207-34) [n:127.0.0.1:39019_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n5] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n5/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n5/../../../../dist
   [junit4]   2> 64860 DEBUG (ScheduledTrigger-14-thread-1) [n:127.0.0.1:39019_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 64591 WARN  (qtp1447694748-45) [n:127.0.0.1:36739_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../contrib/clustering/lib
   [junit4]   2> 65501 WARN  (qtp1447694748-45) [n:127.0.0.1:36739_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../dist
   [junit4]   2> 65501 WARN  (qtp1447694748-45) [n:127.0.0.1:36739_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../contrib/langid/lib
   [junit4]   2> 65501 WARN  (qtp1447694748-45) [n:127.0.0.1:36739_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../dist
   [junit4]   2> 65501 WARN  (qtp1447694748-45) [n:127.0.0.1:36739_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../contrib/velocity/lib
   [junit4]   2> 65502 WARN  (qtp1447694748-45) [n:127.0.0.1:36739_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../dist
   [junit4]   2> 64591 WARN  (qtp1212769377-28) [n:127.0.0.1:45158_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node2 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib
   [junit4]   2> 65517 WARN  (qtp1212769377-28) [n:127.0.0.1:45158_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node2 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
   [junit4]   2> 65517 WARN  (qtp1212769377-28) [n:127.0.0.1:45158_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node2 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib
   [junit4]   2> 65517 WARN  (qtp1212769377-28) [n:127.0.0.1:45158_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node2 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
   [junit4]   2> 65517 WARN  (qtp1212769377-28) [n:127.0.0.1:45158_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node2 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_6B76F5E1536ABDFC-001/tempDir-001/node2/

[...truncated too long message...]

s-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/bcprov-jdk15on-1.54.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/byte-buddy-1.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/commons-math3-3.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/hadoop-common-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/hadoop-hdfs-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/hadoop-minikdc-2.7.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/jetty-sslengine-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/mockito-core-2.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/objenesis-2.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/analysis/icu/lucene-analyzers-icu-8.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/backward-codecs/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/contrib/analysis-extras/lib/icu4j-59.1.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.4.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_144/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.5.3.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/temp/junit4-J0-20171228_112555_9466106093411507083313.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/temp/junit4-J0-20171228_112555_9468384397485327318758.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Forked process returned with error code: 1. Very likely a JVM crash.  See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/temp/junit4-J0-20171228_112555_9462357586352288899207.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/temp/junit4-J0-20171228_112555_9467772148069778494442.syserr
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
   [junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4] at java.lang.Thread.run(Thread.java:748)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:835: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:779: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build.xml:262: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/common-build.xml:556: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:1512: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:1038: At least one slave process threw an exception, first: Forked process returned with error code: 1. Very likely a JVM crash.  See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/temp/junit4-J0-20171228_112555_9462357586352288899207.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/temp/junit4-J0-20171228_112555_9467772148069778494442.syserr

Total time: 218 minutes 33 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[Fast Archiver] No artifacts from Lucene-Solr-Tests-master #2233 to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2243 - Still Failing

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2243/

All tests passed

Build Log:
[...truncated 6636 lines...]
   [junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/temp/junit4-J0-20171229_022543_4684363538288798594480.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] #
   [junit4] # There is insufficient memory for the Java Runtime Environment to continue.
   [junit4] # Native memory allocation (mmap) failed to map 65536 bytes for committing reserved memory.
   [junit4] # An error report file with more information is saved as:
   [junit4] # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/J0/hs_err_pid30384.log
   [junit4] #
   [junit4] # Compiler replay data is saved as:
   [junit4] # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/J0/replay_pid30384.log
   [junit4] <<< JVM J0: EOF ----

   [junit4] JVM J0: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/temp/junit4-J0-20171229_022543_4681235482176639385064.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00007f718ee00000, 65536, 1) failed; error='Cannot allocate memory' (errno=12)
   [junit4] <<< JVM J0: EOF ----

[...truncated 56 lines...]
   [junit4] ERROR: JVM J0 ended with an exception, command line: /usr/local/asfpackages/java/jdk1.8.0_144/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=E8F2F8FB8509E49B -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=8.0.0 -Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=2 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/temp -Dcommon.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene -Dclover.db.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/clover/db -Djava.security.policy=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/tools/junit4/tests.policy -Dtests.LUCENE_VERSION=8.0.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.src.home=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master -Djava.security.egd=file:/dev/./urandom -Djunit4.childvm.cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/J0 -Djunit4.childvm.id=0 -Djunit4.childvm.count=3 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -classpath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/test-framework/lib/junit-4.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/test-framework/lib/randomizedtesting-runner-2.5.3.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.4.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_144/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.5.3.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/temp/junit4-J0-20171229_022543_4681933860763498712203.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/temp/junit4-J0-20171229_022543_4688994715924491269860.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Forked process returned with error code: 1. Very likely a JVM crash.  See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/temp/junit4-J0-20171229_022543_4684363538288798594480.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/temp/junit4-J0-20171229_022543_4681235482176639385064.syserr
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
   [junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4] at java.lang.Thread.run(Thread.java:748)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:835: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:779: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build.xml:495: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:2280: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:1512: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:1038: At least one slave process threw an exception, first: Forked process returned with error code: 1. Very likely a JVM crash.  See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/temp/junit4-J0-20171229_022543_4684363538288798594480.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/temp/junit4-J0-20171229_022543_4681235482176639385064.syserr

Total time: 125 minutes 24 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[Fast Archiver] No artifacts from Lucene-Solr-Tests-master #2233 to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2244 - Still Failing

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2244/

All tests passed

Build Log:
[...truncated 869 lines...]
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 183500800 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/hs_err_pid6532.log
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[Fast Archiver] No artifacts from Lucene-Solr-Tests-master #2233 to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2245 - Still Failing

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2245/

All tests passed

Build Log:
[...truncated 1816 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/temp/junit4-J1-20171230_012413_6818575161146518053455.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] #
   [junit4] # There is insufficient memory for the Java Runtime Environment to continue.
   [junit4] # Native memory allocation (mmap) failed to map 83886080 bytes for committing reserved memory.
   [junit4] # An error report file with more information is saved as:
   [junit4] # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/J1/hs_err_pid10823.log
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/temp/junit4-J1-20171230_012413_6813665667205941078748.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000e9580000, 83886080, 0) failed; error='Cannot allocate memory' (errno=12)
   [junit4] <<< JVM J1: EOF ----

[...truncated 48 lines...]
   [junit4] ERROR: JVM J1 ended with an exception, command line: /usr/local/asfpackages/java/jdk1.8.0_144/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=1DB8121C72024C05 -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=8.0.0 -Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=2 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/temp -Dcommon.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene -Dclover.db.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/clover/db -Djava.security.policy=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/tools/junit4/tests.policy -Dtests.LUCENE_VERSION=8.0.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.src.home=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master -Djava.security.egd=file:/dev/./urandom -Djunit4.childvm.cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/J1 -Djunit4.childvm.id=1 -Djunit4.childvm.count=3 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -classpath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/test-framework/lib/junit-4.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/test-framework/lib/randomizedtesting-runner-2.5.3.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/classes/test:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.4.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_144/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.5.3.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/temp/junit4-J1-20171230_012413_6816022761302951036188.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/temp/junit4-J1-20171230_012413_6819150722681586805636.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Forked process returned with error code: 1. Very likely a JVM crash.  See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/temp/junit4-J1-20171230_012413_6818575161146518053455.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/temp/junit4-J1-20171230_012413_6813665667205941078748.syserr
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
   [junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4] at java.lang.Thread.run(Thread.java:748)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:835: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:779: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build.xml:50: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:1512: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:1038: At least one slave process threw an exception, first: Forked process returned with error code: 1. Very likely a JVM crash.  See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/temp/junit4-J1-20171230_012413_6818575161146518053455.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/temp/junit4-J1-20171230_012413_6813665667205941078748.syserr

Total time: 64 minutes 53 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[Fast Archiver] No artifacts from Lucene-Solr-Tests-master #2233 to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2246 - Still Failing

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2246/

All tests passed

Build Log:
[...truncated 6667 lines...]
   [junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/temp/junit4-J2-20171230_191031_655496179495134783366.sysout
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] #
   [junit4] # There is insufficient memory for the Java Runtime Environment to continue.
   [junit4] # Native memory allocation (mmap) failed to map 65536 bytes for committing reserved memory.
   [junit4] # An error report file with more information is saved as:
   [junit4] # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/J2/hs_err_pid14004.log
   [junit4] #
   [junit4] # Compiler replay data is saved as:
   [junit4] # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/J2/replay_pid14004.log
   [junit4] <<< JVM J2: EOF ----

   [junit4] JVM J2: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/temp/junit4-J2-20171230_191031_6556721643211864357063.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00007fc32f010000, 65536, 1) failed; error='Cannot allocate memory' (errno=12)
   [junit4] <<< JVM J2: EOF ----

[...truncated 33 lines...]
   [junit4] ERROR: JVM J2 ended with an exception, command line: /usr/local/asfpackages/java/jdk1.8.0_144/jre/bin/java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/heapdumps -ea -esa -Dtests.prefix=tests -Dtests.seed=D6B4593F27951E16 -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=8.0.0 -Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=2 -DtempDir=./temp -Djava.io.tmpdir=./temp -Djunit4.tempDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/temp -Dcommon.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene -Dclover.db.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/clover/db -Djava.security.policy=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/tools/junit4/tests.policy -Dtests.LUCENE_VERSION=8.0.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.src.home=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master -Djava.security.egd=file:/dev/./urandom -Djunit4.childvm.cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/J2 -Djunit4.childvm.id=2 -Djunit4.childvm.count=3 -Dtests.leaveTemporary=false -Dtests.filterstacks=true -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -classpath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/test-framework/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/test-framework/lib/junit-4.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/test-framework/lib/randomizedtesting-runner-2.5.3.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.4.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_144/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.5.3.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/temp/junit4-J2-20171230_191031_6557946698025519594129.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/temp/junit4-J2-20171230_191031_6554178105271018903184.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Forked process returned with error code: 1. Very likely a JVM crash.  See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/temp/junit4-J2-20171230_191031_655496179495134783366.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/temp/junit4-J2-20171230_191031_6556721643211864357063.syserr
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
   [junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4] at java.lang.Thread.run(Thread.java:748)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:835: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:779: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build.xml:495: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:2280: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/module-build.xml:58: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:1512: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:1038: At least one slave process threw an exception, first: Forked process returned with error code: 1. Very likely a JVM crash.  See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/temp/junit4-J2-20171230_191031_655496179495134783366.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/codecs/test/temp/junit4-J2-20171230_191031_6556721643211864357063.syserr

Total time: 134 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[Fast Archiver] No artifacts from Lucene-Solr-Tests-master #2233 to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2247 - Still Failing

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2247/

6 tests failed.
FAILED:  org.apache.lucene.codecs.TestCodecLoadingDeadlock.testDeadlock

Error Message:
Process died abnormally expected:<0> but was:<1>

Stack Trace:
java.lang.AssertionError: Process died abnormally expected:<0> but was:<1>
        at __randomizedtesting.SeedInfo.seed([E8F5138F1DD0689D:E59EF29B1B8AC54B]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.apache.lucene.codecs.TestCodecLoadingDeadlock.testDeadlock(TestCodecLoadingDeadlock.java:72)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:404)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:705)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:139)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:626)


FAILED:  org.apache.solr.cloud.ZkSolrClientTest.testMultipleWatchesAsync

Error Message:


Stack Trace:
java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([93F715ADD3027E66:FB41C60F46E67FF8]:0)
        at org.junit.Assert.fail(Assert.java:92)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertTrue(Assert.java:54)
        at org.apache.solr.cloud.ZkSolrClientTest.testMultipleWatchesAsync(ZkSolrClientTest.java:252)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple

Error Message:
Error from server at https://127.0.0.1:39222/solr: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-0000000002

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:39222/solr: KeeperErrorCode = Session expired for /overseer/collection-queue-work/qnr-0000000002
        at __randomizedtesting.SeedInfo.seed([93F715ADD3027E66:AB443153F4F1AAB7]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:73)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.SystemLogListenerTest.test

Error Message:
Trigger was not fired

Stack Trace:
java.lang.AssertionError: Trigger was not fired
        at __randomizedtesting.SeedInfo.seed([93F715ADD3027E66:1BA32A777DFE139E]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.SystemLogListenerTest.test(SystemLogListenerTest.java:151)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeMarkersRegistration

Error Message:
Path /autoscaling/nodeAdded/127.0.0.1:37646_solr wasn't created

Stack Trace:
java.lang.AssertionError: Path /autoscaling/nodeAdded/127.0.0.1:37646_solr wasn't created
        at __randomizedtesting.SeedInfo.seed([93F715ADD3027E66:8B4D9DA1DD37B389]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeMarkersRegistration(TriggerIntegrationTest.java:936)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.update.TestInPlaceUpdatesDistrib.test

Error Message:
Error from server at https://127.0.0.1:35766/xg_/qv/collection1: Cannot talk to ZooKeeper - Updates are disabled.

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:35766/xg_/qv/collection1: Cannot talk to ZooKeeper - Updates are disabled.
        at __randomizedtesting.SeedInfo.seed([93F715ADD3027E66:1BA32A777DFE139E]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.update.TestInPlaceUpdatesDistrib.addDocAndGetVersion(TestInPlaceUpdatesDistrib.java:1066)
        at org.apache.solr.update.TestInPlaceUpdatesDistrib.buildRandomIndex(TestInPlaceUpdatesDistrib.java:1106)
        at org.apache.solr.update.TestInPlaceUpdatesDistrib.docValuesUpdateTest(TestInPlaceUpdatesDistrib.java:315)
        at org.apache.solr.update.TestInPlaceUpdatesDistrib.test(TestInPlaceUpdatesDistrib.java:142)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 1413 lines...]
   [junit4] Suite: org.apache.lucene.codecs.TestCodecLoadingDeadlock
   [junit4] FAILURE 1.69s J0 | TestCodecLoadingDeadlock.testDeadlock <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Process died abnormally expected:<0> but was:<1>
   [junit4]    > at __randomizedtesting.SeedInfo.seed([E8F5138F1DD0689D:E59EF29B1B8AC54B]:0)
   [junit4]    > at org.apache.lucene.codecs.TestCodecLoadingDeadlock.testDeadlock(TestCodecLoadingDeadlock.java:72)
   [junit4] Completed [355/477 (1!)] on J0 in 1.72s, 1 test, 1 failure <<< FAILURES!

[...truncated 431 lines...]
   [junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/temp/junit4-J0-20171231_033958_8996114779215921873539.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] #
   [junit4] # There is insufficient memory for the Java Runtime Environment to continue.
   [junit4] # Native memory allocation (mmap) failed to map 88080384 bytes for committing reserved memory.
   [junit4] # An error report file with more information is saved as:
   [junit4] # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/J0/hs_err_pid18466.log
   [junit4] <<< JVM J0: EOF ----

   [junit4] JVM J0: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/core/test/temp/junit4-J0-20171231_033958_8993636149693335046404.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x000000076c600000, 88080384, 0) failed; error='Cannot allocate memory' (errno=12)
   [junit4] <<< JVM J0: EOF ----

[...truncated 9970 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_93F715ADD3027E66-001/init-core-data-001
   [junit4]   2> 311006 WARN  (SUITE-TriggerIntegrationTest-seed#[93F715ADD3027E66]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=18 numCloses=18
   [junit4]   2> 311023 INFO  (SUITE-TriggerIntegrationTest-seed#[93F715ADD3027E66]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 311035 INFO  (SUITE-TriggerIntegrationTest-seed#[93F715ADD3027E66]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 311047 INFO  (SUITE-TriggerIntegrationTest-seed#[93F715ADD3027E66]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 311056 INFO  (SUITE-TriggerIntegrationTest-seed#[93F715ADD3027E66]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_93F715ADD3027E66-001/tempDir-001
   [junit4]   2> 311057 INFO  (SUITE-TriggerIntegrationTest-seed#[93F715ADD3027E66]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 311060 INFO  (Thread-141) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 311060 INFO  (Thread-141) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 311120 ERROR (Thread-141) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 311160 INFO  (SUITE-TriggerIntegrationTest-seed#[93F715ADD3027E66]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:60656
   [junit4]   2> 311302 INFO  (zkConnectionManagerCallback-99-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 311489 INFO  (jetty-launcher-96-thread-1) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 311518 INFO  (jetty-launcher-96-thread-2) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 311540 INFO  (jetty-launcher-96-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1fe77c45{/solr,null,AVAILABLE}
   [junit4]   2> 311585 INFO  (jetty-launcher-96-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5e0a0d86{SSL,[ssl, http/1.1]}{127.0.0.1:46161}
   [junit4]   2> 311681 INFO  (jetty-launcher-96-thread-1) [    ] o.e.j.s.Server Started @333977ms
   [junit4]   2> 311683 INFO  (jetty-launcher-96-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@bbe682a{/solr,null,AVAILABLE}
   [junit4]   2> 311684 INFO  (jetty-launcher-96-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4568973a{SSL,[ssl, http/1.1]}{127.0.0.1:55867}
   [junit4]   2> 311684 INFO  (jetty-launcher-96-thread-2) [    ] o.e.j.s.Server Started @333980ms
   [junit4]   2> 311681 INFO  (jetty-launcher-96-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46161}
   [junit4]   2> 311684 INFO  (jetty-launcher-96-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55867}
   [junit4]   2> 311760 ERROR (jetty-launcher-96-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 311762 ERROR (jetty-launcher-96-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 311762 INFO  (jetty-launcher-96-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 311762 INFO  (jetty-launcher-96-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 311762 INFO  (jetty-launcher-96-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 311763 INFO  (jetty-launcher-96-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-12-31T06:25:13.474Z
   [junit4]   2> 311762 INFO  (jetty-launcher-96-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 311764 INFO  (jetty-launcher-96-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 311764 INFO  (jetty-launcher-96-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 311764 INFO  (jetty-launcher-96-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-12-31T06:25:13.476Z
   [junit4]   2> 311911 INFO  (zkConnectionManagerCallback-101-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 312026 INFO  (zkConnectionManagerCallback-103-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 312063 INFO  (jetty-launcher-96-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 312125 INFO  (jetty-launcher-96-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 312347 INFO  (jetty-launcher-96-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60656/solr
   [junit4]   2> 312507 INFO  (zkConnectionManagerCallback-107-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 312600 INFO  (jetty-launcher-96-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60656/solr
   [junit4]   2> 312685 INFO  (zkConnectionManagerCallback-113-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 312721 INFO  (zkConnectionManagerCallback-109-thread-1-processing-n:127.0.0.1:46161_solr) [n:127.0.0.1:46161_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 312998 INFO  (zkConnectionManagerCallback-115-thread-1-processing-n:127.0.0.1:55867_solr) [n:127.0.0.1:55867_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 313151 INFO  (jetty-launcher-96-thread-2) [n:127.0.0.1:55867_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 313162 INFO  (jetty-launcher-96-thread-2) [n:127.0.0.1:55867_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55867_solr
   [junit4]   2> 313182 INFO  (jetty-launcher-96-thread-2) [n:127.0.0.1:55867_solr    ] o.a.s.c.Overseer Overseer (id=99267478341812230-127.0.0.1:55867_solr-n_0000000000) starting
   [junit4]   2> 313376 INFO  (jetty-launcher-96-thread-2) [n:127.0.0.1:55867_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55867_solr
   [junit4]   2> 313421 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 313643 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 313643 INFO  (OverseerStateUpdate-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 313659 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:55867_solr]
   [junit4]   2> 313659 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 313659 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 313759 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 313759 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 313820 DEBUG (ScheduledTrigger-212-thread-1) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 315007 INFO  (jetty-launcher-96-thread-1) [n:127.0.0.1:46161_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 315009 INFO  (jetty-launcher-96-thread-1) [n:127.0.0.1:46161_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 315039 INFO  (jetty-launcher-96-thread-1) [n:127.0.0.1:46161_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46161_solr
   [junit4]   2> 315055 INFO  (zkCallback-114-thread-1-processing-n:127.0.0.1:55867_solr) [n:127.0.0.1:55867_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 315243 INFO  (zkCallback-108-thread-1-processing-n:127.0.0.1:46161_solr) [n:127.0.0.1:46161_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 316046 DEBUG (ScheduledTrigger-212-thread-1) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 316292 INFO  (jetty-launcher-96-thread-2) [n:127.0.0.1:55867_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55867.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ec4d78b
   [junit4]   2> 316350 INFO  (jetty-launcher-96-thread-2) [n:127.0.0.1:55867_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55867.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ec4d78b
   [junit4]   2> 316350 INFO  (jetty-launcher-96-thread-2) [n:127.0.0.1:55867_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55867.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ec4d78b
   [junit4]   2> 316470 INFO  (jetty-launcher-96-thread-1) [n:127.0.0.1:46161_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46161.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ec4d78b
   [junit4]   2> 316483 INFO  (jetty-launcher-96-thread-2) [n:127.0.0.1:55867_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_93F715ADD3027E66-001/tempDir-001/node2/.
   [junit4]   2> 316581 INFO  (jetty-launcher-96-thread-1) [n:127.0.0.1:46161_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46161.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ec4d78b
   [junit4]   2> 316582 INFO  (jetty-launcher-96-thread-1) [n:127.0.0.1:46161_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46161.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ec4d78b
   [junit4]   2> 316749 INFO  (jetty-launcher-96-thread-1) [n:127.0.0.1:46161_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_93F715ADD3027E66-001/tempDir-001/node1/.
   [junit4]   2> 317375 DEBUG (ScheduledTrigger-212-thread-2) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 317512 INFO  (zkConnectionManagerCallback-121-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 317655 INFO  (zkConnectionManagerCallback-125-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 317699 INFO  (SUITE-TriggerIntegrationTest-seed#[93F715ADD3027E66]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 317700 INFO  (SUITE-TriggerIntegrationTest-seed#[93F715ADD3027E66]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:60656/solr ready
   [junit4]   2> 318078 INFO  (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[93F715ADD3027E66]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTrigger
   [junit4]   2> 318080 DEBUG (zkCallback-114-thread-1-processing-n:127.0.0.1:55867_solr) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 318080 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 318087 INFO  (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[93F715ADD3027E66]) [    ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
   [junit4]   2> 318088 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 318089 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 318091 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 318320 INFO  (qtp1068125444-508) [n:127.0.0.1:55867_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 318910 INFO  (qtp1068125444-508) [n:127.0.0.1:55867_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=590
   [junit4]   2> 318911 INFO  (OverseerCollectionConfigSetProcessor-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 319828 INFO  (qtp973466138-498) [n:127.0.0.1:46161_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CORE.coreName&wt=javabin&version=2&group=solr.core} status=0 QTime=56
   [junit4]   2> 320276 INFO  (qtp1068125444-511) [n:127.0.0.1:55867_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CORE.coreName&wt=javabin&version=2&group=solr.core} status=0 QTime=0
   [junit4]   2> 320278 DEBUG (qtp1068125444-509) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 320279 DEBUG (zkCallback-114-thread-1-processing-n:127.0.0.1:55867_solr) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 320279 INFO  (qtp1068125444-509) [n:127.0.0.1:55867_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=1117
   [junit4]   2> 320308 DEBUG (zkCallback-114-thread-1-processing-n:127.0.0.1:55867_solr) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:55867_solr, 127.0.0.1:46161_solr]
   [junit4]   2> 320308 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 320310 INFO  (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
   [junit4]   2> 320447 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 320448 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 320479 DEBUG (ScheduledTrigger-212-thread-3) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 320568 INFO  (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[93F715ADD3027E66]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@5e0a0d86{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 320579 INFO  (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[93F715ADD3027E66]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=504747662
   [junit4]   2> 320579 INFO  (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[93F715ADD3027E66]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 320579 INFO  (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[93F715ADD3027E66]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@554145a: rootName = solr_46161, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@155be37b
   [junit4]   2> 320992 INFO  (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[93F715ADD3027E66]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 320993 INFO  (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[93F715ADD3027E66]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@d9530f0: rootName = solr_46161, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@7ed015fa
   [junit4]   2> 321053 INFO  (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[93F715ADD3027E66]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 321053 INFO  (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[93F715ADD3027E66]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6a3cc54d: rootName = solr_46161, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@23db0257
   [junit4]   2> 321081 INFO  (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[93F715ADD3027E66]) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:46161_solr
   [junit4]   2> 321083 INFO  (zkCallback-114-thread-1-processing-n:127.0.0.1:55867_solr) [n:127.0.0.1:55867_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 321085 INFO  (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[93F715ADD3027E66]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 321085 INFO  (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[93F715ADD3027E66]) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 321161 INFO  (zkCallback-124-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 321571 DEBUG (ScheduledTrigger-212-thread-3) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
   [junit4]   2> 321571 DEBUG (ScheduledTrigger-212-thread-3) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:46161_solr
   [junit4]   2> 322595 DEBUG (ScheduledTrigger-212-thread-3) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
   [junit4]   2> 322675 WARN  (zkCallback-108-thread-1-processing-n:127.0.0.1:46161_solr) [n:127.0.0.1:46161_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 322736 INFO  (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[93F715ADD3027E66]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1fe77c45{/solr,null,UNAVAILABLE}
   [junit4]   2> 323603 DEBUG (ScheduledTrigger-212-thread-3) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
   [junit4]   2> 323604 DEBUG (ScheduledTrigger-212-thread-3) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:46161_solr]
   [junit4]   2> 323619 DEBUG (ScheduledTrigger-212-thread-3) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"3520138e38346cT69abirbf5zf4eovp9enitddv1",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":14953442128180332,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[14953442128180332],
   [junit4]   2>     "nodeNames":["127.0.0.1:46161_solr"]}}
   [junit4]   2> 323629 INFO  (ScheduledTrigger-212-thread-3) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
   [junit4]   2> 323830 DEBUG (AutoscalingActionExecutor-213-thread-1-processing-n:127.0.0.1:55867_solr) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"3520138e38346cT69abirbf5zf4eovp9enitddv1",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":14953442128180332,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[14953442128180332],
   [junit4]   2>     "_enqueue_time_":14953444176073962,
   [junit4]   2>     "nodeNames":["127.0.0.1:46161_solr"]}}
   [junit4]   2> 323920 INFO  (qtp1068125444-507) [n:127.0.0.1:55867_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CORE.coreName&wt=javabin&version=2&group=solr.core} status=0 QTime=0
   [junit4]   2> 323924 DEBUG (qtp1068125444-510) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 323928 DEBUG (zkCallback-114-thread-1-processing-n:127.0.0.1:55867_solr) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 323929 DEBUG (zkCallback-114-thread-1-processing-n:127.0.0.1:55867_solr) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:55867_solr]
   [junit4]   2> 323943 INFO  (qtp1068125444-510) [n:127.0.0.1:55867_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=87
   [junit4]   2> 324047 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 324049 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 324050 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 324050 DEBUG (AutoscalingActionExecutor-213-thread-1-processing-n:127.0.0.1:55867_solr) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.ScheduledTriggers -- processing took 220 ms for event id=3520138e38346cT69abirbf5zf4eovp9enitddv1
   [junit4]   2> 324837 DEBUG (ScheduledTrigger-212-thread-3) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
   [junit4]   2> 325838 DEBUG (ScheduledTrigger-212-thread-3) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
   [junit4]   2> 326855 DEBUG (ScheduledTrigger-212-thread-3) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
   [junit4]   2> 327187 INFO  (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[93F715ADD3027E66]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testNodeLostTrigger
   [junit4]   2> 327247 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testContinueTriggersOnOverseerRestart
   [junit4]   2> 327259 DEBUG (zkCallback-114-thread-1-processing-n:127.0.0.1:55867_solr) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 327260 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 327260 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 5
   [junit4]   2> 327277 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 327277 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 327277 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 327298 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 327395 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6b3ea02a{/solr,null,AVAILABLE}
   [junit4]   2> 327396 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5150bb86{SSL,[ssl, http/1.1]}{127.0.0.1:37877}
   [junit4]   2> 327396 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.e.j.s.Server Started @349692ms
   [junit4]   2> 327396 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37877}
   [junit4]   2> 327411 ERROR (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 327605 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 327605 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 327605 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 327605 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-12-31T06:25:29.317Z
   [junit4]   2> 327671 INFO  (zkConnectionManagerCallback-127-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 327723 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 327843 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60656/solr
   [junit4]   2> 328101 INFO  (zkConnectionManagerCallback-131-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 328172 INFO  (zkConnectionManagerCallback-133-thread-1-processing-n:127.0.0.1:37877_solr) [n:127.0.0.1:37877_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 328224 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [n:127.0.0.1:37877_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 328239 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [n:127.0.0.1:37877_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 328260 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [n:127.0.0.1:37877_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37877_solr
   [junit4]   2> 328280 INFO  (zkCallback-124-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 328280 INFO  (zkCallback-114-thread-1-processing-n:127.0.0.1:55867_solr) [n:127.0.0.1:55867_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 328438 INFO  (zkCallback-132-thread-1-processing-n:127.0.0.1:37877_solr) [n:127.0.0.1:37877_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 329678 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [n:127.0.0.1:37877_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37877.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ec4d78b
   [junit4]   2> 329729 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [n:127.0.0.1:37877_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37877.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ec4d78b
   [junit4]   2> 329729 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [n:127.0.0.1:37877_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37877.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ec4d78b
   [junit4]   2> 329756 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [n:127.0.0.1:37877_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_93F715ADD3027E66-001/tempDir-001/node3/.
   [junit4]   2> 330360 INFO  (qtp986256798-568) [n:127.0.0.1:37877_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 330664 INFO  (qtp986256798-568) [n:127.0.0.1:37877_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=304
   [junit4]   2> 330715 INFO  (qtp986256798-563) [n:127.0.0.1:37877_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CORE.coreName&wt=javabin&version=2&group=solr.core} status=0 QTime=0
   [junit4]   2> 330780 INFO  (qtp1068125444-511) [n:127.0.0.1:55867_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CORE.coreName&wt=javabin&version=2&group=solr.core} status=0 QTime=0
   [junit4]   2> 330781 DEBUG (qtp986256798-569) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 330781 DEBUG (zkCallback-114-thread-1-processing-n:127.0.0.1:55867_solr) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 330800 DEBUG (zkCallback-114-thread-1-processing-n:127.0.0.1:55867_solr) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:55867_solr, 127.0.0.1:37877_solr]
   [junit4]   2> 330800 DEBUG (zkCallback-114-thread-1-processing-n:127.0.0.1:55867_solr) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=2, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
   [junit4]   2> 330800 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 330801 INFO  (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
   [junit4]   2> 330802 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 330802 DEBUG (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
   [junit4]   2> 330802 INFO  (qtp986256798-569) [n:127.0.0.1:37877_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=117
   [junit4]   2> 330891 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@4568973a{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 330802 DEBUG (ScheduledTrigger-212-thread-4) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 330907 DEBUG (ScheduledTrigger-212-thread-4) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 330911 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1230232921
   [junit4]   2> 330911 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 330912 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@24c431db: rootName = solr_55867, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@5e4d4aff
   [junit4]   2> 330939 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 330940 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@621baeae: rootName = solr_55867, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@7ed015fa
   [junit4]   2> 331052 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 331052 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4e73b211: rootName = solr_55867, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@23db0257
   [junit4]   2> 331057 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:55867_solr
   [junit4]   2> 331061 INFO  (zkCallback-124-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 331062 INFO  (zkCallback-132-thread-1-processing-n:127.0.0.1:37877_solr) [n:127.0.0.1:37877_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 331062 INFO  (zkCallback-114-thread-1-processing-n:127.0.0.1:55867_solr) [n:127.0.0.1:55867_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 331091 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 331095 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 331095 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.c.Overseer Overseer (id=99267478341812230-127.0.0.1:55867_solr-n_0000000000) closing
   [junit4]   2> 331095 INFO  (OverseerStateUpdate-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:55867_solr
   [junit4]   2> 331096 WARN  (OverseerAutoScalingTriggerThread-99267478341812230-127.0.0.1:55867_solr-n_0000000000) [n:127.0.0.1:55867_solr    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 331135 DEBUG (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 331227 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@bbe682a{/solr,null,UNAVAILABLE}
   [junit4]   2> 331243 INFO  (zkCallback-132-thread-1-processing-n:127.0.0.1:37877_solr) [n:127.0.0.1:37877_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37877_solr
   [junit4]   2> 331623 INFO  (zkCallback-132-thread-1-processing-n:127.0.0.1:37877_solr) [n:127.0.0.1:37877_solr    ] o.a.s.c.Overseer Overseer (id=99267478341812235-127.0.0.1:37877_solr-n_0000000002) starting
   [junit4]   2> 331864 DEBUG (OverseerAutoScalingTriggerThread-99267478341812235-127.0.0.1:37877_solr-n_0000000002) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 331865 DEBUG (OverseerAutoScalingTriggerThread-99267478341812235-127.0.0.1:37877_solr-n_0000000002) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
   [junit4]   2> 331871 DEBUG (OverseerAutoScalingTriggerThread-99267478341812235-127.0.0.1:37877_solr-n_0000000002) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:37877_solr]
   [junit4]   2> 331871 DEBUG (OverseerAutoScalingTriggerThread-99267478341812235-127.0.0.1:37877_solr-n_0000000002) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=2, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
   [junit4]   2> 331871 DEBUG (OverseerAutoScalingTriggerThread-99267478341812235-127.0.0.1:37877_solr-n_0000000002) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:37877_solr]
   [junit4]   2> 331871 DEBUG (OverseerAutoScalingTriggerThread-99267478341812235-127.0.0.1:37877_solr-n_0000000002) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
   [junit4]   2> 331871 DEBUG (OverseerAutoScalingTriggerThread-99267478341812235-127.0.0.1:37877_solr-n_0000000002) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
   [junit4]   2> 332125 INFO  (OverseerAutoScalingTriggerThread-99267478341812235-127.0.0.1:37877_solr-n_0000000002) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
   [junit4]   2> 332127 DEBUG (ScheduledTrigger-228-thread-1) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 332131 DEBUG (OverseerAutoScalingTriggerThread-99267478341812235-127.0.0.1:37877_solr-n_0000000002) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
   [junit4]   2> 332132 DEBUG (ScheduledTrigger-228-thread-2) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 332132 DEBUG (ScheduledTrigger-228-thread-2) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 333137 DEBUG (ScheduledTrigger-228-thread-1) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 333139 DEBUG (ScheduledTrigger-228-thread-1) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 333139 DEBUG (ScheduledTrigger-228-thread-1) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 334139 DEBUG (ScheduledTrigger-228-thread-3) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 334139 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 334139 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 335162 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 335162 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 335162 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 336163 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 336163 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 336163 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 337163 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 337163 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 337163 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 338167 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 338167 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 338167 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 339203 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 339203 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 339203 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 340203 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 340203 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 340203 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 341203 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 341203 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 341203 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 341265 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 341427 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3b6d7dc2{/solr,null,AVAILABLE}
   [junit4]   2> 341428 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@1687f5a4{SSL,[ssl, http/1.1]}{127.0.0.1:55187}
   [junit4]   2> 341428 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.e.j.s.Server Started @363724ms
   [junit4]   2> 341428 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55187}
   [junit4]   2> 341428 ERROR (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 341471 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 341472 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 341472 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 341472 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-12-31T06:25:43.184Z
   [junit4]   2> 341539 INFO  (zkConnectionManagerCallback-137-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 341540 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 341580 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60656/solr
   [junit4]   2> 341780 INFO  (zkConnectionManagerCallback-141-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 341987 INFO  (zkConnectionManagerCallback-143-thread-1-processing-n:127.0.0.1:55187_solr) [n:127.0.0.1:55187_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 342215 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 342215 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 342215 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
   [junit4]   2> 342248 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [n:127.0.0.1:55187_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 342264 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [n:127.0.0.1:55187_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 342267 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [n:127.0.0.1:55187_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55187_solr
   [junit4]   2> 342285 INFO  (zkCallback-132-thread-1-processing-n:127.0.0.1:37877_solr) [n:127.0.0.1:37877_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 342287 INFO  (zkCallback-124-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 342327 INFO  (zkCallback-142-thread-1-processing-n:127.0.0.1:55187_solr) [n:127.0.0.1:55187_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 343064 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [n:127.0.0.1:55187_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55187.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ec4d78b
   [junit4]   2> 343191 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [n:127.0.0.1:55187_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55187.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ec4d78b
   [junit4]   2> 343191 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [n:127.0.0.1:55187_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55187.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ec4d78b
   [junit4]   2> 343192 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [n:127.0.0.1:55187_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_93F715ADD3027E66-001/tempDir-001/node4/.
   [junit4]   2> 343215 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 343215 DEBUG (ScheduledTrigger-228-thread-3) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 343215 DEBUG (ScheduledTrigger-228-thread-3) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 343215 DEBUG (ScheduledTrigger-228-thread-3) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:55187_solr at time 14953463772650141
   [junit4]   2> 344790 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 344791 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 344791 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 345808 DEBUG (ScheduledTrigger-228-thread-1) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 345818 DEBUG (ScheduledTrigger-228-thread-1) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 345818 DEBUG (ScheduledTrigger-228-thread-1) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 345818 DEBUG (ScheduledTrigger-228-thread-1) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:55187_solr] added at times [14953463772650141], now=14953466375693138
   [junit4]   2> 345820 DEBUG (ScheduledTrigger-228-thread-1) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"35201898549a9dT69abirbf5zf4eovp9enitddv3",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":14953463772650141,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[14953463772650141],
   [junit4]   2>     "nodeNames":["127.0.0.1:55187_solr"]}}
   [junit4]   2> 345824 INFO  (ScheduledTrigger-228-thread-1) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
   [junit4]   2> 345889 DEBUG (AutoscalingActionExecutor-229-thread-1-processing-n:127.0.0.1:37877_solr) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"35201898549a9dT69abirbf5zf4eovp9enitddv3",
   [junit4]   2>   "source":"node_added_trigger",
   [junit4]   2>   "eventTime":14953463772650141,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[14953463772650141],
   [junit4]   2>     "_enqueue_time_":14953466377755412,
   [junit4]   2>     "nodeNames":["127.0.0.1:55187_solr"]}}
   [junit4]   2> 345889 INFO  (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[93F715ADD3027E66]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testContinueTriggersOnOverseerRestart
   [junit4]   2> 345936 DEBUG (AutoscalingActionExecutor-229-thread-1-processing-n:127.0.0.1:37877_solr) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.ScheduledTriggers -- processing took 47 ms for event id=35201898549a9dT69abirbf5zf4eovp9enitddv3
   [junit4]   2> 345982 INFO  (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[93F715ADD3027E66]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTrigger
   [junit4]   2> 345998 DEBUG (zkCallback-132-thread-1-processing-n:127.0.0.1:37877_solr) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
   [junit4]   2> 345998 DEBUG (OverseerAutoScalingTriggerThread-99267478341812235-127.0.0.1:37877_solr-n_0000000002) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
   [junit4]   2> 346006 INFO  (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[93F715ADD3027E66]) [    ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 8
   [junit4]   2> 346019 DEBUG (OverseerAutoScalingTriggerThread-99267478341812235-127.0.0.1:37877_solr-n_0000000002) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 346024 DEBUG (OverseerAutoScalingTriggerThread-99267478341812235-127.0.0.1:37877_solr-n_0000000002) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 346040 DEBUG (OverseerAutoScalingTriggerThread-99267478341812235-127.0.0.1:37877_solr-n_0000000002) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
   [junit4]   2> 346507 INFO  (qtp986256798-566) [n:127.0.0.1:37877_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CORE.coreName&wt=javabin&version=2&group=solr.core} status=0 QTime=0
   [junit4]   2> 346798 INFO  (qtp250739613-601) [n:127.0.0.1:55187_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CORE.coreName&wt=javabin&version=2&group=solr.core} status=0 QTime=0
   [junit4]   2> 346799 DEBUG (qtp986256798-570) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 347567 INFO  (qtp986256798-570) [n:127.0.0.1:37877_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=1516
   [junit4]   2> 347579 DEBUG (zkCallback-132-thread-1-processing-n:127.0.0.1:37877_solr) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
   [junit4]   2> 347580 DEBUG (zkCallback-132-thread-1-processing-n:127.0.0.1:37877_solr) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:55187_solr, 127.0.0.1:37877_solr]
   [junit4]   2> 347580 DEBUG (zkCallback-132-thread-1-processing-n:127.0.0.1:37877_solr) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
   [junit4]   2> 347583 DEBUG (OverseerAutoScalingTriggerThread-99267478341812235-127.0.0.1:37877_solr-n_0000000002) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
   [junit4]   2> 347615 INFO  (OverseerAutoScalingTriggerThread-99267478341812235-127.0.0.1:37877_solr-n_0000000002) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
   [junit4]   2> 347627 DEBUG (OverseerAutoScalingTriggerThread-99267478341812235-127.0.0.1:37877_solr-n_0000000002) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 347627 DEBUG (OverseerAutoScalingTriggerThread-99267478341812235-127.0.0.1:37877_solr-n_0000000002) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
   [junit4]   2> 347628 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
   [junit4]   2> 347628 DEBUG (ScheduledTrigger-228-thread-4) [n:127.0.0.1:37877_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 347635 INFO  (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[93F715ADD3027E66]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 347716 INFO  (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[93F715ADD3027E66]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@38549e20{/solr,null,AVAILABLE}
   [junit4]   2> 347717 INFO  (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[93F715ADD3027E66]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@589d51e5{SSL,[ssl, http/1.1]}{127.0.0.1:44852}
   [junit4]   2> 347717 INFO  (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[93F715ADD3027E66]) [    ] o.e.j.s.Server Started @370013ms
   [junit4]   2> 347717 INFO  (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[93F715ADD3027E66]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort

[...truncated too long message...]

ds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/bcprov-jdk15on-1.54.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/byte-buddy-1.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/commons-math3-3.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/hadoop-common-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/hadoop-hdfs-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/hadoop-minikdc-2.7.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/jetty-sslengine-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/mockito-core-2.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/objenesis-2.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/analysis/icu/lucene-analyzers-icu-8.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/backward-codecs/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/contrib/analysis-extras/lib/icu4j-59.1.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.4.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_144/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.5.3.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/temp/junit4-J2-20171231_061939_2191257146958656118099.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/temp/junit4-J2-20171231_061939_219930266154706267325.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Forked process returned with error code: 1. Very likely a JVM crash.  See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/temp/junit4-J2-20171231_061939_2196394776889947929657.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/temp/junit4-J2-20171231_061939_2198892544205887871783.syserr
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
   [junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4] at java.lang.Thread.run(Thread.java:748)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:835: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:779: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build.xml:262: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/common-build.xml:556: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:1512: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:1038: At least one slave process threw an exception, first: Forked process returned with error code: 1. Very likely a JVM crash.  See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/temp/junit4-J1-20171231_061939_1992499189715676068664.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/temp/junit4-J1-20171231_061939_1995416051985173901051.syserr

Total time: 387 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[Fast Archiver] No artifacts from Lucene-Solr-Tests-master #2233 to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2248 - Still unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2248/

13 tests failed.
FAILED:  org.apache.solr.cloud.HttpPartitionTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:49663/_jcf/d

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:49663/_jcf/d
        at __randomizedtesting.SeedInfo.seed([D9D7500777D18398:51836FDDD92DEE60]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:414)
        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:339)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:171)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
        at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
        at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
        at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
        at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
        at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
        at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
        at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
        ... 44 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.HttpPartitionTest

Error Message:
58 threads leaked from SUITE scope at org.apache.solr.cloud.HttpPartitionTest:     1) Thread[id=7981, name=qtp135330343-7981, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)    2) Thread[id=8037, name=Thread-2829, state=WAITING, group=TGRP-HttpPartitionTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1725)    3) Thread[id=8016, name=qtp828389506-8016, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)    4) Thread[id=8083, name=OverseerAutoScalingTriggerThread-99280357355028494-127.0.0.1:49663__jcf%2Fd-n_0000000003, state=TIMED_WAITING, group=Overseer autoscaling triggers]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)         at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:315)         at org.apache.solr.client.solrj.impl.ZkDistribStateManager.hasData(ZkDistribStateManager.java:57)         at org.apache.solr.cloud.autoscaling.TriggerBase.<init>(TriggerBase.java:87)         at org.apache.solr.cloud.autoscaling.NodeLostTrigger.<init>(NodeLostTrigger.java:53)         at org.apache.solr.cloud.autoscaling.AutoScaling$TriggerFactoryImpl.create(AutoScaling.java:155)         at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.loadTriggers(OverseerTriggerThread.java:361)         at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.refreshAutoScalingConf(OverseerTriggerThread.java:308)         at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:138)         at java.lang.Thread.run(Thread.java:748)    5) Thread[id=8020, name=qtp828389506-8020, state=RUNNABLE, group=TGRP-HttpPartitionTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)    6) Thread[id=8056, name=zkCallback-1521-thread-4, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    7) Thread[id=7989, name=org.eclipse.jetty.server.session.HashSessionManager@4504b282Timer, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    8) Thread[id=8002, name=Thread-2820, state=WAITING, group=TGRP-HttpPartitionTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1725)    9) Thread[id=8023, name=qtp828389506-8023, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   10) Thread[id=8057, name=zkCallback-1531-thread-2, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   11) Thread[id=8066, name=zkConnectionManagerCallback-1532-thread-1-processing-n:127.0.0.1:49663__jcf%2Fd-SendThread(127.0.0.1:43205), state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)   12) Thread[id=8041, name=Scheduler-1063660173, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   13) Thread[id=8000, name=zkConnectionManagerCallback-1522-thread-1, state=WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   14) Thread[id=8018, name=qtp828389506-8018-acceptor-0@7638cc9f-ServerConnector@18a1a163{HTTP/1.1,[http/1.1]}{127.0.0.1:53586}, state=RUNNABLE, group=TGRP-HttpPartitionTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   15) Thread[id=7988, name=qtp135330343-7988, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   16) Thread[id=8058, name=zkCallback-1531-thread-3, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   17) Thread[id=8085, name=zkCallback-1521-thread-6, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   18) Thread[id=8082, name=OverseerCollectionConfigSetProcessor-99280357355028494-127.0.0.1:49663__jcf%2Fd-n_0000000003, state=WAITING, group=Overseer collection creation process.]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406)         at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1210)         at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)         at org.apache.solr.common.cloud.SolrZkClient$$Lambda$26/981023021.execute(Unknown Source)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)         at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)         at org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:387)         at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:193)         at java.lang.Thread.run(Thread.java:748)   19) Thread[id=7983, name=qtp135330343-7983-acceptor-0@5eb834c5-ServerConnector@51c2b996{HTTP/1.1,[http/1.1]}{127.0.0.1:58179}, state=RUNNABLE, group=TGRP-HttpPartitionTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   20) Thread[id=8054, name=zkCallback-1521-thread-2, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   21) Thread[id=8029, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   22) Thread[id=8008, name=Scheduler-2063177852, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   23) Thread[id=8065, name=zkCallback-1531-thread-5, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   24) Thread[id=8004, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   25) Thread[id=8035, name=zkConnectionManagerCallback-1532-thread-1, state=WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   26) Thread[id=8021, name=qtp828389506-8021, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   27) Thread[id=7985, name=qtp135330343-7985, state=RUNNABLE, group=TGRP-HttpPartitionTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   28) Thread[id=8070, name=zkConnectionManagerCallback-1522-thread-1-processing-n:127.0.0.1:60656__jcf%2Fd-SendThread(127.0.0.1:43205), state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)   29) Thread[id=8013, name=searcherExecutor-2599-thread-1, state=WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   30) Thread[id=8068, name=zkCallback-1521-thread-5, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   31) Thread[id=8081, name=OverseerStateUpdate-99280357355028494-127.0.0.1:49663__jcf%2Fd-n_0000000003, state=TIMED_WAITING, group=Overseer state updater.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:463)         at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:203)         at java.lang.Thread.run(Thread.java:748)   32) Thread[id=8015, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   33) Thread[id=8022, name=qtp828389506-8022, state=RUNNABLE, group=TGRP-HttpPartitionTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   34) Thread[id=8001, name=zkCallback-1521-thread-1, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   35) Thread[id=8059, name=zkCallback-1531-thread-4, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   36) Thread[id=8073, name=coreZkRegister-2593-thread-2, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   37) Thread[id=7993, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   38) Thread[id=7986, name=qtp135330343-7986, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   39) Thread[id=8055, name=zkCallback-1521-thread-3, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   40) Thread[id=8067, name=zkConnectionManagerCallback-1532-thread-1-processing-n:127.0.0.1:49663__jcf%2Fd-EventThread, state=WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)   41) Thread[id=8039, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   42) Thread[id=7987, name=qtp135330343-7987, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   43) Thread[id=7994, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   44) Thread[id=8087, name=zkCallback-1531-thread-7, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   45) Thread[id=8028, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   46) Thread[id=8024, name=org.eclipse.jetty.server.session.HashSessionManager@5781b116Timer, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   47) Thread[id=8019, name=qtp828389506-8019, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at java.lang.Object.wait(Native Method)         at org.apache.solr.cloud.OverseerTaskQueue$LatchWatcher.await(OverseerTaskQueue.java:149)         at org.apache.solr.cloud.OverseerTaskQueue.offer(OverseerTaskQueue.java:199)         at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:302)         at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:246)         at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:224)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)         at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)         at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)         at org.eclipse.jetty.server.Server.handle(Server.java:534)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)         at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   48) Thread[id=8077, name=searcherExecutor-2610-thread-1, state=WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   49) Thread[id=8048, name=watches-1533-thread-1, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   50) Thread[id=8036, name=zkCallback-1531-thread-1, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   51) Thread[id=8071, name=zkConnectionManagerCallback-1522-thread-1-processing-n:127.0.0.1:60656__jcf%2Fd-EventThread, state=WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)   52) Thread[id=8086, name=zkCallback-1531-thread-6, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   53) Thread[id=8017, name=qtp828389506-8017, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   54) Thread[id=7984, name=qtp135330343-7984, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   55) Thread[id=8069, name=coreZkRegister-2604-thread-1, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   56) Thread[id=8072, name=coreZkRegister-2593-thread-1, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   57) Thread[id=7982, name=qtp135330343-7982, state=RUNNABLE, group=TGRP-HttpPartitionTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   58) Thread[id=8011, name=watches-1523-thread-1, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 58 threads leaked from SUITE scope at org.apache.solr.cloud.HttpPartitionTest:
   1) Thread[id=7981, name=qtp135330343-7981, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
   2) Thread[id=8037, name=Thread-2829, state=WAITING, group=TGRP-HttpPartitionTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at org.apache.solr.core.CloserThread.run(CoreContainer.java:1725)
   3) Thread[id=8016, name=qtp828389506-8016, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
   4) Thread[id=8083, name=OverseerAutoScalingTriggerThread-99280357355028494-127.0.0.1:49663__jcf%2Fd-n_0000000003, state=TIMED_WAITING, group=Overseer autoscaling triggers]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111)
        at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)
        at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:315)
        at org.apache.solr.client.solrj.impl.ZkDistribStateManager.hasData(ZkDistribStateManager.java:57)
        at org.apache.solr.cloud.autoscaling.TriggerBase.<init>(TriggerBase.java:87)
        at org.apache.solr.cloud.autoscaling.NodeLostTrigger.<init>(NodeLostTrigger.java:53)
        at org.apache.solr.cloud.autoscaling.AutoScaling$TriggerFactoryImpl.create(AutoScaling.java:155)
        at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.loadTriggers(OverseerTriggerThread.java:361)
        at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.refreshAutoScalingConf(OverseerTriggerThread.java:308)
        at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:138)
        at java.lang.Thread.run(Thread.java:748)
   5) Thread[id=8020, name=qtp828389506-8020, state=RUNNABLE, group=TGRP-HttpPartitionTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
   6) Thread[id=8056, name=zkCallback-1521-thread-4, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   7) Thread[id=7989, name=org.eclipse.jetty.server.session.HashSessionManager@4504b282Timer, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
   8) Thread[id=8002, name=Thread-2820, state=WAITING, group=TGRP-HttpPartitionTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at org.apache.solr.core.CloserThread.run(CoreContainer.java:1725)
   9) Thread[id=8023, name=qtp828389506-8023, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
  10) Thread[id=8057, name=zkCallback-1531-thread-2, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  11) Thread[id=8066, name=zkConnectionManagerCallback-1532-thread-1-processing-n:127.0.0.1:49663__jcf%2Fd-SendThread(127.0.0.1:43205), state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
  12) Thread[id=8041, name=Scheduler-1063660173, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  13) Thread[id=8000, name=zkConnectionManagerCallback-1522-thread-1, state=WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  14) Thread[id=8018, name=qtp828389506-8018-acceptor-0@7638cc9f-ServerConnector@18a1a163{HTTP/1.1,[http/1.1]}{127.0.0.1:53586}, state=RUNNABLE, group=TGRP-HttpPartitionTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
        at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
  15) Thread[id=7988, name=qtp135330343-7988, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
  16) Thread[id=8058, name=zkCallback-1531-thread-3, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  17) Thread[id=8085, name=zkCallback-1521-thread-6, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  18) Thread[id=8082, name=OverseerCollectionConfigSetProcessor-99280357355028494-127.0.0.1:49663__jcf%2Fd-n_0000000003, state=WAITING, group=Overseer collection creation process.]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406)
        at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1210)
        at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
        at org.apache.solr.common.cloud.SolrZkClient$$Lambda$26/981023021.execute(Unknown Source)
        at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
        at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
        at org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:387)
        at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:193)
        at java.lang.Thread.run(Thread.java:748)
  19) Thread[id=7983, name=qtp135330343-7983-acceptor-0@5eb834c5-ServerConnector@51c2b996{HTTP/1.1,[http/1.1]}{127.0.0.1:58179}, state=RUNNABLE, group=TGRP-HttpPartitionTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
        at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
  20) Thread[id=8054, name=zkCallback-1521-thread-2, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  21) Thread[id=8029, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  22) Thread[id=8008, name=Scheduler-2063177852, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  23) Thread[id=8065, name=zkCallback-1531-thread-5, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  24) Thread[id=8004, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  25) Thread[id=8035, name=zkConnectionManagerCallback-1532-thread-1, state=WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  26) Thread[id=8021, name=qtp828389506-8021, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
  27) Thread[id=7985, name=qtp135330343-7985, state=RUNNABLE, group=TGRP-HttpPartitionTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
  28) Thread[id=8070, name=zkConnectionManagerCallback-1522-thread-1-processing-n:127.0.0.1:60656__jcf%2Fd-SendThread(127.0.0.1:43205), state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
  29) Thread[id=8013, name=searcherExecutor-2599-thread-1, state=WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  30) Thread[id=8068, name=zkCallback-1521-thread-5, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  31) Thread[id=8081, name=OverseerStateUpdate-99280357355028494-127.0.0.1:49663__jcf%2Fd-n_0000000003, state=TIMED_WAITING, group=Overseer state updater.]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:463)
        at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:203)
        at java.lang.Thread.run(Thread.java:748)
  32) Thread[id=8015, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  33) Thread[id=8022, name=qtp828389506-8022, state=RUNNABLE, group=TGRP-HttpPartitionTest]
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
        at java.lang.Thread.run(Thread.java:748)
  34) Thread[id=8001, name=zkCallback-1521-thread-1, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  35) Thread[id=8059, name=zkCallback-1531-thread-4, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  36) Thread[id=8073, name=coreZkRegister-2593-thread-2, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  37) Thread[id=7993, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  38) Thread[id=7986, name=qtp135330343-7986, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
  39) Thread[id=8055, name=zkCallback-1521-thread-3, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  40) Thread[id=8067, name=zkConnectionManagerCallback-1532-thread-1-processing-n:127.0.0.1:49663__jcf%2Fd-EventThread, state=WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
  41) Thread[id=8039, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  42) Thread[id=7987, name=qtp135330343-7987, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
        at java.lang.Thread.run(Thread.java:748)
  43) Thread[id=7994, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  44) Thread[id=8087, name=zkCallback-1531-thread-7, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  45) Thread[id=8028, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
        at java.lang.Thread.run(Thread.java:748)
  46) Thread[id=8024, name=org.eclipse.jetty.server.session.HashSessionManager@5781b116Timer, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
  47) Thread[id=8019, name=qtp828389506-8019, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
        at java.lang.Object.wait(Native Method)
        at org.apache.solr.cloud.OverseerTaskQueue$LatchWatcher.await(OverseerTaskQueue.java:149)
        at org.apache.solr.cloud.OverseerTaskQueue.offer(OverseerTaskQueue.java:199)
        at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:302)
        at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:246)
        at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:224)
        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
        at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)
        at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)
        at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
        at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(Serv

[...truncated too long message...]

ng-n:127.0.0.1:40883_solr) [n:127.0.0.1:40883_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 909013 INFO  (coreCloseExecutor-347-thread-1) [n:127.0.0.1:43957_solr c:streams_collection s:shard2 r:core_node4 x:streams_collection_shard2_replica_n2] o.a.s.c.SolrCore [streams_collection_shard2_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@7fd75791
   [junit4]   2> 909013 INFO  (coreCloseExecutor-347-thread-1) [n:127.0.0.1:43957_solr c:streams_collection s:shard2 r:core_node4 x:streams_collection_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.streams_collection.shard2.replica_n2, tag=2144819089
   [junit4]   2> 909014 INFO  (coreCloseExecutor-347-thread-1) [n:127.0.0.1:43957_solr c:streams_collection s:shard2 r:core_node4 x:streams_collection_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5b70fa2a: rootName = solr_43957, domain = solr.core.streams_collection.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.streams_collection.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@47d43e8b
   [junit4]   2> 909113 INFO  (zkCallback-218-thread-4-processing-n:127.0.0.1:43957_solr) [n:127.0.0.1:43957_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/streams_collection/state.json] for collection [streams_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 909113 INFO  (zkCallback-218-thread-3-processing-n:127.0.0.1:43957_solr) [n:127.0.0.1:43957_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/streams_collection/state.json] for collection [streams_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 909125 INFO  (zkCallback-216-thread-5-processing-n:127.0.0.1:40883_solr) [n:127.0.0.1:40883_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/streams_collection/state.json] for collection [streams_collection] has occurred - updating... (live nodes size: [0])
   [junit4]   2> 909125 INFO  (zkCallback-216-thread-7-processing-n:127.0.0.1:40883_solr) [n:127.0.0.1:40883_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/streams_collection/state.json] for collection [streams_collection] has occurred - updating... (live nodes size: [0])
   [junit4]   2> 909178 INFO  (zkCallback-218-thread-5-processing-n:127.0.0.1:43957_solr) [n:127.0.0.1:43957_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 909297 INFO  (coreCloseExecutor-346-thread-1) [n:127.0.0.1:40883_solr c:streams_collection s:shard1 r:core_node3 x:streams_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.streams_collection.shard1.leader, tag=1322754374
   [junit4]   2> 909369 INFO  (coreCloseExecutor-347-thread-1) [n:127.0.0.1:43957_solr c:streams_collection s:shard2 r:core_node4 x:streams_collection_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.streams_collection.shard2.leader, tag=2144819089
   [junit4]   2> 909527 INFO  (jetty-closer-201-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2c7f2c05{/solr,null,UNAVAILABLE}
   [junit4]   2> 909546 INFO  (jetty-closer-201-thread-2) [    ] o.a.s.c.Overseer Overseer (id=99281055687573509-127.0.0.1:40883_solr-n_0000000000) closing
   [junit4]   2> 909547 INFO  (OverseerStateUpdate-99281055687573509-127.0.0.1:40883_solr-n_0000000000) [n:127.0.0.1:40883_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40883_solr
   [junit4]   2> 909595 WARN  (OverseerAutoScalingTriggerThread-99281055687573509-127.0.0.1:40883_solr-n_0000000000) [n:127.0.0.1:40883_solr    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 909633 INFO  (jetty-closer-201-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@147ee1e{/solr,null,UNAVAILABLE}
   [junit4]   2> 909649 ERROR (SUITE-StreamingTest-seed#[F712A001C0025A76]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 909649 INFO  (SUITE-StreamingTest-seed#[F712A001C0025A76]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:50922 50922
   [junit4]   2> 909833 INFO  (Thread-241) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:50922 50922
   [junit4]   2> 909833 WARN  (Thread-241) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 45 /solr/aliases.json
   [junit4]   2> 2 /solr/security.json
   [junit4]   2> 2 /solr/configs/conf
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 45 /solr/clusterstate.json
   [junit4]   2> 45 /solr/clusterprops.json
   [junit4]   2> 18 /solr/collections/streams_collection/state.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 45 /solr/live_nodes
   [junit4]   2> 45 /solr/collections
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamingTest_F712A001C0025A76-001
   [junit4]   2> Jan 02, 2018 3:59:44 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {multiDefault=PostingsFormat(name=MockRandom), a_f=PostingsFormat(name=MockRandom), intDefault=PostingsFormat(name=Memory), a_i=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), dt_sing=PostingsFormat(name=Memory), level1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), dt_multi=PostingsFormat(name=Memory), l_multi=PostingsFormat(name=Memory), s_multi=PostingsFormat(name=LuceneVarGapDocFreqInterval), a_ss=PostingsFormat(name=Memory), level2_s=PostingsFormat(name=Memory), a_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), d_multi=PostingsFormat(name=Memory), timestamp=PostingsFormat(name=Memory), d_sing=PostingsFormat(name=Memory), i_multi=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), b_sing=PostingsFormat(name=MockRandom), b_multi=PostingsFormat(name=MockRandom), f_multi=PostingsFormat(name=MockRandom), _version_=PostingsFormat(name=Memory), f_sing=PostingsFormat(name=MockRandom), s_sing=PostingsFormat(name=LuceneVarGapDocFreqInterval), i_sing=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), l_sing=PostingsFormat(name=Memory)}, docValues:{multiDefault=DocValuesFormat(name=Memory), a_f=DocValuesFormat(name=Memory), intDefault=DocValuesFormat(name=Lucene70), a_i=DocValuesFormat(name=Asserting), dt_sing=DocValuesFormat(name=Lucene70), level1_s=DocValuesFormat(name=Asserting), dt_multi=DocValuesFormat(name=Lucene70), l_multi=DocValuesFormat(name=Lucene70), s_multi=DocValuesFormat(name=Lucene70), a_ss=DocValuesFormat(name=Lucene70), level2_s=DocValuesFormat(name=Lucene70), a_s=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Asserting), d_multi=DocValuesFormat(name=Lucene70), timestamp=DocValuesFormat(name=Lucene70), d_sing=DocValuesFormat(name=Lucene70), i_multi=DocValuesFormat(name=Asserting), b_sing=DocValuesFormat(name=Memory), b_multi=DocValuesFormat(name=Memory), f_multi=DocValuesFormat(name=Memory), _version_=DocValuesFormat(name=Lucene70), f_sing=DocValuesFormat(name=Memory), s_sing=DocValuesFormat(name=Lucene70), i_sing=DocValuesFormat(name=Asserting), l_sing=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1289, maxMBSortInHeap=5.543353428800498, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@f46b8bd), locale=ms, timezone=America/Montserrat
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=294659440,total=407896064
   [junit4]   2> NOTE: All tests run in this JVM: [GreaterThanEqualToEvaluatorTest, TestHash, StreamExpressionToExplanationTest, TestBatchUpdate, OperationsTest, SolrExampleBinaryTest, RegressionEvaluatorTest, LessThanEqualToEvaluatorTest, CommonParamsTest, TangentEvaluatorTest, ArrayEvaluatorTest, NormalizeEvaluatorTest, LargeVolumeJettyTest, GetByIdTest, JdbcDriverTest, TestJsonRecordReader, MultiplyEvaluatorTest, CoalesceEvaluatorTest, SolrExampleXMLTest, ArcCosineEvaluatorTest, SolrExampleJettyTest, DocumentAnalysisResponseTest, SolrExceptionTest, TestSuggesterResponse, LargeVolumeBinaryJettyTest, JettyWebappTest, LargeVolumeEmbeddedTest, MergeIndexesEmbeddedTest, TestNamedListCodec, TestJavaBinCodec, TestDocumentObjectBinder, SolrQueryTest, ModifiableSolrParamsTest, SolrDocumentTest, TestUpdateRequestCodec, FieldAnalysisResponseTest, ClientUtilsTest, NamedListTest, AnlysisResponseBaseTest, SolrSchemalessExampleTest, TestSolrJErrorHandling, ConcurrentUpdateSolrClientBuilderTest, ConcurrentUpdateSolrClientTest, HttpSolrClientBuilderTest, LBHttpSolrClientBuilderTest, TestCloudSolrClientConnections, GraphTest, SelectWithEvaluatorsTest, StreamExpressionToExpessionTest, StreamingTest]
   [junit4] Completed [138/146 (1!)] on J2 in 98.64s, 28 tests, 1 error, 3 skipped <<< FAILURES!

[...truncated 47167 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2249 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2249/

12 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test

Error Message:
Error from server at https://127.0.0.1:50429/nre: collection already exists: testcollection

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:50429/nre: collection already exists: testcollection
        at __randomizedtesting.SeedInfo.seed([CCA613C3F293E2E5:44F22C195C6F8F1D]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1675)
        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1702)
        at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test(ChaosMonkeySafeLeaderWithPullReplicasTest.java:225)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.FullSolrCloudDistribCmdsTest

Error Message:
116 threads leaked from SUITE scope at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest:     1) Thread[id=16864, name=Thread-2746, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1725)    2) Thread[id=16798, name=Connection evictor, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)    3) Thread[id=16839, name=Connection evictor, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)    4) Thread[id=16940, name=zkConnectionManagerCallback-3935-thread-1-processing-n:127.0.0.1:35834_-SendThread(127.0.0.1:53808), state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)    5) Thread[id=16895, name=zkCallback-3964-thread-1, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    6) Thread[id=16948, name=coreZkRegister-5435-thread-1, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    7) Thread[id=16939, name=coreZkRegister-5468-thread-1, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    8) Thread[id=16951, name=ScheduledTrigger-5475-thread-3, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)    9) Thread[id=16796, name=Thread-2728, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1725)   10) Thread[id=16860, name=zkCallback-3954-thread-1, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   11) Thread[id=16957, name=zkCallback-3944-thread-6, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   12) Thread[id=16845, name=qtp128831529-16845, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   13) Thread[id=16869, name=Scheduler-546217732, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   14) Thread[id=16875, name=qtp636122867-16875, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   15) Thread[id=16811, name=qtp1478415046-16811, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   16) Thread[id=16795, name=zkCallback-3934-thread-1, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   17) Thread[id=16847, name=qtp128831529-16847, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   18) Thread[id=16959, name=zkCallback-3944-thread-7, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   19) Thread[id=16874, name=Connection evictor, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   20) Thread[id=16927, name=zkConnectionManagerCallback-3955-thread-1-processing-n:127.0.0.1:45818_-SendThread(127.0.0.1:53808), state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)   21) Thread[id=16812, name=qtp1478415046-16812, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   22) Thread[id=16909, name=zkCallback-3934-thread-2, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   23) Thread[id=16846, name=qtp128831529-16846, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   24) Thread[id=16915, name=zkCallback-3964-thread-5, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   25) Thread[id=16852, name=Connection evictor, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   26) Thread[id=16881, name=qtp636122867-16881, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   27) Thread[id=16913, name=zkCallback-3954-thread-3, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   28) Thread[id=16917, name=zkConnectionManagerCallback-3945-thread-1-processing-n:127.0.0.1:47587_-SendThread(127.0.0.1:53808), state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)   29) Thread[id=16896, name=Thread-2755, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1725)   30) Thread[id=16899, name=zkCallback-3964-thread-2, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   31) Thread[id=16853, name=Connection evictor, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   32) Thread[id=16821, name=Connection evictor, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   33) Thread[id=16914, name=zkCallback-3954-thread-4, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   34) Thread[id=16814, name=qtp1478415046-16814, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   35) Thread[id=16953, name=ScheduledTrigger-5475-thread-4, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   36) Thread[id=16787, name=Connection evictor, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   37) Thread[id=16926, name=zkCallback-3954-thread-5, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   38) Thread[id=16866, name=Connection evictor, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   39) Thread[id=16774, name=qtp1857031859-16774, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   40) Thread[id=16793, name=zkConnectionManagerCallback-3935-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   41) Thread[id=16894, name=zkConnectionManagerCallback-3965-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   42) Thread[id=16907, name=zkCallback-3944-thread-3, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   43) Thread[id=16879, name=qtp636122867-16879, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   44) Thread[id=16810, name=qtp1478415046-16810-acceptor-0@5c777254-ServerConnector@19abb512{HTTP/1.1,[http/1.1]}{127.0.0.1:47587}, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   45) Thread[id=16859, name=zkConnectionManagerCallback-3955-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   46) Thread[id=16928, name=zkConnectionManagerCallback-3955-thread-1-processing-n:127.0.0.1:45818_-EventThread, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)   47) Thread[id=16815, name=qtp1478415046-16815, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   48) Thread[id=16880, name=qtp636122867-16880, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   49) Thread[id=16805, name=searcherExecutor-5441-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   50) Thread[id=16921, name=OverseerAutoScalingTriggerThread-99294364493414419-127.0.0.1:47587_-n_0000000005, state=WAITING, group=Overseer autoscaling triggers]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:166)         at java.lang.Thread.run(Thread.java:748)   51) Thread[id=16897, name=coreContainerWorkExecutor-5469-thread-1, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   52) Thread[id=16844, name=qtp128831529-16844, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   53) Thread[id=16809, name=qtp1478415046-16809, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   54) Thread[id=16919, name=OverseerStateUpdate-99294364493414419-127.0.0.1:47587_-n_0000000005, state=TIMED_WAITING, group=Overseer state updater.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.apache.solr.cloud.ZkDistributedQueue.peekElements(ZkDistributedQueue.java:463)         at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:203)         at java.lang.Thread.run(Thread.java:748)   55) Thread[id=16781, name=qtp1857031859-16781, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   56) Thread[id=16898, name=Connection evictor, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   57) Thread[id=16887, name=Connection evictor, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   58) Thread[id=16843, name=qtp128831529-16843, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   59) Thread[id=16956, name=zkCallback-3964-thread-6, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   60) Thread[id=16779, name=qtp1857031859-16779, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   61) Thread[id=16938, name=coreZkRegister-5457-thread-2, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   62) Thread[id=16786, name=Connection evictor, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   63) Thread[id=16834, name=Scheduler-2104140649, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   64) Thread[id=16780, name=qtp1857031859-16780, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   65) Thread[id=16958, name=zkCallback-3954-thread-6, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   66) Thread[id=16813, name=qtp1478415046-16813, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   67) Thread[id=16820, name=Connection evictor, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   68) Thread[id=16941, name=zkConnectionManagerCallback-3935-thread-1-processing-n:127.0.0.1:35834_-EventThread, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)   69) Thread[id=16908, name=zkCallback-3944-thread-4, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   70) Thread[id=16900, name=zkCallback-3964-thread-3, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   71) Thread[id=16831, name=Connection evictor, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   72) Thread[id=16888, name=Connection evictor, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   73) Thread[id=16911, name=zkCallback-3934-thread-4, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   74) Thread[id=16906, name=zkCallback-3944-thread-2, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   75) Thread[id=16837, name=searcherExecutor-5452-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   76) Thread[id=16943, name=ScheduledTrigger-5475-thread-2, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   77) Thread[id=16901, name=zkCallback-3964-thread-4, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   78) Thread[id=16922, name=zkCallback-3934-thread-5, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   79) Thread[id=16912, name=zkCallback-3954-thread-2, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   80) Thread[id=16778, name=qtp1857031859-16778, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   81) Thread[id=16878, name=qtp636122867-16878, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   82) Thread[id=16930, name=zkConnectionManagerCallback-3965-thread-1-processing-n:127.0.0.1:41210_-EventThread, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)   83) Thread[id=16955, name=zkCallback-3934-thread-6, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   84) Thread[id=16775, name=qtp1857031859-16775, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   85) Thread[id=16782, name=org.eclipse.jetty.server.session.HashSessionManager@7180dad4Timer, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   86) Thread[id=16776, name=qtp1857031859-16776-acceptor-0@1ce8b3da-ServerConnector@350c9ccc{HTTP/1.1,[http/1.1]}{127.0.0.1:35834}, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   87) Thread[id=16829, name=Thread-2737, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.solr.core.CloserThread.run(CoreContainer.java:1725)   88) Thread[id=16929, name=zkConnectionManagerCallback-3965-thread-1-processing-n:127.0.0.1:41210_-SendThread(127.0.0.1:53808), state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)   89) Thread[id=16877, name=qtp636122867-16877-acceptor-0@165d3c05-ServerConnector@3d9d8e90{HTTP/1.1,[http/1.1]}{127.0.0.1:41210}, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   90) Thread[id=16876, name=qtp636122867-16876, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   91) Thread[id=16803, name=watches-3936-thread-2, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   92) Thread[id=16840, name=qtp128831529-16840, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   93) Thread[id=16910, name=zkCallback-3934-thread-3, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   94) Thread[id=16870, name=watches-3956-thread-1, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   95) Thread[id=16936, name=ScheduledTrigger-5475-thread-1, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)         at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:364)         at org.apache.solr.cloud.ZkDistributedQueue.offer(ZkDistributedQueue.java:321)         at org.apache.solr.cloud.autoscaling.TriggerEventQueue.offerEvent(TriggerEventQueue.java:58)         at org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTrigger.enqueue(ScheduledTriggers.java:488)         at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$add$4(ScheduledTriggers.java:267)         at org.apache.solr.cloud.autoscaling.ScheduledTriggers$$Lambda$328/1617091509.process(Unknown Source)         at org.apache.solr.cloud.autoscaling.NodeLostTrigger.run(NodeLostTrigger.java:159)         at org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTrigger.run(ScheduledTriggers.java:536)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   96) Thread[id=16916, name=zkCallback-3944-thread-5, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   97) Thread[id=16842, name=qtp128831529-16842-acceptor-0@4c81a025-ServerConnector@6bcfa61f{HTTP/1.1,[http/1.1]}{127.0.0.1:45818}, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   98) Thread[id=16937, name=coreZkRegister-5457-thread-1, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   99) Thread[id=16872, name=searcherExecutor-5463-thread-1, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   100) Thread[id=16808, name=qtp1478415046-16808, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   101) Thread[id=16934, name=coreZkRegister-5446-thread-2, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   102) Thread[id=16949, name=coreZkRegister-5435-thread-2, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   103) Thread[id=16777, name=qtp1857031859-16777, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)         at java.lang.Thread.run(Thread.java:748)   104) Thread[id=16816, name=org.eclipse.jetty.server.session.HashSessionManager@59a8a933Timer, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   105) Thread[id=16848, name=org.eclipse.jetty.server.session.HashSessionManager@35347e14Timer, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   106) Thread[id=16807, name=Connection evictor, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)         at java.lang.Thread.run(Thread.java:748)   107) Thread[id=16918, name=zkConnectionManagerCallback-3945-thread-1-processing-n:127.0.0.1:47587_-EventThread, state=WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)   108) Thread[id=16799, name=Scheduler-1674535437, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)   109) Thread[id=16841, name=qtp128831529-16841, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)         at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)         at java.lang.Thread.run(Thread.java:748)   110) Thread[id=16920, name=OverseerCollectionConfigSetProcessor-99294364493414419-127.0.0.1:47587_-n_0000000005, state=TIMED_WAITING, group=Overseer collection creation process.]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111)         at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)         at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)         at org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:387)         at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:193)         at java.lang.Thread.run(Thread.java:748)   111) Thread[id=16828, name=zkCallback-3944-thread-1, state=TIMED_WAITING, group=TGRP-FullSolrCloudDistribCmdsTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.

[...truncated too long message...]

AFB99]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 324721 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[CCDB4C2436AAFB99]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2d0e0bdd: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@3d5c3f3f
   [junit4]   2> 324728 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[CCDB4C2436AAFB99]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 324728 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[CCDB4C2436AAFB99]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@8850c98: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@606580cb
   [junit4]   2> 324732 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[CCDB4C2436AAFB99]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 324733 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[CCDB4C2436AAFB99]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2aaead3e: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@44224f9d
   [junit4]   2> 324741 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4aeece74
   [junit4]   2> 324741 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1257164404
   [junit4]   2> 324742 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@61d60d9d: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@66a14f3c
   [junit4]   2> 324905 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]: now flush at close
   [junit4]   2> 324905 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]:   start flush: applyAllDeletes=true
   [junit4]   2> 324906 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]:   index before flush _0(8.0.0):C272 _1(8.0.0):C228 _2(8.0.0):C1
   [junit4]   2> 324906 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-199-thread-1]: startFullFlush
   [junit4]   2> 324906 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-199-thread-1]: coreCloseExecutor-199-thread-1 finishFullFlush success=true
   [junit4]   2> 324906 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]: now apply all deletes for all segments buffered updates bytesUsed=0 reader pool bytesUsed=0
   [junit4]   2> 324906 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-199-thread-1]: waitApply: no deletes to apply
   [junit4]   2> 324945 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-199-thread-1]: updateMergeThreads ioThrottle=true targetMBPerSec=10240.0 MB/sec
   [junit4]   2> 324945 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-199-thread-1]: now merge
   [junit4]   2> 324945 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-199-thread-1]:   index: _0(8.0.0):C272 _1(8.0.0):C228 _2(8.0.0):C1
   [junit4]   2> 324945 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-199-thread-1]:   no more merges pending; now return
   [junit4]   2> 324945 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]: waitForMerges
   [junit4]   2> 324945 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]: waitForMerges done
   [junit4]   2> 324945 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]: commit: start
   [junit4]   2> 324945 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]: commit: enter lock
   [junit4]   2> 324945 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]: commit: now prepare
   [junit4]   2> 324945 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]: prepareCommit: flush
   [junit4]   2> 324945 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]:   index before flush _0(8.0.0):C272 _1(8.0.0):C228 _2(8.0.0):C1
   [junit4]   2> 324945 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-199-thread-1]: startFullFlush
   [junit4]   2> 324945 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]: now apply all deletes for all segments buffered updates bytesUsed=0 reader pool bytesUsed=0
   [junit4]   2> 324946 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-199-thread-1]: waitApply: no deletes to apply
   [junit4]   2> 324946 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-199-thread-1]: coreCloseExecutor-199-thread-1 finishFullFlush success=true
   [junit4]   2> 324946 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]: startCommit(): start
   [junit4]   2> 324946 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]:   skip startCommit(): no changes pending
   [junit4]   2> 324946 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]: commit: pendingCommit == null; skip
   [junit4]   2> 324946 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]: commit: took 0.5 msec
   [junit4]   2> 324946 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]: commit: done
   [junit4]   2> 324946 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]: rollback
   [junit4]   2> 324946 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]: all running merges have aborted
   [junit4]   2> 325081 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]: rollback: done finish merges
   [junit4]   2> 325081 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-199-thread-1]: abort
   [junit4]   2> 325081 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-199-thread-1]: done abort success=true
   [junit4]   2> 325081 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-199-thread-1]: rollback: infos=_0(8.0.0):C272 _1(8.0.0):C228 _2(8.0.0):C1
   [junit4]   2> 325081 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-199-thread-1]: now checkpoint "_0(8.0.0):C272 _1(8.0.0):C228 _2(8.0.0):C1" [3 segments ; isCommit = false]
   [junit4]   2> 325081 INFO  (coreCloseExecutor-199-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-199-thread-1]: 0 msec to checkpoint
   [junit4]   2> 325310 INFO  (SUITE-ConcurrentUpdateSolrClientTest-seed#[CCDB4C2436AAFB99]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@552827aa{/solr,null,UNAVAILABLE}
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.impl.ConcurrentUpdateSolrClientTest_CCDB4C2436AAFB99-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1326, maxMBSortInHeap=5.8238623254794595, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@2ae706be), locale=fi, timezone=Africa/Bujumbura
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=235764408,total=358088704
   [junit4]   2> NOTE: All tests run in this JVM: [BasicHttpSolrClientTest, CommonAdminParamsTest, AbsoluteValueEvaluatorTest, ClientUtilsTest, AppendEvaluatorTest, FacetFieldTest, SubtractEvaluatorTest, GraphTest, HyperbolicCosineEvaluatorTest, TestRetryUtil, RecursiveEvaluatorTest, TestLBHttpSolrClient, CeilingEvaluatorTest, JsonValidatorTest, ConcurrentUpdateSolrClientTest]
   [junit4] Completed [31/146 (1!)] on J1 in 32.47s, 3 tests, 1 failure <<< FAILURES!

[...truncated 47495 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2250 - Failure

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2250/

8 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:44121/collection1: Cannot talk to ZooKeeper - Updates are disabled.

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:44121/collection1: Cannot talk to ZooKeeper - Updates are disabled.
        at __randomizedtesting.SeedInfo.seed([F0CA3935467EAEDB:789E06EFE882C323]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:484)
        at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:463)
        at org.apache.solr.cloud.BasicDistributedZkTest.testUpdateProcessorsRunOnlyOnce(BasicDistributedZkTest.java:685)
        at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:377)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.TestCloudRecovery.leaderRecoverFromLogOnStartupTest

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
        at __randomizedtesting.SeedInfo.seed([F0CA3935467EAEDB]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestCloudRecovery

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
        at __randomizedtesting.SeedInfo.seed([F0CA3935467EAEDB]:0)


FAILED:  org.apache.solr.cloud.PeerSyncReplicationTest.test

Error Message:
expected:<0> but was:<1>

Stack Trace:
java.lang.AssertionError: expected:<0> but was:<1>
        at __randomizedtesting.SeedInfo.seed([F0CA3935467EAEDB:789E06EFE882C323]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:203)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue

Error Message:
action wasn't interrupted

Stack Trace:
java.lang.AssertionError: action wasn't interrupted
        at __randomizedtesting.SeedInfo.seed([F0CA3935467EAEDB:397F7B9B4F19682E]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue(TestTriggerIntegration.java:638)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeMarkersRegistration

Error Message:
Path /autoscaling/nodeAdded/127.0.0.1:10022_solr should have been deleted

Stack Trace:
java.lang.AssertionError: Path /autoscaling/nodeAdded/127.0.0.1:10022_solr should have been deleted
        at __randomizedtesting.SeedInfo.seed([F0CA3935467EAEDB:E870B139484B6334]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertFalse(Assert.java:68)
        at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeMarkersRegistration(TestTriggerIntegration.java:842)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.core.HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig

Error Message:
The max direct memory is likely too low.  Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~134.217728 MB.

Stack Trace:
java.lang.RuntimeException: The max direct memory is likely too low.  Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~134.217728 MB.
        at __randomizedtesting.SeedInfo.seed([F0CA3935467EAEDB:765F01EBBF744F0]:0)
        at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:312)
        at org.apache.solr.core.HdfsDirectoryFactory.getBlockDirectoryCache(HdfsDirectoryFactory.java:288)
        at org.apache.solr.core.HdfsDirectoryFactory.create(HdfsDirectoryFactory.java:228)
        at org.apache.solr.core.HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig(HdfsDirectoryFactoryTest.java:111)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.OutOfMemoryError
        at sun.misc.Unsafe.allocateMemory(Native Method)
        at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:127)
        at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
        at org.apache.solr.store.blockcache.BlockCache.<init>(BlockCache.java:71)
        at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:310)
        ... 42 more


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode

Error Message:
did not finish processing changes

Stack Trace:
java.lang.AssertionError: did not finish processing changes
        at __randomizedtesting.SeedInfo.seed([F0CA3935467EAEDB:57252496893321C3]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode(TestLargeCluster.java:299)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12327 lines...]
   [junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/init-core-data-001
   [junit4]   2> 2588885 WARN  (SUITE-BasicDistributedZkTest-seed#[F0CA3935467EAEDB]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=10 numCloses=10
   [junit4]   2> 2588982 INFO  (SUITE-BasicDistributedZkTest-seed#[F0CA3935467EAEDB]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 2588984 INFO  (SUITE-BasicDistributedZkTest-seed#[F0CA3935467EAEDB]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 2588984 INFO  (SUITE-BasicDistributedZkTest-seed#[F0CA3935467EAEDB]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 2588984 INFO  (SUITE-BasicDistributedZkTest-seed#[F0CA3935467EAEDB]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 2589007 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2589059 INFO  (Thread-2703) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2589059 INFO  (Thread-2703) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2589107 ERROR (Thread-2703) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2589187 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.ZkTestServer start zk server on port:47174
   [junit4]   2> 2589262 INFO  (zkConnectionManagerCallback-2692-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2589294 INFO  (zkConnectionManagerCallback-2694-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2589329 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2589363 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2589387 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2589388 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2589408 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2589413 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2589429 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2589448 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2589463 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2589482 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2589488 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2589509 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
   [junit4]   2> 2593031 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 2593135 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@d2fa5a7{/,null,AVAILABLE}
   [junit4]   2> 2593137 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@567c0a6d{HTTP/1.1,[http/1.1]}{127.0.0.1:58372}
   [junit4]   2> 2593137 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.e.j.s.Server Started @2617253ms
   [junit4]   2> 2593137 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/tempDir-001/control/data, hostContext=/, hostPort=58372, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/control-001/cores}
   [junit4]   2> 2593143 ERROR (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2593143 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 2593143 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2593143 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2593143 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-07T06:56:21.968Z
   [junit4]   2> 2593300 INFO  (zkConnectionManagerCallback-2696-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2593302 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2593302 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/control-001/solr.xml
   [junit4]   2> 2593320 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 2593320 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 2593322 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@77399015, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 2593395 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47174/solr
   [junit4]   2> 2593397 INFO  (zkConnectionManagerCallback-2700-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2593483 INFO  (zkConnectionManagerCallback-2702-thread-1-processing-n:127.0.0.1:58372_) [n:127.0.0.1:58372_    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2594227 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:58372_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2594228 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:58372_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:58372_
   [junit4]   2> 2594229 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:58372_    ] o.a.s.c.Overseer Overseer (id=99307236745871364-127.0.0.1:58372_-n_0000000000) starting
   [junit4]   2> 2594459 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:58372_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58372_
   [junit4]   2> 2594575 INFO  (zkCallback-2701-thread-1-processing-n:127.0.0.1:58372_) [n:127.0.0.1:58372_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2595701 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:58372_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77399015
   [junit4]   2> 2595959 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:58372_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77399015
   [junit4]   2> 2595959 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:58372_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77399015
   [junit4]   2> 2595962 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:58372_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/control-001/cores
   [junit4]   2> 2596315 INFO  (zkConnectionManagerCallback-2708-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2596316 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2596323 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:47174/solr ready
   [junit4]   2> 2596508 INFO  (qtp361529707-7498) [n:127.0.0.1:58372_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:58372_&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2596514 INFO  (OverseerThreadFactory-1277-thread-1-processing-n:127.0.0.1:58372_) [n:127.0.0.1:58372_    ] o.a.s.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 2596743 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2596744 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 2596956 INFO  (zkCallback-2701-thread-1-processing-n:127.0.0.1:58372_) [n:127.0.0.1:58372_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2598642 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2598821 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 2600481 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2600608 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 2600609 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77399015
   [junit4]   2> 2600609 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2600609 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 2600695 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=20, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.17076751499946008]
   [junit4]   2> 2600856 WARN  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2603495 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2603495 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2603497 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2603497 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2603499 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=38, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 2603531 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5281aa28[control_collection_shard1_replica_n1] main]
   [junit4]   2> 2603572 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2603572 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2603573 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2603593 INFO  (searcherExecutor-1280-thread-1-processing-n:127.0.0.1:58372_ x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@5281aa28[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2603611 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588915803191771136
   [junit4]   2> 2603672 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2603672 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2603672 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:58372/control_collection_shard1_replica_n1/
   [junit4]   2> 2603672 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2603672 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:58372/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 2603672 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2603712 INFO  (zkCallback-2701-thread-1-processing-n:127.0.0.1:58372_) [n:127.0.0.1:58372_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2603713 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:58372/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 2603714 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2603744 INFO  (qtp361529707-7496) [n:127.0.0.1:58372_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=7000
   [junit4]   2> 2603825 INFO  (qtp361529707-7498) [n:127.0.0.1:58372_    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 2603826 INFO  (zkCallback-2701-thread-1-processing-n:127.0.0.1:58372_) [n:127.0.0.1:58372_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2604576 INFO  (OverseerCollectionConfigSetProcessor-99307236745871364-127.0.0.1:58372_-n_0000000000) [n:127.0.0.1:58372_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2604859 INFO  (qtp361529707-7498) [n:127.0.0.1:58372_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:58372_&wt=javabin&version=2} status=0 QTime=8350
   [junit4]   2> 2605100 INFO  (zkConnectionManagerCallback-2712-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2605102 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2605106 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:47174/solr ready
   [junit4]   2> 2605106 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2605108 INFO  (qtp361529707-7499) [n:127.0.0.1:58372_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2605123 INFO  (OverseerThreadFactory-1277-thread-2-processing-n:127.0.0.1:58372_) [n:127.0.0.1:58372_    ] o.a.s.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 2605131 WARN  (OverseerThreadFactory-1277-thread-2-processing-n:127.0.0.1:58372_) [n:127.0.0.1:58372_    ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 2605509 INFO  (qtp361529707-7499) [n:127.0.0.1:58372_    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 2605510 INFO  (qtp361529707-7499) [n:127.0.0.1:58372_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=402
   [junit4]   2> 2607140 INFO  (OverseerCollectionConfigSetProcessor-99307236745871364-127.0.0.1:58372_-n_0000000000) [n:127.0.0.1:58372_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2608194 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/shard-1-001 of type NRT
   [junit4]   2> 2608211 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 2608213 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6614e024{/,null,AVAILABLE}
   [junit4]   2> 2608213 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7170ff8f{HTTP/1.1,[http/1.1]}{127.0.0.1:32781}
   [junit4]   2> 2608213 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.e.j.s.Server Started @2632329ms
   [junit4]   2> 2608213 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/tempDir-001/jetty1, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=32781, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/shard-1-001/cores}
   [junit4]   2> 2608213 ERROR (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2608214 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 2608214 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2608214 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2608214 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-07T06:56:37.039Z
   [junit4]   2> 2609266 INFO  (zkConnectionManagerCallback-2714-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2609269 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2609269 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/shard-1-001/solr.xml
   [junit4]   2> 2609290 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 2609290 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 2609320 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@77399015, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 2609335 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47174/solr
   [junit4]   2> 2609427 INFO  (zkConnectionManagerCallback-2718-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2609459 INFO  (zkConnectionManagerCallback-2720-thread-1-processing-n:127.0.0.1:32781_) [n:127.0.0.1:32781_    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2610132 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:32781_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2610143 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:32781_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2610195 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:32781_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32781_
   [junit4]   2> 2610235 INFO  (zkCallback-2701-thread-1-processing-n:127.0.0.1:58372_) [n:127.0.0.1:58372_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2610455 INFO  (zkCallback-2711-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2610456 INFO  (zkCallback-2719-thread-1-processing-n:127.0.0.1:32781_) [n:127.0.0.1:32781_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2611614 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:32781_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77399015
   [junit4]   2> 2612708 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:32781_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77399015
   [junit4]   2> 2612709 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:32781_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77399015
   [junit4]   2> 2612747 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:32781_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/shard-1-001/cores
   [junit4]   2> 2613057 INFO  (qtp1442292805-7545) [n:127.0.0.1:32781_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:32781_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2613105 INFO  (OverseerThreadFactory-1277-thread-3-processing-n:127.0.0.1:58372_) [n:127.0.0.1:58372_    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:32781_ for creating new replica
   [junit4]   2> 2613121 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2613121 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 2613217 INFO  (zkCallback-2719-thread-1-processing-n:127.0.0.1:32781_) [n:127.0.0.1:32781_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2615984 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2616198 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema [collection1_shard2_replica_n41] Schema name=test
   [junit4]   2> 2618207 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2618923 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n41' using configuration from collection collection1, trusted=true
   [junit4]   2> 2618924 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n41' (registry 'solr.core.collection1.shard2.replica_n41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77399015
   [junit4]   2> 2618924 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2618925 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [[collection1_shard2_replica_n41] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/shard-1-001/cores/collection1_shard2_replica_n41], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/shard-1-001/cores/collection1_shard2_replica_n41/data/]
   [junit4]   2> 2618976 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=20, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.17076751499946008]
   [junit4]   2> 2618993 WARN  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2620048 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2620048 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2620049 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2620049 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2620095 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=38, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 2620128 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.SolrIndexSearcher Opening [Searcher@2ff5832a[collection1_shard2_replica_n41] main]
   [junit4]   2> 2620147 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2620148 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2620149 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2620184 INFO  (searcherExecutor-1291-thread-1-processing-n:127.0.0.1:32781_ x:collection1_shard2_replica_n41 s:shard2 c:collection1) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [collection1_shard2_replica_n41] Registered new searcher Searcher@2ff5832a[collection1_shard2_replica_n41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2620184 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588915820569821184
   [junit4]   2> 2620248 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2620248 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2620248 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:32781/collection1_shard2_replica_n41/
   [junit4]   2> 2620248 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2620248 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy http://127.0.0.1:32781/collection1_shard2_replica_n41/ has no replicas
   [junit4]   2> 2620248 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2620287 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:32781/collection1_shard2_replica_n41/ shard2
   [junit4]   2> 2620301 INFO  (zkCallback-2719-thread-1-processing-n:127.0.0.1:32781_) [n:127.0.0.1:32781_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2620347 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2620365 INFO  (qtp1442292805-7543) [n:127.0.0.1:32781_ c:collection1 s:shard2  x:collection1_shard2_replica_n41] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=7244
   [junit4]   2> 2620390 INFO  (qtp1442292805-7545) [n:127.0.0.1:32781_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:32781_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=7332
   [junit4]   2> 2620467 INFO  (zkCallback-2719-thread-1-processing-n:127.0.0.1:32781_) [n:127.0.0.1:32781_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2621155 INFO  (OverseerCollectionConfigSetProcessor-99307236745871364-127.0.0.1:58372_-n_0000000000) [n:127.0.0.1:58372_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2624179 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/shard-2-001 of type NRT
   [junit4]   2> 2624195 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 2624277 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@15afe159{/,null,AVAILABLE}
   [junit4]   2> 2624278 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@63442b0f{HTTP/1.1,[http/1.1]}{127.0.0.1:58836}
   [junit4]   2> 2624278 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.e.j.s.Server Started @2648394ms
   [junit4]   2> 2624278 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=58836, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/shard-2-001/cores}
   [junit4]   2> 2624278 ERROR (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2624285 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 2624285 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2624285 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2624285 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-07T06:56:53.110Z
   [junit4]   2> 2624346 INFO  (zkConnectionManagerCallback-2724-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2624347 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2624347 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/shard-2-001/solr.xml
   [junit4]   2> 2624352 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 2624352 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 2624353 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@77399015, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 2624364 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47174/solr
   [junit4]   2> 2624403 INFO  (zkConnectionManagerCallback-2728-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2624407 INFO  (zkConnectionManagerCallback-2730-thread-1-processing-n:127.0.0.1:58836_) [n:127.0.0.1:58836_    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2624414 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:58836_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2624416 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:58836_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2624419 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:58836_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58836_
   [junit4]   2> 2624420 INFO  (zkCallback-2711-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2624446 INFO  (zkCallback-2729-thread-1-processing-n:127.0.0.1:58836_) [n:127.0.0.1:58836_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2624447 INFO  (zkCallback-2719-thread-1-processing-n:127.0.0.1:32781_) [n:127.0.0.1:32781_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2624462 INFO  (zkCallback-2701-thread-2-processing-n:127.0.0.1:58372_) [n:127.0.0.1:58372_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 2624790 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:58836_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77399015
   [junit4]   2> 2624950 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:58836_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77399015
   [junit4]   2> 2624950 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:58836_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77399015
   [junit4]   2> 2624980 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [n:127.0.0.1:58836_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/shard-2-001/cores
   [junit4]   2> 2627672 INFO  (qtp1442292805-7546) [n:127.0.0.1:32781_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:58836_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2627735 INFO  (OverseerThreadFactory-1277-thread-4-processing-n:127.0.0.1:58372_) [n:127.0.0.1:58372_    ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:58836_ for creating new replica
   [junit4]   2> 2627918 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2627919 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 2627966 INFO  (zkCallback-2719-thread-1-processing-n:127.0.0.1:32781_) [n:127.0.0.1:32781_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2628151 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2628416 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema [collection1_shard1_replica_n43] Schema name=test
   [junit4]   2> 2630428 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2632568 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n43' using configuration from collection collection1, trusted=true
   [junit4]   2> 2632568 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n43' (registry 'solr.core.collection1.shard1.replica_n43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77399015
   [junit4]   2> 2632569 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2632569 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [[collection1_shard1_replica_n43] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/shard-2-001/cores/collection1_shard1_replica_n43], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/shard-2-001/cores/collection1_shard1_replica_n43/data/]
   [junit4]   2> 2632606 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=20, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.17076751499946008]
   [junit4]   2> 2632630 WARN  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 2633383 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2633383 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2633392 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2633392 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2633499 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=38, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 2633503 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.SolrIndexSearcher Opening [Searcher@55aaf85b[collection1_shard1_replica_n43] main]
   [junit4]   2> 2633524 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2633524 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2633526 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2633528 INFO  (searcherExecutor-1302-thread-1-processing-n:127.0.0.1:58836_ x:collection1_shard1_replica_n43 s:shard1 c:collection1) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [collection1_shard1_replica_n43] Registered new searcher Searcher@55aaf85b[collection1_shard1_replica_n43] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2633543 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1588915834577747968
   [junit4]   2> 2633625 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2633625 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2633625 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:58836/collection1_shard1_replica_n43/
   [junit4]   2> 2633625 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2633625 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy http://127.0.0.1:58836/collection1_shard1_replica_n43/ has no replicas
   [junit4]   2> 2633625 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2633708 INFO  (zkCallback-2729-thread-1-processing-n:127.0.0.1:58836_) [n:127.0.0.1:58836_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2633709 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:58836/collection1_shard1_replica_n43/ shard1
   [junit4]   2> 2633710 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2633715 INFO  (zkCallback-2719-thread-1-processing-n:127.0.0.1:32781_) [n:127.0.0.1:32781_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2633787 INFO  (qtp2061996360-7578) [n:127.0.0.1:58836_ c:collection1 s:shard1  x:collection1_shard1_replica_n43] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=5868
   [junit4]   2> 2633807 INFO  (qtp1442292805-7546) [n:127.0.0.1:32781_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:58836_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=6135
   [junit4]   2> 2633828 INFO  (zkCallback-2719-thread-1-processing-n:127.0.0.1:32781_) [n:127.0.0.1:32781_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2633828 INFO  (zkCallback-2729-thread-1-processing-n:127.0.0.1:58836_) [n:127.0.0.1:58836_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 2635740 INFO  (OverseerCollectionConfigSetProcessor-99307236745871364-127.0.0.1:58372_-n_0000000000) [n:127.0.0.1:58372_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2637746 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.BasicDistributedZkTest_F0CA3935467EAEDB-001/shard-3-001 of type NRT
   [junit4]   2> 2637764 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 2637853 INFO  (TEST-BasicDistributedZkTest.test-seed#[F0CA3935467EAEDB]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7ed43914{/,null,AVAILABLE}
   [junit4]

[...truncated too long message...]

ore/test-lib/apacheds-core-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-i18n-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/bcprov-jdk15on-1.54.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/byte-buddy-1.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/commons-math3-3.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/hadoop-common-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/hadoop-hdfs-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/hadoop-minikdc-2.7.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/jetty-sslengine-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/mockito-core-2.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/test-lib/objenesis-2.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/analysis/icu/lucene-analyzers-icu-8.0.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/build/backward-codecs/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/contrib/analysis-extras/lib/icu4j-59.1.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.4.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_144/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.5.3.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/temp/junit4-J2-20180107_061244_3046405698271743522883.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/temp/junit4-J2-20180107_061244_3048802237980850734790.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Forked process returned with error code: 1. Very likely a JVM crash.  See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/temp/junit4-J2-20180107_061244_304662966802451399547.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/temp/junit4-J2-20180107_061244_3047417112679670004727.syserr
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
   [junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
   [junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4] at java.lang.Thread.run(Thread.java:748)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:835: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:779: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build.xml:262: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/common-build.xml:556: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:1512: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:1038: At least one slave process threw an exception, first: Forked process returned with error code: 137.

Total time: 424 minutes 30 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[Fast Archiver] No artifacts from Lucene-Solr-Tests-master #2249 to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2251 - Still unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2251/

13 tests failed.
FAILED:  org.apache.lucene.classification.BooleanPerceptronClassifierTest.testPerformance

Error Message:
training took more than 10s: 12s

Stack Trace:
java.lang.AssertionError: training took more than 10s: 12s
        at __randomizedtesting.SeedInfo.seed([6AE944E07A33D8D5:AD08B6C21187E07A]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.lucene.classification.BooleanPerceptronClassifierTest.testPerformance(BooleanPerceptronClassifierTest.java:93)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.handler.dataimport.TestContentStreamDataSource.testCommitWithin

Error Message:
expected:<0> but was:<2>

Stack Trace:
java.lang.AssertionError: expected:<0> but was:<2>
        at __randomizedtesting.SeedInfo.seed([C3DC9995477B3A24:790EF6EDC455D431]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:472)
        at org.junit.Assert.assertEquals(Assert.java:456)
        at org.apache.solr.handler.dataimport.TestContentStreamDataSource.testCommitWithin(TestContentStreamDataSource.java:98)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.CustomCollectionTest.testCreateShardRepFactor

Error Message:
Error from server at http://127.0.0.1:38667/solr: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qnr-0000000012

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:38667/solr: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qnr-0000000012
        at __randomizedtesting.SeedInfo.seed([6CA08B043810C623:984E01802044D3E3]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:444)
        at org.apache.solr.cloud.CustomCollectionTest.ensureClusterEmpty(CustomCollectionTest.java:53)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:968)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit

Error Message:
Shard split did not succeed after a previous failed split attempt left sub-shards in construction state

Stack Trace:
java.lang.AssertionError: Shard split did not succeed after a previous failed split attempt left sub-shards in construction state
        at __randomizedtesting.SeedInfo.seed([6CA08B043810C623:95ED18AB04658BA9]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit(ShardSplitTest.java:294)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
        at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:     1) Thread[id=5221, name=jetty-launcher-2493-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)         at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)         at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)         at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)         at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)         at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)         at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)         at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)         at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)         at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
   1) Thread[id=5221, name=jetty-launcher-2493-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
        at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
        at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
        at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
        at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
        at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
        at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
        at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
        at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
        at __randomizedtesting.SeedInfo.seed([6CA08B043810C623]:0)


FAILED:  org.apache.solr.cloud.autoscaling.AutoScalingHandlerTest.testSuspendTrigger

Error Message:
KeeperErrorCode = Session expired for /autoscaling.json

Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling.json
        at __randomizedtesting.SeedInfo.seed([6CA08B043810C623:AEA6B27F7AB88782]:0)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
        at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
        at org.apache.zookeeper.ZooKeeper.setData(ZooKeeper.java:1327)
        at org.apache.solr.common.cloud.SolrZkClient.lambda$setData$6(SolrZkClient.java:351)
        at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
        at org.apache.solr.common.cloud.SolrZkClient.setData(SolrZkClient.java:351)
        at org.apache.solr.common.cloud.SolrZkClient.setData(SolrZkClient.java:539)
        at org.apache.solr.cloud.autoscaling.AutoScalingHandlerTest.beforeTest(AutoScalingHandlerTest.java:112)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:968)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeMarkersRegistration

Error Message:
Path /autoscaling/nodeAdded/127.0.0.1:39247_solr wasn't created

Stack Trace:
java.lang.AssertionError: Path /autoscaling/nodeAdded/127.0.0.1:39247_solr wasn't created
        at __randomizedtesting.SeedInfo.seed([6CA08B043810C623:741A030836250BCC]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeMarkersRegistration(TriggerIntegrationTest.java:936)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeLostTriggerRestoreState

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
        at __randomizedtesting.SeedInfo.seed([6CA08B043810C623:475F5E5FA268D3F3]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeLostTriggerRestoreState(TestTriggerIntegration.java:307)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.core.snapshots.TestSolrCloudSnapshots.testSnapshots

Error Message:
expected:<COMPLETED> but was:<RUNNING>

Stack Trace:
java.lang.AssertionError: expected:<COMPLETED> but was:<RUNNING>
        at __randomizedtesting.SeedInfo.seed([6CA08B043810C623:5AEC0F9B4D49C46C]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.failNotEquals(Assert.java:647)
        at org.junit.Assert.assertEquals(Assert.java:128)
        at org.junit.Assert.assertEquals(Assert.java:147)
        at org.apache.solr.core.snapshots.TestSolrCloudSnapshots.testSnapshots(TestSolrCloudSnapshots.java:189)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.update.SoftAutoCommitTest.testHardCommitWithinAndSoftCommitMaxTimeRapidAdds

Error Message:
softCommit: did not find a single commit

Stack Trace:
java.lang.AssertionError: softCommit: did not find a single commit
        at __randomizedtesting.SeedInfo.seed([6CA08B043810C623:F90E2D53DA02B45E]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertNotNull(Assert.java:526)
        at org.apache.solr.update.SoftAutoCommitTest.assertRapidMultiCommitQueues(SoftAutoCommitTest.java:495)
        at org.apache.solr.update.SoftAutoCommitTest.doTestSoftAndHardCommitMaxTimeRapidAdds(SoftAutoCommitTest.java:460)
        at org.apache.solr.update.SoftAutoCommitTest.testHardCommitWithinAndSoftCommitMaxTimeRapidAdds(SoftAutoCommitTest.java:412)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest.test

Error Message:
Error from server at http://127.0.0.1:60983/solr: no core retrieved for timeConfig

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException: Error from server at http://127.0.0.1:60983/solr: no core retrieved for timeConfig
        at __randomizedtesting.SeedInfo.seed([6CA08B043810C623:E4F4B4DE96ECABDB]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException.create(HttpSolrClient.java:829)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:620)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
        at org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest.test(TimeRoutedAliasUpdateProcessorTest.java:95)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue

Error Message:
action wasn't interrupted

Stack Trace:
java.lang.AssertionError: action wasn't interrupted
        at __randomizedtesting.SeedInfo.seed([6CA08B043810C623:A515C9AA317700D6]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue(TestTriggerIntegration.java:638)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeMarkersRegistration

Error Message:
Path /autoscaling/nodeAdded/127.0.0.1:10013_solr should have been deleted

Stack Trace:
java.lang.AssertionError: Path /autoscaling/nodeAdded/127.0.0.1:10013_solr should have been deleted
        at __randomizedtesting.SeedInfo.seed([6CA08B043810C623:741A030836250BCC]:0)
        at org.junit.Assert.fail(Assert.java:93)
        at org.junit.Assert.assertTrue(Assert.java:43)
        at org.junit.Assert.assertFalse(Assert.java:68)
        at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeMarkersRegistration(TestTriggerIntegration.java:842)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 6448 lines...]
   [junit4] Suite: org.apache.lucene.classification.BooleanPerceptronClassifierTest
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BooleanPerceptronClassifierTest -Dtests.method=testPerformance -Dtests.seed=6AE944E07A33D8D5 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=es-SV -Dtests.timezone=Australia/LHI -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 17.3s J2 | BooleanPerceptronClassifierTest.testPerformance <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: training took more than 10s: 12s
   [junit4]    > at __randomizedtesting.SeedInfo.seed([6AE944E07A33D8D5:AD08B6C21187E07A]:0)
   [junit4]    > at org.apache.lucene.classification.BooleanPerceptronClassifierTest.testPerformance(BooleanPerceptronClassifierTest.java:93)
   [junit4]    > at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@654ba3d3), locale=es-SV, timezone=Australia/LHI
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=197609848,total=341311488
   [junit4]   2> NOTE: All tests run in this JVM: [SimpleNaiveBayesDocumentClassifierTest, CachingNaiveBayesClassifierTest, BooleanPerceptronClassifierTest]
   [junit4] Completed [9/11 (1!)] on J2 in 21.03s, 4 tests, 1 failure <<< FAILURES!

[...truncated 5557 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
   [junit4]   2> 1143942 INFO  (SUITE-TestTriggerIntegration-seed#[6CA08B043810C623]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_6CA08B043810C623-001/init-core-data-001
   [junit4]   2> 1143944 WARN  (SUITE-TestTriggerIntegration-seed#[6CA08B043810C623]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=228 numCloses=228
   [junit4]   2> 1143944 INFO  (SUITE-TestTriggerIntegration-seed#[6CA08B043810C623]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1143946 INFO  (SUITE-TestTriggerIntegration-seed#[6CA08B043810C623]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1144015 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 1144016 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
   [junit4]   2> 1144016 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 1144016 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
   [junit4]   2> 1144016 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
   [junit4]   2> 1144065 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTrigger
   [junit4]   2> 1144065 DEBUG (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 1144067 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1144129 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1144129 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException
   [junit4]   2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
   [junit4]   2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
   [junit4]   2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:154)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1144129 DEBUG (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1144184 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 1144185 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 1144185 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 1144185 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
   [junit4]   2> 1144185 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 1144190 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1144190 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 1144191 DEBUG (ScheduledTrigger-489-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1144214 DEBUG (ScheduledTrigger-489-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1144269 DEBUG (ScheduledTrigger-489-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1144290 DEBUG (ScheduledTrigger-489-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1144311 DEBUG (ScheduledTrigger-489-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1144333 DEBUG (ScheduledTrigger-489-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1144355 DEBUG (ScheduledTrigger-489-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1144357 DEBUG (simCloudManagerPool-488-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1144357 DEBUG (simCloudManagerPool-488-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 1144365 DEBUG (simCloudManagerPool-488-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 1144365 DEBUG (simCloudManagerPool-488-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
   [junit4]   2> 1144365 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 1144366 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 1144366 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1144366 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 1144366 DEBUG (ScheduledTrigger-489-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
   [junit4]   2> 1144381 DEBUG (ScheduledTrigger-489-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1144381 DEBUG (ScheduledTrigger-489-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10000_solr
   [junit4]   2> 1144390 DEBUG (ScheduledTrigger-489-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
   [junit4]   2> 1144390 DEBUG (ScheduledTrigger-489-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10000_solr
   [junit4]   2> 1144409 DEBUG (ScheduledTrigger-489-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1144415 DEBUG (ScheduledTrigger-489-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
   [junit4]   2> 1144415 DEBUG (ScheduledTrigger-489-thread-2) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10000_solr]
   [junit4]   2> 1144416 DEBUG (ScheduledTrigger-489-thread-2) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"37a339b5b2a076T7liad83y2wbgbnje8naj1foct",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":15660591975932022,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[15660591975932022],
   [junit4]   2>     "nodeNames":["127.0.0.1:10000_solr"]}}
   [junit4]   2> 1144561 DEBUG (AutoscalingActionExecutor-490-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"37a339b5b2a076T7liad83y2wbgbnje8naj1foct",
   [junit4]   2>   "source":"node_lost_trigger",
   [junit4]   2>   "eventTime":15660591975932022,
   [junit4]   2>   "eventType":"NODELOST",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[15660591975932022],
   [junit4]   2>     "_enqueue_time_":15660593255090572,
   [junit4]   2>     "nodeNames":["127.0.0.1:10000_solr"]}}
   [junit4]   2> 1144589 DEBUG (simCloudManagerPool-488-thread-3) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1144589 DEBUG (simCloudManagerPool-488-thread-3) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 1144589 DEBUG (simCloudManagerPool-488-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr]
   [junit4]   2> 1144589 DEBUG (simCloudManagerPool-488-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr]
   [junit4]   2> 1144589 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 1144594 DEBUG (AutoscalingActionExecutor-490-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 1641 ms for event id=37a339b5b2a076T7liad83y2wbgbnje8naj1foct
   [junit4]   2> 1144598 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1144598 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 1144604 DEBUG (ScheduledTrigger-489-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
   [junit4]   2> 1144617 DEBUG (ScheduledTrigger-489-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1144629 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testNodeLostTrigger
   [junit4]   2> 1144629 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase
   [junit4]   2>
   [junit4]   2> 1144629 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
   [junit4]   2> 1144629 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
   [junit4]   2> 1144629 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
   [junit4]   2>
   [junit4]   2> 1144629 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 1
   [junit4]   2> 1144630 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 1
   [junit4]   2> 1144630 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 1
   [junit4]   2> 1144630 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10000_solr
   [junit4]   2> 1144631 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
   [junit4]   2> 1144631 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
   [junit4]   2> 1144631 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
   [junit4]   2> 1144631 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
   [junit4]   2> 1144631 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
   [junit4]   2> 1144632 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling        2
   [junit4]   2> 1144632 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update             2
   [junit4]   2> 1144632 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
   [junit4]   2> 1144633 DEBUG (ScheduledTrigger-489-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
   [junit4]   2> 1144633 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_lost_trigger
   [junit4]   2> 1144634 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED         1
   [junit4]   2> 1144634 INFO  (TEST-TestTriggerIntegration.testNodeLostTrigger-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED       1
   [junit4]   2> 1144644 DEBUG (ScheduledTrigger-489-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 1144661 DEBUG (ScheduledTrigger-489-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
   [junit4]   2> 1144661 INFO  (TEST-TestTriggerIntegration.testEventQueue-seed#[6CA08B043810C623]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testEventQueue
   [junit4]   2> 1144661 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 1144661 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 1144662 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 1144663 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1144663 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 1144663 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10000_solr
   [junit4]   2> 1144663 INFO  (TEST-TestTriggerIntegration.testEventQueue-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1144663 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Interrupted
   [junit4]   2> java.lang.InterruptedException
   [junit4]   2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
   [junit4]   2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
   [junit4]   2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:166)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1144664 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1144665 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 1144665 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 1144665 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
   [junit4]   2> 1144665 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion -1
   [junit4]   2> 1144665 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 1144666 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 1144666 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
   [junit4]   2> 1144666 DEBUG (ScheduledTrigger-492-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1144689 DEBUG (ScheduledTrigger-492-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1144725 DEBUG (ScheduledTrigger-492-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1144746 DEBUG (ScheduledTrigger-492-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1144767 DEBUG (ScheduledTrigger-492-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1144781 DEBUG (simCloudManagerPool-491-thread-1) [    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 1144781 DEBUG (simCloudManagerPool-491-thread-1) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
   [junit4]   2> 1144782 DEBUG (simCloudManagerPool-491-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
   [junit4]   2> 1144785 INFO  (simCloudManagerPool-491-thread-1) [    ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
   [junit4]   2> 1144789 DEBUG (simCloudManagerPool-491-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
   [junit4]   2> 1144797 DEBUG (ScheduledTrigger-492-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1144821 DEBUG (simCloudManagerPool-491-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
   [junit4]   2> 1144825 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
   [junit4]   2> 1144825 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 1144826 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
   [junit4]   2> 1144826 DEBUG (ScheduledTrigger-492-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1144826 DEBUG (ScheduledTrigger-492-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 1144826 DEBUG (ScheduledTrigger-492-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1144827 DEBUG (ScheduledTrigger-492-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10003_solr at time 15660613812531222
   [junit4]   2> 1144853 DEBUG (ScheduledTrigger-492-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 1144853 DEBUG (ScheduledTrigger-492-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 1144853 DEBUG (ScheduledTrigger-492-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 1144853 DEBUG (ScheduledTrigger-492-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:10003_solr] added at times [15660613812531222], now=15660615122247072
   [junit4]   2> 1144853 DEBUG (ScheduledTrigger-492-thread-1) [    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"37a33ecb42b016T7liad83y2wbgbnje8naj1focw",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":15660613812531222,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[15660613812531222],
   [junit4]   2>     "nodeNames":["127.0.0.1:10003_solr"]}}
   [junit4]   2> 1144861 DEBUG (AutoscalingActionExecutor-493-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"37a33ecb42b016T7liad83y2wbgbnje8naj1focw",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":15660613812531222,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[15660613812531222],
   [junit4]   2>     "_enqueue_time_":15660615141126722,
   [junit4]   2>     "nodeNames":["127.0.0.1:10003_solr"]}}
   [junit4]   2> 1144862 INFO  (AutoscalingActionExecutor-493-thread-1) [    ] o.a.s.c.a.s.TestTriggerIntegration -- event: {
   [junit4]   2>   "id":"37a33ecb42b016T7liad83y2wbgbnje8naj1focw",
   [junit4]   2>   "source":"node_added_trigger1",
   [junit4]   2>   "eventTime":15660613812531222,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[15660613812531222],
   [junit4]   2>     "_enqueue_time_":15660615141126722,
   [junit4]   2>     "nodeNames":["127.0.0.1:10003_solr"]}}
   [junit4]   2> 1144862 INFO  (TEST-TestTriggerIntegration.testEventQueue-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
   [junit4]   2> 1144862 WARN  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1145014 WARN  (AutoscalingActionExecutor-493-thread-1) [    ] o.a.s.c.a.SystemLogListener Exception sending event to collection .system
   [junit4]   2> java.io.IOException: java.lang.InterruptedException
   [junit4]   2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:443)
   [junit4]   2> at org.apache.solr.cloud.autoscaling.SystemLogListener.onEvent(SystemLogListener.java:122)
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:712)
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:682)
   [junit4]   2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:308)
   [junit4]   2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: java.lang.InterruptedException
   [junit4]   2> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404)
   [junit4]   2> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
   [junit4]   2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:441)
   [junit4]   2> ... 10 more
   [junit4]   2> 1145014 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[6CA08B043810C623]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 1145015 DEBUG (AutoscalingActionExecutor-493-thread-1) [    ] o.a.s.c.a.ScheduledTriggers -- processing took 7659 ms for event id=37a33ecb42b016T7liad83y2wbgbnje8naj1focw
   [junit4]   2> 1145015 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
   [junit4]   2> 1145015 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10003_solr]
   [junit4]   2> 1145015 INFO  (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
   [junit4]   2> 1145015 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10003_solr]
   [junit4]   2> 1145015 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
   [junit4]   2> 1145015 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
   [junit4]   2> 1145015 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
   [junit4]   2> 1145016 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10002_solr
   [junit4]   2> 1145025 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
   [junit4]   2> 1145025 DEBUG (ScheduledTrigger-495-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1145025 DEBUG (ScheduledTrigger-495-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10002_solr
   [junit4]   2> 1145025 DEBUG (Simulated OverseerAutoScalingTriggerThread) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
   [junit4]   2> 1145026 DEBUG (ScheduledTrigger-495-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 1145026 DEBUG (ScheduledTrigger-495-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 1145047 DEBUG (ScheduledTrigger-495-thread-1) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1145057 DEBUG (ScheduledTrigger-495-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 1145057 DEBUG (ScheduledTrigger-495-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 1145077 DEBUG (ScheduledTrigger-495-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1145077 DEBUG (ScheduledTrigger-495-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 1145077 DEBUG (ScheduledTrigger-495-thread-3) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 1145097 DEBUG (ScheduledTrigger-495-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 1145097 DEBUG (ScheduledTrigger-495-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 1145097 DEBUG (ScheduledTrigger-495-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1145142 DEBUG (ScheduledTrigger-495-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 1145142 DEBUG (ScheduledTrigger-495-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 1145142 DEBUG (ScheduledTrigger-495-thread-2) [    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 1145162 DEBUG (ScheduledTrigger-495-thread-1) [    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
   [junit4]   2> 1145162 DEBUG (ScheduledTrigger-495-thread-1) [    ] o.a.s.c

[...truncated too long message...]

ispatchFilter.logWelcomeBanner / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 203997 T77 oass.SolrDispatchFilter.logWelcomeBanner \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 203998 T77 oass.SolrDispatchFilter.logWelcomeBanner |___/\___/_|_|    Start time: 2018-01-08T15:06:52.264Z
   [junit4]   2> 203999 T77 oasc.SolrXmlConfig.fromFile Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/contrib/solr-dataimporthandler/test/J0/temp/solr.handler.dataimport.TestContentStreamDataSource_C3DC9995477B3A24-001/inst-002/solr.xml
   [junit4]   2> 204007 T77 oasc.SolrXmlConfig.getMetricReporterPluginInfos MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@3ecfdf79, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 204474 T77 oasmr.SolrJmxReporter.doInit JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3ecfdf79
   [junit4]   2> 204696 T77 oasmr.SolrJmxReporter.doInit JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3ecfdf79
   [junit4]   2> 204697 T77 oasmr.SolrJmxReporter.doInit JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3ecfdf79
   [junit4]   2> 204775 T77 oasc.CorePropertiesLocator.discover Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/contrib/solr-dataimporthandler/test/J0/temp/solr.handler.dataimport.TestContentStreamDataSource_C3DC9995477B3A24-001/inst-002
   [junit4]   2> 204775 T77 oasc.CorePropertiesLocator.discover Cores are: [collection1]
   [junit4]   2> 204847 T111 oasc.TransientSolrCoreCacheDefault.doInit Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 205108 T111 x:x:collection1 oasc.SolrConfig.<init> Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 205619 T111 x:x:collection1 oass.IndexSchema.readSchema [collection1] Schema name=dih_test
   [junit4]   2> 205735 T111 x:x:collection1 oass.IndexSchema.readSchema Loaded schema dih_test/4.0 with uniqueid field id
   [junit4]   2> 205735 T111 x:x:collection1 oasc.CoreContainer.createFromDescriptor Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/contrib/solr-dataimporthandler/test/J0/temp/solr.handler.dataimport.TestContentStreamDataSource_C3DC9995477B3A24-001/inst-002/collection1, trusted=true
   [junit4]   2> 205736 T111 x:x:collection1 oasmr.SolrJmxReporter.doInit JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3ecfdf79
   [junit4]   2> 205736 T111 x:x:collection1 oasc.SolrCore.initRecoveryStrategyBuilder solr.RecoveryStrategy.Builder
   [junit4]   2> 205737 T111 x:x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/contrib/solr-dataimporthandler/test/J0/temp/solr.handler.dataimport.TestContentStreamDataSource_C3DC9995477B3A24-001/inst-002/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/contrib/solr-dataimporthandler/test/J0/temp/solr.handler.dataimport.TestContentStreamDataSource_C3DC9995477B3A24-001/inst-002/collection1/data/]
   [junit4]   2> 206583 T111 x:x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 206583 T111 x:x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 206639 T111 x:x:collection1 oass.SolrIndexSearcher.<init> Opening [Searcher@3fdea6f7[collection1] main]
   [junit4]   2> 206640 T111 x:x:collection1 oasr.ManagedResourceStorage$FileStorageIO.configure File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/contrib/solr-dataimporthandler/test/J0/temp/solr.handler.dataimport.TestContentStreamDataSource_C3DC9995477B3A24-001/inst-002/collection1/conf
   [junit4]   2> 206642 T111 x:x:collection1 oash.ReplicationHandler.inform Commits will be reserved for 10000ms.
   [junit4]   2> 206644 T115 x:x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3fdea6f7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> ASYNC  NEW_CORE C6 name=collection1 org.apache.solr.core.SolrCore@4d508feb
   [junit4]   2> 207798 T105 x:x:collection1 C6 oashd.DataImporter.maybeReloadConfiguration Loading DIH Configuration: data-config.xml
   [junit4]   2> 207811 T105 x:x:collection1 C6 oashd.DataImporter.loadDataConfig Data Configuration loaded successfully
   [junit4]   2> 207811 T105 x:x:collection1 C6 oashd.DataImporter.doFullImport Starting Full Import
   [junit4]   2> 207850 T105 x:x:collection1 C6 oashd.SimplePropertiesWriter.readIndexerProperties WARN Unable to read: dataimport.properties
   [junit4]   2> 207890 T105 x:x:collection1 C6 oashd.DocBuilder.finish Import completed successfully
   [junit4]   2> 207890 T105 x:x:collection1 C6 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 207891 T105 x:x:collection1 C6 oasu.SolrIndexWriter.setCommitData Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@7d386cd4 commitCommandVersion:0
   [junit4]   2> 208470 T105 x:x:collection1 C6 oass.SolrIndexSearcher.<init> Opening [Searcher@884bf7b[collection1] main]
   [junit4]   2> 208477 T115 x:x:collection1 C6 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@884bf7b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C2)))}
   [junit4]   2> 208477 T105 x:x:collection1 C6 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 208498 T105 x:x:collection1 C6 oashd.SimplePropertiesWriter.readIndexerProperties WARN Unable to read: dataimport.properties
   [junit4]   2> 208498 T105 x:x:collection1 C6 oashd.SimplePropertiesWriter.persist Wrote last indexed time to dataimport.properties
   [junit4]   2> 208498 T105 x:x:collection1 C6 oashd.DocBuilder.execute Time taken = 0:0:0.647
   [junit4]   2> 208499 T105 x:x:collection1 C6 oasup.LogUpdateProcessorFactory$LogUpdateProcessor.finish [collection1]  webapp=/solr path=/dataimport params={clean=false&wt=javabin&version=2&command=full-import}{add=[1, 2],commit=} 0 742
   [junit4]   2> 208542 T105 x:x:collection1 C6 oasc.SolrCore.execute [collection1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=2 status=0 QTime=20
   [junit4]   2> 208667 T77 oejs.AbstractConnector.doStop Stopped ServerConnector@3e2f2ab{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 208667 T77 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=728309046
   [junit4]   2> 208667 T77 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 208668 T77 oasmr.SolrJmxReporter.close Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@44586dd7: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@616b106c
   [junit4]   2> 208796 T77 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 208796 T77 oasmr.SolrJmxReporter.close Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6d45007e: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@1c0bbc5b
   [junit4]   2> 209019 T77 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 209019 T77 oasmr.SolrJmxReporter.close Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@539f1e75: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@f0fcfa7
   [junit4]   2> 209030 T119 x:x:collection1 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4d508feb
   [junit4]   2> 209030 T119 x:x:collection1 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.core.collection1, tag=1297125355
   [junit4]   2> 209030 T119 x:x:collection1 oasmr.SolrJmxReporter.close Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6c3da9c9: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@390d31f5
   [junit4]   2> 209303 T77 oejsh.ContextHandler.doStop Stopped o.e.j.s.ServletContextHandler@1c1aea51{/solr,null,UNAVAILABLE}
   [junit4]   2> 209347 T77 oas.SolrTestCaseJ4.tearDown ###Ending testSimple
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/contrib/solr-dataimporthandler/test/J0/temp/solr.handler.dataimport.TestContentStreamDataSource_C3DC9995477B3A24-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {id=Lucene50(blocksize=128), desc=FSTOrd50}, docValues:{}, maxPointsInLeafNode=544, maxMBSortInHeap=5.470968634285386, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@21891f6c), locale=sk-SK, timezone=Africa/Bissau
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=240164904,total=338690048
   [junit4]   2> NOTE: All tests run in this JVM: [TestHierarchicalDocBuilder, TestZKPropertiesWriter, TestSqlEntityProcessorDelta, TestContentStreamDataSource]
   [junit4] Completed [25/38 (1!)] on J0 in 29.21s, 2 tests, 1 failure <<< FAILURES!

[...truncated 42440 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2252 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2252/

1 tests failed.
FAILED:  org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest.testParse

Error Message:
Error from server at https://127.0.0.1:36154/solr: Collection : myalias_2017-10-24 is part of alias myalias remove or modify the alias before removing this collection.

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:36154/solr: Collection : myalias_2017-10-24 is part of alias myalias remove or modify the alias before removing this collection.
        at __randomizedtesting.SeedInfo.seed([DF868EAC660D298E:B003C5F4DA7B1A61]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest.doBefore(TimeRoutedAliasUpdateProcessorTest.java:84)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:968)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12411 lines...]
   [junit4] Suite: org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/init-core-data-001
   [junit4]   2> 1504130 WARN  (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[DF868EAC660D298E]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 1504130 INFO  (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[DF868EAC660D298E]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1504132 INFO  (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[DF868EAC660D298E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1504132 INFO  (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[DF868EAC660D298E]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1504132 INFO  (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[DF868EAC660D298E]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001
   [junit4]   2> 1504132 INFO  (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[DF868EAC660D298E]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1504132 INFO  (Thread-4105) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1504132 INFO  (Thread-4105) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1504147 ERROR (Thread-4105) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1504232 INFO  (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[DF868EAC660D298E]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:39218
   [junit4]   2> 1504236 INFO  (zkConnectionManagerCallback-1295-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1504249 INFO  (jetty-launcher-1292-thread-1) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1504256 INFO  (jetty-launcher-1292-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@26aa2fdc{/solr,null,AVAILABLE}
   [junit4]   2> 1504263 INFO  (jetty-launcher-1292-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@62f7c9ee{SSL,[ssl, http/1.1]}{127.0.0.1:37758}
   [junit4]   2> 1504263 INFO  (jetty-launcher-1292-thread-1) [    ] o.e.j.s.Server Started @1514065ms
   [junit4]   2> 1504263 INFO  (jetty-launcher-1292-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37758}
   [junit4]   2> 1504263 ERROR (jetty-launcher-1292-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1504264 INFO  (jetty-launcher-1292-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1504264 INFO  (jetty-launcher-1292-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1504264 INFO  (jetty-launcher-1292-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1504264 INFO  (jetty-launcher-1292-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-10T05:45:31.001Z
   [junit4]   2> 1504266 INFO  (jetty-launcher-1292-thread-2) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 1504268 INFO  (jetty-launcher-1292-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@15ea0a01{/solr,null,AVAILABLE}
   [junit4]   2> 1504268 INFO  (zkConnectionManagerCallback-1297-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1504269 INFO  (jetty-launcher-1292-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1504269 INFO  (jetty-launcher-1292-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@21ca7b83{SSL,[ssl, http/1.1]}{127.0.0.1:36154}
   [junit4]   2> 1504269 INFO  (jetty-launcher-1292-thread-2) [    ] o.e.j.s.Server Started @1514071ms
   [junit4]   2> 1504269 INFO  (jetty-launcher-1292-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36154}
   [junit4]   2> 1504269 ERROR (jetty-launcher-1292-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1504269 INFO  (jetty-launcher-1292-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1504269 INFO  (jetty-launcher-1292-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1504269 INFO  (jetty-launcher-1292-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1504269 INFO  (jetty-launcher-1292-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-10T05:45:31.006Z
   [junit4]   2> 1504273 INFO  (zkConnectionManagerCallback-1299-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1504274 INFO  (jetty-launcher-1292-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1504281 INFO  (jetty-launcher-1292-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39218/solr
   [junit4]   2> 1504283 INFO  (jetty-launcher-1292-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39218/solr
   [junit4]   2> 1504302 INFO  (zkConnectionManagerCallback-1307-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1504304 INFO  (zkConnectionManagerCallback-1303-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1504308 INFO  (zkConnectionManagerCallback-1309-thread-1-processing-n:127.0.0.1:36154_solr) [n:127.0.0.1:36154_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1504310 INFO  (zkConnectionManagerCallback-1311-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1504447 INFO  (jetty-launcher-1292-thread-1) [n:127.0.0.1:37758_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1504448 INFO  (jetty-launcher-1292-thread-1) [n:127.0.0.1:37758_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37758_solr
   [junit4]   2> 1504449 INFO  (jetty-launcher-1292-thread-1) [n:127.0.0.1:37758_solr    ] o.a.s.c.Overseer Overseer (id=99323945344696326-127.0.0.1:37758_solr-n_0000000000) starting
   [junit4]   2> 1504492 INFO  (jetty-launcher-1292-thread-1) [n:127.0.0.1:37758_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37758_solr
   [junit4]   2> 1504501 INFO  (jetty-launcher-1292-thread-2) [n:127.0.0.1:36154_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1504502 INFO  (zkCallback-1310-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1504503 INFO  (jetty-launcher-1292-thread-2) [n:127.0.0.1:36154_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1504504 INFO  (jetty-launcher-1292-thread-2) [n:127.0.0.1:36154_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36154_solr
   [junit4]   2> 1504556 INFO  (zkCallback-1308-thread-1-processing-n:127.0.0.1:36154_solr) [n:127.0.0.1:36154_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1504570 INFO  (zkCallback-1310-thread-2-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1504599 INFO  (jetty-launcher-1292-thread-2) [n:127.0.0.1:36154_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36154.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6390bd4e
   [junit4]   2> 1504617 INFO  (jetty-launcher-1292-thread-2) [n:127.0.0.1:36154_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36154.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6390bd4e
   [junit4]   2> 1504617 INFO  (jetty-launcher-1292-thread-2) [n:127.0.0.1:36154_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36154.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6390bd4e
   [junit4]   2> 1504618 INFO  (jetty-launcher-1292-thread-2) [n:127.0.0.1:36154_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/.
   [junit4]   2> 1504649 INFO  (jetty-launcher-1292-thread-1) [n:127.0.0.1:37758_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37758.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6390bd4e
   [junit4]   2> 1504655 INFO  (jetty-launcher-1292-thread-1) [n:127.0.0.1:37758_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37758.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6390bd4e
   [junit4]   2> 1504656 INFO  (jetty-launcher-1292-thread-1) [n:127.0.0.1:37758_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37758.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6390bd4e
   [junit4]   2> 1504657 INFO  (jetty-launcher-1292-thread-1) [n:127.0.0.1:37758_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node1/.
   [junit4]   2> 1504693 INFO  (zkConnectionManagerCallback-1317-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1504713 INFO  (TEST-TimeRoutedAliasUpdateProcessorTest.test-seed#[DF868EAC660D298E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 1504730 INFO  (zkConnectionManagerCallback-1322-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1504731 INFO  (TEST-TimeRoutedAliasUpdateProcessorTest.test-seed#[DF868EAC660D298E]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1504732 INFO  (TEST-TimeRoutedAliasUpdateProcessorTest.test-seed#[DF868EAC660D298E]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39218/solr ready
   [junit4]   2> 1504746 INFO  (qtp849971433-8762) [n:127.0.0.1:36154_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1504746 INFO  (qtp849971433-8762) [n:127.0.0.1:36154_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1504747 INFO  (qtp849971433-8758) [n:127.0.0.1:36154_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=1&name=timeConfig&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1504758 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.CreateCollectionCmd Create collection timeConfig
   [junit4]   2> 1504758 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.CreateCollectionCmd Only _default config set found, using it.
   [junit4]   2> 1504759 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/protwords.txt to /configs/timeConfig/protwords.txt
   [junit4]   2> 1504761 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/managed-schema to /configs/timeConfig/managed-schema
   [junit4]   2> 1504762 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/solrconfig.xml to /configs/timeConfig/solrconfig.xml
   [junit4]   2> 1504763 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/synonyms.txt to /configs/timeConfig/synonyms.txt
   [junit4]   2> 1504764 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/stopwords.txt to /configs/timeConfig/stopwords.txt
   [junit4]   2> 1504765 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/params.json to /configs/timeConfig/params.json
   [junit4]   2> 1504766 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_it.txt to /configs/timeConfig/lang/contractions_it.txt
   [junit4]   2> 1504768 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_no.txt to /configs/timeConfig/lang/stopwords_no.txt
   [junit4]   2> 1504769 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hy.txt to /configs/timeConfig/lang/stopwords_hy.txt
   [junit4]   2> 1504770 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stemdict_nl.txt to /configs/timeConfig/lang/stemdict_nl.txt
   [junit4]   2> 1504771 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ca.txt to /configs/timeConfig/lang/contractions_ca.txt
   [junit4]   2> 1504772 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_id.txt to /configs/timeConfig/lang/stopwords_id.txt
   [junit4]   2> 1504773 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ga.txt to /configs/timeConfig/lang/stopwords_ga.txt
   [junit4]   2> 1504774 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_da.txt to /configs/timeConfig/lang/stopwords_da.txt
   [junit4]   2> 1504775 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hi.txt to /configs/timeConfig/lang/stopwords_hi.txt
   [junit4]   2> 1504778 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ja.txt to /configs/timeConfig/lang/stopwords_ja.txt
   [junit4]   2> 1504779 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_pt.txt to /configs/timeConfig/lang/stopwords_pt.txt
   [junit4]   2> 1504781 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hu.txt to /configs/timeConfig/lang/stopwords_hu.txt
   [junit4]   2> 1504782 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_el.txt to /configs/timeConfig/lang/stopwords_el.txt
   [junit4]   2> 1504783 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ru.txt to /configs/timeConfig/lang/stopwords_ru.txt
   [junit4]   2> 1504784 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_tr.txt to /configs/timeConfig/lang/stopwords_tr.txt
   [junit4]   2> 1504785 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ar.txt to /configs/timeConfig/lang/stopwords_ar.txt
   [junit4]   2> 1504786 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/userdict_ja.txt to /configs/timeConfig/lang/userdict_ja.txt
   [junit4]   2> 1504788 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_it.txt to /configs/timeConfig/lang/stopwords_it.txt
   [junit4]   2> 1504789 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_eu.txt to /configs/timeConfig/lang/stopwords_eu.txt
   [junit4]   2> 1504790 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_cz.txt to /configs/timeConfig/lang/stopwords_cz.txt
   [junit4]   2> 1504791 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stoptags_ja.txt to /configs/timeConfig/lang/stoptags_ja.txt
   [junit4]   2> 1504793 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_fr.txt to /configs/timeConfig/lang/contractions_fr.txt
   [junit4]   2> 1504794 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fa.txt to /configs/timeConfig/lang/stopwords_fa.txt
   [junit4]   2> 1504796 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_de.txt to /configs/timeConfig/lang/stopwords_de.txt
   [junit4]   2> 1504797 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ga.txt to /configs/timeConfig/lang/contractions_ga.txt
   [junit4]   2> 1504807 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ca.txt to /configs/timeConfig/lang/stopwords_ca.txt
   [junit4]   2> 1504808 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_nl.txt to /configs/timeConfig/lang/stopwords_nl.txt
   [junit4]   2> 1504809 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_bg.txt to /configs/timeConfig/lang/stopwords_bg.txt
   [junit4]   2> 1504809 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_es.txt to /configs/timeConfig/lang/stopwords_es.txt
   [junit4]   2> 1504810 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_en.txt to /configs/timeConfig/lang/stopwords_en.txt
   [junit4]   2> 1504811 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_lv.txt to /configs/timeConfig/lang/stopwords_lv.txt
   [junit4]   2> 1504812 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_th.txt to /configs/timeConfig/lang/stopwords_th.txt
   [junit4]   2> 1504813 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ro.txt to /configs/timeConfig/lang/stopwords_ro.txt
   [junit4]   2> 1504813 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fr.txt to /configs/timeConfig/lang/stopwords_fr.txt
   [junit4]   2> 1504814 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_sv.txt to /configs/timeConfig/lang/stopwords_sv.txt
   [junit4]   2> 1504816 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fi.txt to /configs/timeConfig/lang/stopwords_fi.txt
   [junit4]   2> 1504817 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/hyphenations_ga.txt to /configs/timeConfig/lang/hyphenations_ga.txt
   [junit4]   2> 1504818 INFO  (OverseerThreadFactory-2619-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_gl.txt to /configs/timeConfig/lang/stopwords_gl.txt
   [junit4]   2> 1504926 INFO  (OverseerStateUpdate-99323945344696326-127.0.0.1:37758_solr-n_0000000000) [n:127.0.0.1:37758_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"timeConfig",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"timeConfig_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:36154/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1505136 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=timeConfig&newCollection=true&name=timeConfig_shard1_replica_n1&action=CREATE&numShards=1&collection=timeConfig&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1505136 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1505242 INFO  (zkCallback-1308-thread-1-processing-n:127.0.0.1:36154_solr) [n:127.0.0.1:36154_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/timeConfig/state.json] for collection [timeConfig] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1505243 INFO  (zkCallback-1308-thread-2-processing-n:127.0.0.1:36154_solr) [n:127.0.0.1:36154_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/timeConfig/state.json] for collection [timeConfig] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1506157 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
   [junit4]   2> 1506159 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 1506159 WARN  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../contrib/extraction/lib
   [junit4]   2> 1506160 WARN  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../dist
   [junit4]   2> 1506160 WARN  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../contrib/clustering/lib
   [junit4]   2> 1506160 WARN  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../dist
   [junit4]   2> 1506160 WARN  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../contrib/langid/lib
   [junit4]   2> 1506160 WARN  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../dist
   [junit4]   2> 1506160 WARN  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../contrib/velocity/lib
   [junit4]   2> 1506160 WARN  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../dist
   [junit4]   2> 1506167 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1506188 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.s.IndexSchema [timeConfig_shard1_replica_n1] Schema name=default-config
   [junit4]   2> 1506423 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
   [junit4]   2> 1506494 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'timeConfig_shard1_replica_n1' using configuration from collection timeConfig, trusted=true
   [junit4]   2> 1506495 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36154.solr.core.timeConfig.shard1.replica_n1' (registry 'solr.core.timeConfig.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6390bd4e
   [junit4]   2> 1506495 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1506495 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SolrCore [[timeConfig_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/./timeConfig_shard1_replica_n1/data/]
   [junit4]   2> 1506514 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.r.XSLTResponseWriter xsltCacheLifetimeSeconds=5
   [junit4]   2> 1506573 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1506573 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1506574 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
   [junit4]   2> 1506574 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1506576 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@dedef20[timeConfig_shard1_replica_n1] main]
   [junit4]   2> 1506577 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/timeConfig
   [junit4]   2> 1506577 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/timeConfig
   [junit4]   2> 1506577 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/timeConfig/managed-schema
   [junit4]   2> 1506578 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 1506578 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 1506578 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.s.DirectSolrSpellChecker init: {name=default,field=_text_,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=2,minPrefix=1,maxInspections=5,minQueryLength=4,maxQueryFrequency=0.01}
   [junit4]   2> 1506586 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1506586 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1589183128087298048
   [junit4]   2> 1506586 INFO  (searcherExecutor-2624-thread-1-processing-n:127.0.0.1:36154_solr x:timeConfig_shard1_replica_n1 s:shard1 c:timeConfig r:core_node2) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@dedef20[timeConfig_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1506586 INFO  (searcherExecutor-2624-thread-1-processing-n:127.0.0.1:36154_solr x:timeConfig_shard1_replica_n1 s:shard1 c:timeConfig r:core_node2) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.QuerySenderListener QuerySenderListener done.
   [junit4]   2> 1506586 INFO  (searcherExecutor-2624-thread-1-processing-n:127.0.0.1:36154_solr x:timeConfig_shard1_replica_n1 s:shard1 c:timeConfig r:core_node2) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 1506587 INFO  (searcherExecutor-2624-thread-1-processing-n:127.0.0.1:36154_solr x:timeConfig_shard1_replica_n1 s:shard1 c:timeConfig r:core_node2) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SolrCore [timeConfig_shard1_replica_n1] Registered new searcher Searcher@dedef20[timeConfig_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1506593 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1506593 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1506593 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:36154/solr/timeConfig_shard1_replica_n1/
   [junit4]   2> 1506593 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1506593 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:36154/solr/timeConfig_shard1_replica_n1/ has no replicas
   [junit4]   2> 1506593 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1506610 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:36154/solr/timeConfig_shard1_replica_n1/ shard1
   [junit4]   2> 1506712 INFO  (zkCallback-1308-thread-2-processing-n:127.0.0.1:36154_solr) [n:127.0.0.1:36154_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/timeConfig/state.json] for collection [timeConfig] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1506718 INFO  (zkCallback-1308-thread-1-processing-n:127.0.0.1:36154_solr) [n:127.0.0.1:36154_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/timeConfig/state.json] for collection [timeConfig] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1506761 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1506763 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=timeConfig&newCollection=true&name=timeConfig_shard1_replica_n1&action=CREATE&numShards=1&collection=timeConfig&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1627
   [junit4]   2> 1506765 INFO  (qtp849971433-8758) [n:127.0.0.1:36154_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1506864 INFO  (zkCallback-1308-thread-1-processing-n:127.0.0.1:36154_solr) [n:127.0.0.1:36154_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/timeConfig/state.json] for collection [timeConfig] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1506864 INFO  (zkCallback-1308-thread-2-processing-n:127.0.0.1:36154_solr) [n:127.0.0.1:36154_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/timeConfig/state.json] for collection [timeConfig] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1507766 INFO  (qtp849971433-8758) [n:127.0.0.1:36154_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&replicationFactor=1&name=timeConfig&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=3018
   [junit4]   2> 1507770 INFO  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler Executed config commands successfully and persisted to ZK [{"set-user-property":{"timePartitionAliasName":"myalias"}}, {"set-user-property":{"update.autoCreateFields":false}}, {"add-updateprocessor":{
   [junit4]   2>     "name":"tolerant",
   [junit4]   2>     "class":"solr.TolerantUpdateProcessorFactory"}}, {"add-updateprocessor":{
   [junit4]   2>     "name":"inc",
   [junit4]   2>     "class":"org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest$IncrementURPFactory",
   [junit4]   2>     "fieldName":"integer_i"}}]
   [junit4]   2> 1507770 INFO  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler Waiting up to 30 secs for 1 replicas to set the property overlay to be of version 0 for collection timeConfig
   [junit4]   2> 1507770 INFO  (Thread-4124) [n:127.0.0.1:36154_solr    ] o.a.s.c.SolrCore config update listener called for core timeConfig_shard1_replica_n1
   [junit4]   2> 1507771 INFO  (Thread-4124) [n:127.0.0.1:36154_solr    ] o.a.s.c.SolrCore core reload timeConfig_shard1_replica_n1
   [junit4]   2> 1507771 INFO  (solrHandlerExecutor-2629-thread-1-processing-n:127.0.0.1:36154_solr x:timeConfig_shard1_replica_n1 s:shard1 c:timeConfig r:core_node2) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler Time elapsed : 0 secs, maxWait 30
   [junit4]   2> 1507779 INFO  (Thread-4124) [n:127.0.0.1:36154_solr    ] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
   [junit4]   2> 1507779 INFO  (Thread-4124) [n:127.0.0.1:36154_solr    ] o.a.s.c.RequestParams request params refreshed to version 0
   [junit4]   2> 1507780 WARN  (Thread-4124) [n:127.0.0.1:36154_solr    ] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../contrib/extraction/lib
   [junit4]   2> 1507780 WARN  (Thread-4124) [n:127.0.0.1:36154_solr    ] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../dist
   [junit4]   2> 1507780 WARN  (Thread-4124) [n:127.0.0.1:36154_solr    ] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../contrib/clustering/lib
   [junit4]   2> 1507780 WARN  (Thread-4124) [n:127.0.0.1:36154_solr    ] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../dist
   [junit4]   2> 1507780 WARN  (Thread-4124) [n:127.0.0.1:36154_solr    ] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../contrib/langid/lib
   [junit4]   2> 1507780 WARN  (Thread-4124) [n:127.0.0.1:36154_solr    ] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../dist
   [junit4]   2> 1507781 WARN  (Thread-4124) [n:127.0.0.1:36154_solr    ] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../contrib/velocity/lib
   [junit4]   2> 1507781 WARN  (Thread-4124) [n:127.0.0.1:36154_solr    ] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1/../../../../dist
   [junit4]   2> 1507791 INFO  (Thread-4124) [n:127.0.0.1:36154_solr    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1507811 INFO  (Thread-4124) [n:127.0.0.1:36154_solr    ] o.a.s.s.IndexSchema [timeConfig_shard1_replica_n1] Schema name=default-config
   [junit4]   2> 1507874 INFO  (qtp849971433-8756) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler expecting overlay version 0 but my version is -1
   [junit4]   2> 1507875 INFO  (qtp849971433-8756) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.S.Request [timeConfig_shard1_replica_n1]  webapp=/solr path=/config/znodeVersion params={overlay=0&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1507875 INFO  (SolrConfigHandler-refreshconf) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler Another reload is in progress . Not doing anything
   [junit4]   2> 1507875 INFO  (solrHandlerExecutor-2629-thread-1-processing-n:127.0.0.1:36154_solr x:timeConfig_shard1_replica_n1 s:shard1 c:timeConfig r:core_node2) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler Could not get expectedVersion 0 from https://127.0.0.1:36154/solr/timeConfig_shard1_replica_n1/ for prop overlay   after 1 attempts
   [junit4]   2> 1507875 INFO  (solrHandlerExecutor-2629-thread-1-processing-n:127.0.0.1:36154_solr x:timeConfig_shard1_replica_n1 s:shard1 c:timeConfig r:core_node2) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler Time elapsed : 0 secs, maxWait 30
   [junit4]   2> 1507960 INFO  (Thread-4124) [n:127.0.0.1:36154_solr    ] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
   [junit4]   2> 1507976 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler expecting overlay version 0 but my version is -1
   [junit4]   2> 1507976 INFO  (qtp849971433-8761) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.S.Request [timeConfig_shard1_replica_n1]  webapp=/solr path=/config/znodeVersion params={overlay=0&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1507976 INFO  (SolrConfigHandler-refreshconf) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler Another reload is in progress . Not doing anything
   [junit4]   2> 1507976 INFO  (solrHandlerExecutor-2629-thread-1-processing-n:127.0.0.1:36154_solr x:timeConfig_shard1_replica_n1 s:shard1 c:timeConfig r:core_node2) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler Could not get expectedVersion 0 from https://127.0.0.1:36154/solr/timeConfig_shard1_replica_n1/ for prop overlay   after 2 attempts
   [junit4]   2> 1507976 INFO  (solrHandlerExecutor-2629-thread-1-processing-n:127.0.0.1:36154_solr x:timeConfig_shard1_replica_n1 s:shard1 c:timeConfig r:core_node2) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler Time elapsed : 0 secs, maxWait 30
   [junit4]   2> 1507998 INFO  (Thread-4124) [n:127.0.0.1:36154_solr    ] o.a.s.c.CoreContainer Reloading SolrCore 'timeConfig_shard1_replica_n1' using configuration from collection timeConfig
   [junit4]   2> 1508016 INFO  (Thread-4124) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36154.solr.core.timeConfig.shard1.replica_n1' (registry 'solr.core.timeConfig.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6390bd4e
   [junit4]   2> 1508016 INFO  (Thread-4124) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SolrCore [[timeConfig_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/timeConfig_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/./timeConfig_shard1_replica_n1/data/]
   [junit4]   2> 1508018 INFO  (Thread-4124) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.r.XSLTResponseWriter xsltCacheLifetimeSeconds=5
   [junit4]   2> 1508047 INFO  (Thread-4124) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
   [junit4]   2> 1508047 INFO  (Thread-4124) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1508049 INFO  (Thread-4124) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4f194aea[timeConfig_shard1_replica_n1] main]
   [junit4]   2> 1508050 INFO  (Thread-4124) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/timeConfig
   [junit4]   2> 1508050 INFO  (Thread-4124) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/timeConfig
   [junit4]   2> 1508050 INFO  (Thread-4124) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/timeConfig/managed-schema
   [junit4]   2> 1508051 INFO  (Thread-4124) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 1508051 INFO  (Thread-4124) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 1508051 INFO  (Thread-4124) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.s.DirectSolrSpellChecker init: {name=default,field=_text_,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=2,minPrefix=1,maxInspections=5,minQueryLength=4,maxQueryFrequency=0.01}
   [junit4]   2> 1508052 INFO  (Thread-4124) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1508052 INFO  (searcherExecutor-2630-thread-1-processing-n:127.0.0.1:36154_solr x:timeConfig_shard1_replica_n1 s:shard1 c:timeConfig r:core_node2) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@4f194aea[timeConfig_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1508062 INFO  (Thread-4124) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1589183129634996224
   [junit4]   2> 1508062 INFO  (searcherExecutor-2630-thread-1-processing-n:127.0.0.1:36154_solr x:timeConfig_shard1_replica_n1 s:shard1 c:timeConfig r:core_node2) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.QuerySenderListener QuerySenderListener done.
   [junit4]   2> 1508062 INFO  (searcherExecutor-2630-thread-1-processing-n:127.0.0.1:36154_solr x:timeConfig_shard1_replica_n1 s:shard1 c:timeConfig r:core_node2) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 1508065 INFO  (Thread-4124) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1508065 INFO  (searcherExecutor-2630-thread-1-processing-n:127.0.0.1:36154_solr x:timeConfig_shard1_replica_n1 s:shard1 c:timeConfig r:core_node2) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SolrCore [timeConfig_shard1_replica_n1] Registered new searcher Searcher@4f194aea[timeConfig_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1508066 INFO  (Thread-4124) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@646362d2[timeConfig_shard1_replica_n1] main]
   [junit4]   2> 1508066 INFO  (searcherExecutor-2630-thread-1-processing-n:127.0.0.1:36154_solr x:timeConfig_shard1_replica_n1 s:shard1 c:timeConfig r:core_node2) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@646362d2[timeConfig_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1508066 INFO  (searcherExecutor-2630-thread-1-processing-n:127.0.0.1:36154_solr x:timeConfig_shard1_replica_n1 s:shard1 c:timeConfig r:core_node2) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.QuerySenderListener QuerySenderListener done.
   [junit4]   2> 1508069 INFO  (searcherExecutor-2630-thread-1-processing-n:127.0.0.1:36154_solr x:timeConfig_shard1_replica_n1 s:shard1 c:timeConfig r:core_node2) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SolrCore [timeConfig_shard1_replica_n1] Registered new searcher Searcher@646362d2[timeConfig_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1508077 INFO  (qtp849971433-8758) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler I already have the expected version 0 of config
   [junit4]   2> 1508077 INFO  (qtp849971433-8758) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler isStale false , resourceloader org.apache.solr.cloud.ZkSolrResourceLoader
   [junit4]   2> 1508077 INFO  (qtp849971433-8758) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.S.Request [timeConfig_shard1_replica_n1]  webapp=/solr path=/config/znodeVersion params={overlay=0&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1508078 INFO  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler Took 307.0ms to set the property overlay to be of version 0 for collection timeConfig
   [junit4]   2> 1508078 INFO  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.SolrCore [timeConfig_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@11e6d630
   [junit4]   2> 1508078 INFO  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.timeConfig.shard1.replica_n1, tag=300340784
   [junit4]   2> 1508078 INFO  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@394c89ea: rootName = solr_36154, domain = solr.core.timeConfig.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.timeConfig.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@2ae286ef
   [junit4]   2> 1508085 INFO  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.timeConfig.shard1.leader, tag=300340784
   [junit4]   2> 1508111 INFO  (qtp849971433-8758) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.ZkController Persisted config data to node /configs/timeConfig/params.json
   [junit4]   2> 1508112 INFO  (qtp849971433-8758) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler Waiting up to 30 secs for 1 replicas to set the property params to be of version 1 for collection timeConfig
   [junit4]   2> 1508115 INFO  (Thread-4125) [n:127.0.0.1:36154_solr    ] o.a.s.c.SolrCore config update listener called for core timeConfig_shard1_replica_n1
   [junit4]   2> 1508116 INFO  (Thread-4125) [n:127.0.0.1:36154_solr    ] o.a.s.c.RequestParams conf resource params.json loaded . version : 1
   [junit4]   2> 1508116 INFO  (Thread-4125) [n:127.0.0.1:36154_solr    ] o.a.s.c.RequestParams request params refreshed to version 1
   [junit4]   2> 1508120 INFO  (solrHandlerExecutor-2634-thread-1-processing-n:127.0.0.1:36154_solr x:timeConfig_shard1_replica_n1 s:shard1 c:timeConfig r:core_node2) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler Time elapsed : 0 secs, maxWait 30
   [junit4]   2> 1508228 INFO  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler I already have the expected version 1 of params
   [junit4]   2> 1508229 INFO  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler isStale false , resourceloader org.apache.solr.cloud.ZkSolrResourceLoader
   [junit4]   2> 1508229 INFO  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.S.Request [timeConfig_shard1_replica_n1]  webapp=/solr path=/config/znodeVersion params={params=1&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1508229 INFO  (qtp849971433-8758) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.h.SolrConfigHandler Took 117.0ms to set the property params to be of version 1 for collection timeConfig
   [junit4]   2> 1508229 INFO  (qtp849971433-8758) [n:127.0.0.1:36154_solr c:timeConfig s:shard1 r:core_node2 x:timeConfig_shard1_replica_n1] o.a.s.c.S.Request [timeConfig_shard1_replica_n1]  webapp=/solr path=/config/params params={wt=javabin&version=2} status=0 QTime=118
   [junit4]   2> 1508230 INFO  (qtp849971433-8756) [n:127.0.0.1:36154_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=timeConfig&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1508232 INFO  (OverseerCollectionConfigSetProcessor-99323945344696326-127.0.0.1:37758_solr-n_0000000000) [n:127.0.0.1:37758_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1508232 INFO  (OverseerThreadFactory-2619-thread-2-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
   [junit4]   2> 1508233 INFO  (qtp849971433-8763) [n:127.0.0.1:36154_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.timeConfig.shard1.replica_n1, tag=null
   [junit4]   2> 1508233 INFO  (qtp849971433-8763) [n:127.0.0.1:36154_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@49255399: rootName = solr_36154, domain = solr.core.timeConfig.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.timeConfig.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@2ae286ef
   [junit4]   2> 1508243 INFO  (qtp849971433-8763) [n:127.0.0.1:36154_solr    ] o.a.s.c.SolrCore [timeConfig_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@353535f5
   [junit4]   2> 1508243 INFO  (qtp849971433-8763) [n:127.0.0.1:36154_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.timeConfig.shard1.replica_n1, tag=892679669
   [junit4]   2> 1508243 INFO  (qtp849971433-8763) [n:127.0.0.1:36154_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.timeConfig.shard1.leader, tag=892679669
   [junit4]   2> 1508247 INFO  (qtp849971433-8763) [n:127.0.0.1:36154_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=timeConfig_shard1_replica_n1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=13
   [junit4]   2> 1508951 INFO  (qtp849971433-8756) [n:127.0.0.1:36154_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=timeConfig&action=DELETE&wt=javabin&version=2} status=0 QTime=721
   [junit4]   2> 1508952 INFO  (qtp849971433-8762) [n:127.0.0.1:36154_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&property.timePartitionAliasName=myalias&collection.configName=timeConfig&maxShardsPerNode=2&name=myalias_2017-10-23&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1508954 INFO  (OverseerThreadFactory-2619-thread-3-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.CreateCollectionCmd Create collection myalias_2017-10-23
   [junit4]   2> 1508954 INFO  (OverseerCollectionConfigSetProcessor-99323945344696326-127.0.0.1:37758_solr-n_0000000000) [n:127.0.0.1:37758_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1509162 INFO  (OverseerStateUpdate-99323945344696326-127.0.0.1:37758_solr-n_0000000000) [n:127.0.0.1:37758_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"myalias_2017-10-23",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"myalias_2017-10-23_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:37758/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1509164 INFO  (OverseerStateUpdate-99323945344696326-127.0.0.1:37758_solr-n_0000000000) [n:127.0.0.1:37758_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"myalias_2017-10-23",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"myalias_2017-10-23_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:36154/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1509175 INFO  (OverseerStateUpdate-99323945344696326-127.0.0.1:37758_solr-n_0000000000) [n:127.0.0.1:37758_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"myalias_2017-10-23",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"myalias_2017-10-23_shard2_replica_n4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:37758/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1509177 INFO  (OverseerStateUpdate-99323945344696326-127.0.0.1:37758_solr-n_0000000000) [n:127.0.0.1:37758_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"myalias_2017-10-23",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"myalias_2017-10-23_shard2_replica_n6",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:36154/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"}
   [junit4]   2> 1509392 INFO  (qtp849971433-8759) [n:127.0.0.1:36154_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=timeConfig&newCollection=true&collection=myalias_2017-10-23&version=2&replicaType=NRT&coreNodeName=core_node5&property.timePartitionAliasName=myalias&name=myalias_2017-10-23_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 1509392 INFO  (qtp892038354-8749) [n:127.0.0.1:37758_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=timeConfig&newCollection=true&collection=myalias_2017-10-23&version=2&replicaType=NRT&coreNodeName=core_node3&property.timePartitionAliasName=myalias&name=myalias_2017-10-23_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin
   [junit4]   2> 1509392 INFO  (qtp892038354-8749) [n:127.0.0.1:37758_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1509409 INFO  (qtp892038354-8750) [n:127.0.0.1:37758_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=timeConfig&newCollection=true&collection=myalias_2017-10-23&version=2&replicaType=NRT&coreNodeName=core_node7&property.timePartitionAliasName=myalias&name=myalias_2017-10-23_shard2_replica_n4&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 1509410 INFO  (qtp849971433-8763) [n:127.0.0.1:36154_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=timeConfig&newCollection=true&collection=myalias_2017-10-23&version=2&replicaType=NRT&coreNodeName=core_node8&property.timePartitionAliasName=myalias&name=myalias_2017-10-23_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin
   [junit4]   2> 1509512 INFO  (zkCallback-1308-thread-1-processing-n:127.0.0.1:36154_solr) [n:127.0.0.1:36154_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/myalias_2017-10-23/state.json] for collection [myalias_2017-10-23] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1509512 INFO  (zkCallback-1308-thread-2-processing-n:127.0.0.1:36154_solr) [n:127.0.0.1:36154_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/myalias_2017-10-23/state.json] for collection [myalias_2017-10-23] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1509513 INFO  (zkCallback-1310-thread-1-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/myalias_2017-10-23/state.json] for collection [myalias_2017-10-23] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1509513 INFO  (zkCallback-1310-thread-2-processing-n:127.0.0.1:37758_solr) [n:127.0.0.1:37758_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/myalias_2017-10-23/state.json] for collection [myalias_2017-10-23] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1510403 INFO  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:myalias_2017-10-23 s:shard1 r:core_node5 x:myalias_2017-10-23_shard1_replica_n2] o.a.s.c.RequestParams conf resource params.json loaded . version : 1
   [junit4]   2> 1510403 INFO  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:myalias_2017-10-23 s:shard1 r:core_node5 x:myalias_2017-10-23_shard1_replica_n2] o.a.s.c.RequestParams request params refreshed to version 1
   [junit4]   2> 1510404 WARN  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:myalias_2017-10-23 s:shard1 r:core_node5 x:myalias_2017-10-23_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/myalias_2017-10-23_shard1_replica_n2/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/myalias_2017-10-23_shard1_replica_n2/../../../../contrib/extraction/lib
   [junit4]   2> 1510404 WARN  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:myalias_2017-10-23 s:shard1 r:core_node5 x:myalias_2017-10-23_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/myalias_2017-10-23_shard1_replica_n2/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/myalias_2017-10-23_shard1_replica_n2/../../../../dist
   [junit4]   2> 1510404 WARN  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:myalias_2017-10-23 s:shard1 r:core_node5 x:myalias_2017-10-23_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/myalias_2017-10-23_shard1_replica_n2/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/myalias_2017-10-23_shard1_replica_n2/../../../../contrib/clustering/lib
   [junit4]   2> 1510404 WARN  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:myalias_2017-10-23 s:shard1 r:core_node5 x:myalias_2017-10-23_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/myalias_2017-10-23_shard1_replica_n2/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/myalias_2017-10-23_shard1_replica_n2/../../../../dist
   [junit4]   2> 1510404 WARN  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:myalias_2017-10-23 s:shard1 r:core_node5 x:myalias_2017-10-23_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/myalias_2017-10-23_shard1_replica_n2/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/myalias_2017-10-23_shard1_replica_n2/../../../../contrib/langid/lib
   [junit4]   2> 1510405 WARN  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:myalias_2017-10-23 s:shard1 r:core_node5 x:myalias_2017-10-23_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/myalias_2017-10-23_shard1_replica_n2/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/myalias_2017-10-23_shard1_replica_n2/../../../../dist
   [junit4]   2> 1510405 WARN  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:myalias_2017-10-23 s:shard1 r:core_node5 x:myalias_2017-10-23_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/myalias_2017-10-23_shard1_replica_n2/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/myalias_2017-10-23_shard1_replica_n2/../../../../contrib/velocity/lib
   [junit4]   2> 1510405 WARN  (qtp849971433-8759) [n:127.0.0.1:36154_solr c:myalias_2017-10-23 s:shard1 r:core_node5 x:myalias_2017-10-23_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/myalias_2017-10-23_shard1_replica_n2/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node2/myalias_2017-10-23_shard1_replica_n2/../../../../dist
   [junit4]   2> 1510406 INFO  (qtp892038354-8749) [n:127.0.0.1:37758_solr c:myalias_2017-10-23 s:shard1 r:core_node3 x:myalias_2017-10-23_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 1
   [junit4]   2> 1510406 INFO  (qtp892038354-8749) [n:127.0.0.1:37758_solr c:myalias_2017-10-23 s:shard1 r:core_node3 x:myalias_2017-10-23_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 1
   [junit4]   2> 1510406 WARN  (qtp892038354-8749) [n:127.0.0.1:37758_solr c:myalias_2017-10-23 s:shard1 r:core_node3 x:myalias_2017-10-23_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node1/myalias_2017-10-23_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node1/myalias_2017-10-23_shard1_replica_n1/../../../../contrib/extraction/lib
   [junit4]   2> 1510407 WARN  (qtp892038354-8749) [n:127.0.0.1:37758_solr c:myalias_2017-10-23 s:shard1 r:core_node3 x:myalias_2017-10-23_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node1/myalias_2017-10-23_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node1/myalias_2017-10-23_shard1_replica_n1/../../../../dist
   [junit4]   2> 1510407 WARN  (qtp892038354-8749) [n:127.0.0.1:37758_solr c:myalias_2017-10-23 s:shard1 r:core_node3 x:myalias_2017-10-23_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node1/myalias_2017-10-23_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node1/myalias_2017-10-23_shard1_replica_n1/../../../../contrib/clustering/lib
   [junit4]   2> 1510407 WARN  (qtp892038354-8749) [n:127.0.0.1:37758_solr c:myalias_2017-10-23 s:shard1 r:core_node3 x:myalias_2017-10-23_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node1/myalias_2017-10-23_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node1/myalias_2017-10-23_shard1_replica_n1/../../../../dist
   [junit4]   2> 1510407 WARN  (qtp892038354-8749) [n:127.0.0.1:37758_solr c:myalias_2017-10-23 s:shard1 r:core_node3 x:myalias_2017-10-23_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001/tempDir-001/node1/myalias_2017-10-23_shard1_replica_n1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRo

[...truncated too long message...]

) -> (0)
   [junit4]   2> 1524421 INFO  (coreCloseExecutor-2672-thread-1) [n:127.0.0.1:37758_solr c:myalias_2017-10-24 s:shard1 r:core_node2 x:myalias_2017-10-24_shard1_replica_n1] o.a.s.c.SolrCore [myalias_2017-10-24_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@6c2dc0a8
   [junit4]   2> 1524421 INFO  (coreCloseExecutor-2672-thread-1) [n:127.0.0.1:37758_solr c:myalias_2017-10-24 s:shard1 r:core_node2 x:myalias_2017-10-24_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.myalias_2017-10-24.shard1.replica_n1, tag=1814937768
   [junit4]   2> 1524421 INFO  (coreCloseExecutor-2672-thread-1) [n:127.0.0.1:37758_solr c:myalias_2017-10-24 s:shard1 r:core_node2 x:myalias_2017-10-24_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@29c6d3d5: rootName = solr_37758, domain = solr.core.myalias_2017-10-24.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.myalias_2017-10-24.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@6302ea68
   [junit4]   2> 1524422 INFO  (coreCloseExecutor-2672-thread-2) [n:127.0.0.1:37758_solr c:myalias_2017-10-26 s:shard1 r:core_node2 x:myalias_2017-10-26_shard1_replica_n1] o.a.s.c.SolrCore [myalias_2017-10-26_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@6850e9d
   [junit4]   2> 1524442 INFO  (coreCloseExecutor-2671-thread-1) [n:127.0.0.1:36154_solr c:myalias_2017-10-25 s:shard1 r:core_node2 x:myalias_2017-10-25_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.myalias_2017-10-25.shard1.leader, tag=514230075
   [junit4]   2> 1524442 INFO  (coreCloseExecutor-2671-thread-1) [n:127.0.0.1:36154_solr c:myalias_2017-10-25 s:shard1 r:core_node2 x:myalias_2017-10-25_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter close.
   [junit4]   2> 1524443 INFO  (coreCloseExecutor-2671-thread-1) [n:127.0.0.1:36154_solr c:myalias_2017-10-25 s:shard1 r:core_node2 x:myalias_2017-10-25_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@6d058e42 commitCommandVersion:0
   [junit4]   2> 1524463 INFO  (jetty-closer-1293-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@15ea0a01{/solr,null,UNAVAILABLE}
   [junit4]   2> 1524474 INFO  (coreCloseExecutor-2672-thread-1) [n:127.0.0.1:37758_solr c:myalias_2017-10-24 s:shard1 r:core_node2 x:myalias_2017-10-24_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.myalias_2017-10-24.shard1.leader, tag=1814937768
   [junit4]   2> 1524475 INFO  (coreCloseExecutor-2672-thread-2) [n:127.0.0.1:37758_solr c:myalias_2017-10-26 s:shard1 r:core_node2 x:myalias_2017-10-26_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.myalias_2017-10-26.shard1.replica_n1, tag=109383325
   [junit4]   2> 1524475 INFO  (coreCloseExecutor-2672-thread-2) [n:127.0.0.1:37758_solr c:myalias_2017-10-26 s:shard1 r:core_node2 x:myalias_2017-10-26_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@782c1921: rootName = solr_37758, domain = solr.core.myalias_2017-10-26.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.myalias_2017-10-26.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@22acfc8f
   [junit4]   2> 1524475 INFO  (coreCloseExecutor-2672-thread-1) [n:127.0.0.1:37758_solr c:myalias_2017-10-24 s:shard1 r:core_node2 x:myalias_2017-10-24_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter close.
   [junit4]   2> 1524475 INFO  (coreCloseExecutor-2672-thread-1) [n:127.0.0.1:37758_solr c:myalias_2017-10-24 s:shard1 r:core_node2 x:myalias_2017-10-24_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@47b33ec commitCommandVersion:0
   [junit4]   2> 1524482 INFO  (coreCloseExecutor-2672-thread-2) [n:127.0.0.1:37758_solr c:myalias_2017-10-26 s:shard1 r:core_node2 x:myalias_2017-10-26_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.myalias_2017-10-26.shard1.leader, tag=109383325
   [junit4]   2> 1524482 INFO  (coreCloseExecutor-2672-thread-2) [n:127.0.0.1:37758_solr c:myalias_2017-10-26 s:shard1 r:core_node2 x:myalias_2017-10-26_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter close.
   [junit4]   2> 1524482 INFO  (coreCloseExecutor-2672-thread-2) [n:127.0.0.1:37758_solr c:myalias_2017-10-26 s:shard1 r:core_node2 x:myalias_2017-10-26_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@285497ff commitCommandVersion:0
   [junit4]   2> 1524501 INFO  (jetty-closer-1293-thread-2) [    ] o.a.s.c.Overseer Overseer (id=99323945344696326-127.0.0.1:37758_solr-n_0000000000) closing
   [junit4]   2> 1524501 INFO  (OverseerStateUpdate-99323945344696326-127.0.0.1:37758_solr-n_0000000000) [n:127.0.0.1:37758_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37758_solr
   [junit4]   2> 1524504 WARN  (OverseerAutoScalingTriggerThread-99323945344696326-127.0.0.1:37758_solr-n_0000000000) [n:127.0.0.1:37758_solr    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1524521 INFO  (jetty-closer-1293-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@26aa2fdc{/solr,null,UNAVAILABLE}
   [junit4]   2> 1524522 ERROR (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[DF868EAC660D298E]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1524523 INFO  (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[DF868EAC660D298E]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:39218 39218
   [junit4]   2> 1524583 INFO  (Thread-4105) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:39218 39218
   [junit4]   2> 1524584 WARN  (Thread-4105) [    ] o.a.s.c.ZkTestServer Watch limit violations:
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2>
   [junit4]   2> 9 /solr/configs/timeConfig/managed-schema
   [junit4]   2> 6 /solr/configs/timeConfig
   [junit4]   2> 6 /solr/aliases.json
   [junit4]   2> 2 /solr/security.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2>
   [junit4]   2> 14 /solr/collections/myalias_2017-10-23/state.json
   [junit4]   2> 6 /solr/clusterprops.json
   [junit4]   2> 6 /solr/clusterstate.json
   [junit4]   2> 5 /solr/collections/myalias_2017-10-25/state.json
   [junit4]   2> 5 /solr/collections/myalias_2017-10-24/state.json
   [junit4]   2> 5 /solr/collections/myalias_2017-10-26/state.json
   [junit4]   2> 5 /solr/collections/timeConfig/state.json
   [junit4]   2>
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2>
   [junit4]   2> 6 /solr/live_nodes
   [junit4]   2> 6 /solr/collections
   [junit4]   2>
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_DF868EAC660D298E-001
   [junit4]   2> Jan 10, 2018 5:45:51 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1168, maxMBSortInHeap=6.514598185947125, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@39f418ef), locale=de-GR, timezone=Africa/Lome
   [junit4]   2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=132661568,total=526385152
   [junit4]   2> NOTE: All tests run in this JVM: [LeaderInitiatedRecoveryOnCommitTest, AnalyticsQueryTest, CustomHighlightComponentTest, TestSubQueryTransformerCrossCore, LeaderElectionContextKeyTest, HdfsRecoveryZkTest, TestOverriddenPrefixQueryForCustomFieldType, TestSolrCloudWithHadoopAuthPlugin, OverseerStatusTest, DistanceUnitsTest, DocValuesTest, OverseerCollectionConfigSetProcessorTest, SoftAutoCommitTest, JavabinLoaderTest, DistributedQueryElevationComponentTest, DistributedFacetExistsSmallTest, PreAnalyzedFieldManagedSchemaCloudTest, BasicFunctionalityTest, TestPointFields, TestAnalyzeInfixSuggestions, MinimalSchemaTest, HdfsWriteToMultipleCollectionsTest, TestFreeTextSuggestions, HttpTriggerListenerTest, TestMissingGroups, DataDrivenBlockJoinTest, HdfsCollectionsAPIDistributedZkTest, CheckHdfsIndexTest, BigEndianAscendingWordDeserializerTest, TestLegacyNumericUtils, TestFoldingMultitermQuery, MetricsHandlerTest, SpellCheckCollatorTest, TestScoreJoinQPNoScore, PeerSyncTest, ClassificationUpdateProcessorTest, TestHttpShardHandlerFactory, TestComplexPhraseQParserPlugin, TestDistribDocBasedVersion, DirectUpdateHandlerOptimizeTest, DeleteInactiveReplicaTest, TestSchemaSimilarityResource, GraphQueryTest, TestFieldCacheVsDocValues, SimpleFacetsTest, SliceStateTest, DistribCursorPagingTest, DistributedQueryComponentCustomSortTest, TestEmbeddedSolrServerSchemaAPI, TestConfig, TestRandomRequestDistribution, ResourceLoaderTest, DistributedQueueTest, RequestLoggingTest, TestStreamBody, TestComputePlanAction, TestManagedSchema, TestConfigOverlay, FastVectorHighlighterTest, SubstringBytesRefFilterTest, SyncSliceTest, TestRecovery, BlockDirectoryTest, TestTolerantUpdateProcessorRandomCloud, CoreAdminRequestStatusTest, TestSolrConfigHandlerConcurrent, TestInitParams, ResponseHeaderTest, TimeRoutedAliasUpdateProcessorTest]
   [junit4] Completed [221/767 (1!)] on J0 in 21.36s, 2 tests, 1 error <<< FAILURES!

[...truncated 50012 lines...]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

[JENKINS] Lucene-Solr-Tests-master - Build # 2253 - Still Unstable

Apache Jenkins Server-2
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2253/

2 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode

Error Message:


Stack Trace:
java.util.concurrent.TimeoutException
        at __randomizedtesting.SeedInfo.seed([323C1DF3FA62646D:95D30050352FEB75]:0)
        at org.apache.solr.cloud.autoscaling.sim.SimSolrCloudTestCase.waitForState(SimSolrCloudTestCase.java:271)
        at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode(TestLargeCluster.java:234)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest.testParse

Error Message:
Error from server at http://127.0.0.1:42939/solr: Collection : myalias_2017-10-24 is part of alias myalias remove or modify the alias before removing this collection.

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42939/solr: Collection : myalias_2017-10-24 is part of alias myalias remove or modify the alias before removing this collection.
        at __randomizedtesting.SeedInfo.seed([323C1DF3FA62646D:5DB956AB46145782]:0)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
        at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
        at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
        at org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest.doBefore(TimeRoutedAliasUpdateProcessorTest.java:84)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:968)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13138 lines...]
   [junit4] Suite: org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest
   [junit4]   2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_323C1DF3FA62646D-001/init-core-data-001
   [junit4]   2> 2796286 WARN  (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[323C1DF3FA62646D]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=30 numCloses=30
   [junit4]   2> 2796286 INFO  (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[323C1DF3FA62646D]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 2796287 INFO  (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[323C1DF3FA62646D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 2796287 INFO  (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[323C1DF3FA62646D]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 2796287 INFO  (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[323C1DF3FA62646D]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_323C1DF3FA62646D-001/tempDir-001
   [junit4]   2> 2796288 INFO  (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[323C1DF3FA62646D]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2796288 INFO  (Thread-1848) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2796288 INFO  (Thread-1848) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2796290 ERROR (Thread-1848) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2796394 INFO  (SUITE-TimeRoutedAliasUpdateProcessorTest-seed#[323C1DF3FA62646D]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:41341
   [junit4]   2> 2796400 INFO  (zkConnectionManagerCallback-1577-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2796451 INFO  (jetty-launcher-1574-thread-2) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 2796467 INFO  (jetty-launcher-1574-thread-1) [    ] o.e.j.s.Server jetty-9.3.20.v20170531
   [junit4]   2> 2796508 INFO  (jetty-launcher-1574-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1541851e{/solr,null,AVAILABLE}
   [junit4]   2> 2796525 INFO  (jetty-launcher-1574-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@331169d3{HTTP/1.1,[http/1.1]}{127.0.0.1:39650}
   [junit4]   2> 2796526 INFO  (jetty-launcher-1574-thread-2) [    ] o.e.j.s.Server Started @2806847ms
   [junit4]   2> 2796526 INFO  (jetty-launcher-1574-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39650}
   [junit4]   2> 2796526 ERROR (jetty-launcher-1574-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2796526 INFO  (jetty-launcher-1574-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 2796526 INFO  (jetty-launcher-1574-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2796526 INFO  (jetty-launcher-1574-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2796526 INFO  (jetty-launcher-1574-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-11T12:43:42.218Z
   [junit4]   2> 2796545 INFO  (jetty-launcher-1574-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2384572a{/solr,null,AVAILABLE}
   [junit4]   2> 2796545 INFO  (jetty-launcher-1574-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@66011214{HTTP/1.1,[http/1.1]}{127.0.0.1:42939}
   [junit4]   2> 2796545 INFO  (jetty-launcher-1574-thread-1) [    ] o.e.j.s.Server Started @2806866ms
   [junit4]   2> 2796545 INFO  (jetty-launcher-1574-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42939}
   [junit4]   2> 2796545 ERROR (jetty-launcher-1574-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2796545 INFO  (jetty-launcher-1574-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 2796545 INFO  (jetty-launcher-1574-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2796545 INFO  (jetty-launcher-1574-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2796545 INFO  (jetty-launcher-1574-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-01-11T12:43:42.237Z
   [junit4]   2> 2796620 INFO  (zkConnectionManagerCallback-1579-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2796620 INFO  (zkConnectionManagerCallback-1581-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2796620 INFO  (jetty-launcher-1574-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2796620 INFO  (jetty-launcher-1574-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2796644 INFO  (jetty-launcher-1574-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41341/solr
   [junit4]   2> 2796644 INFO  (jetty-launcher-1574-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41341/solr
   [junit4]   2> 2796659 INFO  (zkConnectionManagerCallback-1589-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2796660 INFO  (zkConnectionManagerCallback-1586-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2796661 INFO  (zkConnectionManagerCallback-1591-thread-1-processing-n:127.0.0.1:42939_solr) [n:127.0.0.1:42939_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2796689 INFO  (zkConnectionManagerCallback-1594-thread-1-processing-n:127.0.0.1:39650_solr) [n:127.0.0.1:39650_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2796758 INFO  (jetty-launcher-1574-thread-2) [n:127.0.0.1:39650_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2796759 INFO  (jetty-launcher-1574-thread-2) [n:127.0.0.1:39650_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39650_solr
   [junit4]   2> 2796784 INFO  (jetty-launcher-1574-thread-2) [n:127.0.0.1:39650_solr    ] o.a.s.c.Overseer Overseer (id=99331252026146822-127.0.0.1:39650_solr-n_0000000000) starting
   [junit4]   2> 2796800 INFO  (jetty-launcher-1574-thread-2) [n:127.0.0.1:39650_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39650_solr
   [junit4]   2> 2796803 INFO  (OverseerStateUpdate-99331252026146822-127.0.0.1:39650_solr-n_0000000000) [n:127.0.0.1:39650_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2796929 INFO  (jetty-launcher-1574-thread-1) [n:127.0.0.1:42939_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2796939 INFO  (jetty-launcher-1574-thread-1) [n:127.0.0.1:42939_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2796941 INFO  (jetty-launcher-1574-thread-1) [n:127.0.0.1:42939_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42939_solr
   [junit4]   2> 2796962 INFO  (zkCallback-1590-thread-1-processing-n:127.0.0.1:42939_solr) [n:127.0.0.1:42939_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2796962 INFO  (zkCallback-1593-thread-1-processing-n:127.0.0.1:39650_solr) [n:127.0.0.1:39650_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2797056 INFO  (jetty-launcher-1574-thread-2) [n:127.0.0.1:39650_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39650.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6411c731
   [junit4]   2> 2797059 INFO  (jetty-launcher-1574-thread-1) [n:127.0.0.1:42939_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42939.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6411c731
   [junit4]   2> 2797074 INFO  (jetty-launcher-1574-thread-1) [n:127.0.0.1:42939_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42939.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6411c731
   [junit4]   2> 2797074 INFO  (jetty-launcher-1574-thread-1) [n:127.0.0.1:42939_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42939.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6411c731
   [junit4]   2> 2797075 INFO  (jetty-launcher-1574-thread-2) [n:127.0.0.1:39650_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39650.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6411c731
   [junit4]   2> 2797075 INFO  (jetty-launcher-1574-thread-2) [n:127.0.0.1:39650_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39650.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6411c731
   [junit4]   2> 2797075 INFO  (jetty-launcher-1574-thread-1) [n:127.0.0.1:42939_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_323C1DF3FA62646D-001/tempDir-001/node2/.
   [junit4]   2> 2797076 INFO  (jetty-launcher-1574-thread-2) [n:127.0.0.1:39650_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.update.processor.TimeRoutedAliasUpdateProcessorTest_323C1DF3FA62646D-001/tempDir-001/node1/.
   [junit4]   2> 2797163 INFO  (zkConnectionManagerCallback-1599-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2797199 INFO  (TEST-TimeRoutedAliasUpdateProcessorTest.test-seed#[323C1DF3FA62646D]) [    ] o.a.s.SolrTestCaseJ4 ###Starting test
   [junit4]   2> 2797202 INFO  (zkConnectionManagerCallback-1604-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2797203 INFO  (TEST-TimeRoutedAliasUpdateProcessorTest.test-seed#[323C1DF3FA62646D]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2797204 INFO  (TEST-TimeRoutedAliasUpdateProcessorTest.test-seed#[323C1DF3FA62646D]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41341/solr ready
   [junit4]   2> 2797226 INFO  (qtp686708400-13029) [n:127.0.0.1:42939_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2797226 INFO  (qtp686708400-13029) [n:127.0.0.1:42939_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2797226 INFO  (qtp686708400-13030) [n:127.0.0.1:42939_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=1&name=timeConfig&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2797228 INFO  (OverseerThreadFactory-4294-thread-1-processing-n:127.0.0.1:39650_solr) [n:127.0.0.1:39650_solr    ] o.a.s.c.CreateCollectionCmd Create collection timeConfig
   [junit4]   2> 2797229 INFO  (OverseerThreadFactory-4294-thread-1-processing-n:127.0.0.1:39650_solr) [n:127.0.0.1:39650_solr    ] o.a.s.c.CreateCollectionCmd Only _default config set found, using it.
   [junit4]   2> 2797229 INFO  (OverseerThreadFactory-4294-thread-1-processing-n:127.0.0.1:39650_solr) [n:127.0.0.1:39650_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/protwords.txt to /configs/timeConfig/protwords.txt
   [junit4]   2> 2797232 INFO  (OverseerThreadFactory-4294-thread-1-processing-n:127.0.0.1:39650_solr) [n:127.0.0.1:39650_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/managed-schema to /configs/timeConfig/managed-schema
   [junit4]   2> 2797246 INFO  (OverseerThreadFactory-4294-thread-1-processing-n:127.0.0.1:39650_solr) [n:127.0.0.1:39650_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/solrconfig.xml to /configs/timeConfig/solrconfig.xml
   [junit4]   2> 2797248 INFO  (OverseerThreadFactory-4294-thread-1-processing-n:127.0.0.1:39650_solr) [n:127.0.0.1:39650_solr    ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/synonyms.txt to /configs/timeConfig/synonyms.txt
   [junit4]   2> 2797250 INFO  (OverseerThreadFactory-4294-thread-1-processing-n:127.0.0.1:39650_solr) [n:127.0.0.1:39650_solr    ] o.a.s.c.c.ZkConfi